From patchwork Mon Sep 30 18:54:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167059 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1C5B113BD for ; Mon, 30 Sep 2019 18:57:29 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 049CB224D5 for ; Mon, 30 Sep 2019 18:57:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 049CB224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 92CFF5C3B70; Mon, 30 Sep 2019 11:57:16 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id DB6925C3165 for ; Mon, 30 Sep 2019 11:56:58 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 2C03B100536B; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 23265B4; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:20 -0400 Message-Id: <1569869810-23848-2-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 001/151] lnet: fix needed headers for lnet headers X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" The linux kernel now has a sanity check for the global headers, activated by CONFIG_HEADER_TEST=y and CONFIG_KERNEL_HEADER_TEST=y. Some LNet headers are missing a few headers to allow them to build properly. Signed-off-by: James Simmons Signed-off-by: NeilBrown Acked-by: James Simmons --- include/linux/libcfs/libcfs_fail.h | 1 + include/linux/libcfs/libcfs_private.h | 3 +++ include/uapi/linux/lnet/lnetst.h | 2 ++ 3 files changed, 6 insertions(+) diff --git a/include/linux/libcfs/libcfs_fail.h b/include/linux/libcfs/libcfs_fail.h index 0e73eef..f52a82a 100644 --- a/include/linux/libcfs/libcfs_fail.h +++ b/include/linux/libcfs/libcfs_fail.h @@ -35,6 +35,7 @@ #include #include +#include extern unsigned long cfs_fail_loc; extern unsigned int cfs_fail_val; diff --git a/include/linux/libcfs/libcfs_private.h b/include/linux/libcfs/libcfs_private.h index 515fd63..2e8cff6 100644 --- a/include/linux/libcfs/libcfs_private.h +++ b/include/linux/libcfs/libcfs_private.h @@ -39,6 +39,9 @@ #ifndef __LIBCFS_PRIVATE_H__ #define __LIBCFS_PRIVATE_H__ +#include +#include + #ifndef DEBUG_SUBSYSTEM # define DEBUG_SUBSYSTEM S_UNDEFINED #endif diff --git a/include/uapi/linux/lnet/lnetst.h b/include/uapi/linux/lnet/lnetst.h index 27fcd91..449c5e6 100644 --- a/include/uapi/linux/lnet/lnetst.h +++ b/include/uapi/linux/lnet/lnetst.h @@ -38,6 +38,8 @@ #define __LNET_ST_H__ #include +#include +#include #define LST_FEAT_NONE (0) #define LST_FEAT_BULK_LEN (1 << 0) /* enable variable page size */ From patchwork Mon Sep 30 18:54:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167045 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DEC6B16B1 for ; Mon, 30 Sep 2019 18:57:02 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C6E1F224D5 for ; Mon, 30 Sep 2019 18:57:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C6E1F224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 6D0135C344D; Mon, 30 Sep 2019 11:57:01 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 28D835C3165 for ; Mon, 30 Sep 2019 11:56:59 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 2DC0C100536C; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 260FBB5; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:21 -0400 Message-Id: <1569869810-23848-3-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 002/151] lustre: fix signal handling in abortable waits. X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: NeilBrown Instead of blocking all but the fatal signals, we were blocking just the fatal signals. That wasn't what was intended, but is easy to fix. Signed-off-by: NeilBrown Signed-off-by: James Simmons --- fs/lustre/include/lustre_lib.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/fs/lustre/include/lustre_lib.h b/fs/lustre/include/lustre_lib.h index da86e46..64d2e91 100644 --- a/fs/lustre/include/lustre_lib.h +++ b/fs/lustre/include/lustre_lib.h @@ -93,7 +93,7 @@ static inline int l_fatal_signal_pending(struct task_struct *p) ({ \ sigset_t __new_blocked, __old_blocked; \ int __ret = 0; \ - siginitset(&__new_blocked, LUSTRE_FATAL_SIGS); \ + siginitsetinv(&__new_blocked, LUSTRE_FATAL_SIGS); \ sigprocmask(SIG_BLOCK, &__new_blocked, &__old_blocked); \ __ret = wait_event_interruptible(wq, condition); \ sigprocmask(SIG_SETMASK, &__old_blocked, NULL); \ @@ -104,7 +104,7 @@ static inline int l_fatal_signal_pending(struct task_struct *p) ({ \ sigset_t __new_blocked, __old_blocked; \ int __ret = 0; \ - siginitset(&__new_blocked, LUSTRE_FATAL_SIGS); \ + siginitsetinv(&__new_blocked, LUSTRE_FATAL_SIGS); \ sigprocmask(SIG_BLOCK, &__new_blocked, &__old_blocked); \ __ret = wait_event_interruptible_timeout(wq, condition, timeout);\ sigprocmask(SIG_SETMASK, &__old_blocked, NULL); \ @@ -115,7 +115,7 @@ static inline int l_fatal_signal_pending(struct task_struct *p) ({ \ sigset_t __new_blocked, __old_blocked; \ int __ret = 0; \ - siginitset(&__new_blocked, LUSTRE_FATAL_SIGS); \ + siginitsetinv(&__new_blocked, LUSTRE_FATAL_SIGS); \ sigprocmask(SIG_BLOCK, &__new_blocked, &__old_blocked); \ __ret = wait_event_interruptible_exclusive(wq, condition); \ sigprocmask(SIG_SETMASK, &__old_blocked, NULL); \ From patchwork Mon Sep 30 18:54:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167069 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 56B0D1902 for ; Mon, 30 Sep 2019 18:57:50 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3A7B5224D5 for ; Mon, 30 Sep 2019 18:57:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3A7B5224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 020CC5C3C31; Mon, 30 Sep 2019 11:57:26 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 698A75C3165 for ; Mon, 30 Sep 2019 11:56:59 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 2F34F100536D; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 29BCCBB; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:22 -0400 Message-Id: <1569869810-23848-4-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 003/151] lnet: ksocklnd: add secondary IP address handling X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: James Simmons , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" With the current ksocknal_enumerate_interfaces() only primary IP addresses are returned. This disables using network aliasing which some people use. Remove the secondary IP address filtering. Update the string handling since ifa_device names can be different than the net_device name. Discard the 'j' counter and instead keep ksnn_ninterfaces up to date. This means that we return 0 on success, rather than a count of added interfaces. Update the too many interfaces test with a better test using ARRAY_SIZE. WC-bug-id: https://jira.whamcloud.com/browse/LU-11893 Lustre-commit: 9a2013af0668 ("LU-11893 ksocklnd: add secondary IP address handling Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/34392 Reviewed-by: Petros Koutoupis Reviewed-by: Neil Brown Reviewed-by: Amir Shehata Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/klnds/socklnd/socklnd.c | 75 +++++++++++++++++++++++++++------------- 1 file changed, 51 insertions(+), 24 deletions(-) diff --git a/net/lnet/klnds/socklnd/socklnd.c b/net/lnet/klnds/socklnd/socklnd.c index ccc42f6..1c957dc 100644 --- a/net/lnet/klnds/socklnd/socklnd.c +++ b/net/lnet/klnds/socklnd/socklnd.c @@ -2604,54 +2604,58 @@ static int ksocknal_push(struct lnet_ni *ni, struct lnet_process_id id) static int ksocknal_enumerate_interfaces(struct ksock_net *net, char *iname) { - int j = 0; struct net_device *dev; rtnl_lock(); for_each_netdev(&init_net, dev) { - const char *name = dev->name; + /* The iname specified by a user land configuration can + * map to an ifa_label so always treat iname as an ifa_label. + * If iname is NULL then fall back to the net device name. + */ + const char *name = iname ? iname : dev->name; const struct in_ifaddr *ifa; struct in_device *in_dev; - struct ksock_interface *ksi = - &net->ksnn_interfaces[net->ksnn_ninterfaces + j]; - if (strcmp(name, "lo") == 0) /* skip the loopback IF */ - continue; - if (iname && strcmp(name, iname) != 0) + if (strcmp(dev->name, "lo") == 0) /* skip the loopback IF */ continue; if (!(dev_get_flags(dev) & IFF_UP)) { - CWARN("Ignoring interface %s (down)\n", name); + CWARN("Ignoring interface %s (down)\n", dev->name); continue; } - if (j == LNET_INTERFACES_NUM) { - CWARN("Ignoring interface %s (too many interfaces)\n", - name); - continue; - } in_dev = __in_dev_get_rtnl(dev); if (!in_dev) { - CWARN("Interface %s has no IPv4 status.\n", name); + CWARN("Interface %s has no IPv4 status.\n", dev->name); continue; } - in_dev_for_each_ifa_rcu(ifa, in_dev) - if (!(ifa->ifa_flags & IFA_F_SECONDARY) && - strcmp(ifa->ifa_label, name) == 0) { + + in_dev_for_each_ifa_rcu(ifa, in_dev) { + if (strcmp(name, ifa->ifa_label) == 0) { + int idx = net->ksnn_ninterfaces; + struct ksock_interface *ksi; + + if (idx >= ARRAY_SIZE(net->ksnn_interfaces)) { + rtnl_unlock(); + return -E2BIG; + } + + ksi = &net->ksnn_interfaces[idx]; ksi->ksni_ipaddr = ntohl(ifa->ifa_local); ksi->ksni_netmask = ifa->ifa_mask; strlcpy(ksi->ksni_name, name, sizeof(ksi->ksni_name)); - j++; + net->ksnn_ninterfaces++; break; } + } } rtnl_unlock(); - if (!iname && !j) + if (net->ksnn_ninterfaces == 0) CERROR("Can't find any usable interfaces\n"); - return j; + return net->ksnn_ninterfaces > 0 ? 0 : -ENOENT; } static int @@ -2810,17 +2814,40 @@ static int ksocknal_push(struct lnet_ni *ni, struct lnet_process_id id) rc = ksocknal_enumerate_interfaces(net, NULL); if (rc <= 0) goto fail_1; - - net->ksnn_ninterfaces = rc; } else { + /* Before Multi-Rail ksocklnd would manage + * multiple interfaces with its own tcp bonding. + * If we encounter an old configuration using + * this tcp bonding approach then we need to + * handle more than one ni_interfaces. + * + * In Multi-Rail configuration only ONE ni_interface + * should exist. Each IP alias should be mapped to + * each 'struct net_ni'. + */ for (i = 0; i < LNET_INTERFACES_NUM; i++) { + int j; + if (!ni->ni_interfaces[i]) break; - rc = ksocknal_enumerate_interfaces(net, ni->ni_interfaces[i]); + for (j = 0; j < net->ksnn_ninterfaces; j++) { + struct ksock_interface *ksi; + + ksi = &net->ksnn_interfaces[j]; + + if (strcmp(ni->ni_interfaces[i], + ksi->ksni_name) == 0) { + CERROR("found duplicate %s\n", + ksi->ksni_name); + rc = -EEXIST; + goto fail_1; + } + } + + rc = ksocknal_enumerate_interfaces(net, ni->ni_interfaces[i]); if (rc <= 0) goto fail_1; - net->ksnn_ninterfaces += rc; } } From patchwork Mon Sep 30 18:54:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167047 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E229516B1 for ; Mon, 30 Sep 2019 18:57:07 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C9D89224D5 for ; Mon, 30 Sep 2019 18:57:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C9D89224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id EA6525C340D; Mon, 30 Sep 2019 11:57:04 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id C129A5C3165 for ; Mon, 30 Sep 2019 11:56:59 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 31B37100537B; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 2CC36BD; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:23 -0400 Message-Id: <1569869810-23848-5-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 004/151] lnet: o2iblnd: add secondary IP address handling X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: James Simmons , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" Using dev_get_by_name() in kiblnd_create_dev() means we can only discover primary IP addresses. This breaks using network aliasing which some people use. Move away from dev_get_by_name() to using for_ifa() so we can detect any secondary IP addresses. WC-bug-id: https://jira.whamcloud.com/browse/LU-11893 Linux-commit: c4b39bf56bbc ("LU-11893 o2iblnd: add secondary IP address handling") Reviewed-on: https://review.whamcloud.com/34476 WC-bug-id: https://jira.whamcloud.com/browse/LU-12381 Linux-commit: 1dea5aac9d9b ("LU-11893 o2iblnd: ignore down interfaces") Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/35098 Reviewed-by: Petros Koutoupis Reviewed-by: Neil Brown Reviewed-by: Chris Horn Reviewed-by: Andreas Dilger Reviewed-by: Amir Shehata Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/klnds/o2iblnd/o2iblnd.c | 78 ++++++++++++++++++++++------------------ 1 file changed, 44 insertions(+), 34 deletions(-) diff --git a/net/lnet/klnds/o2iblnd/o2iblnd.c b/net/lnet/klnds/o2iblnd/o2iblnd.c index 97ed1ec..e952c0c 100644 --- a/net/lnet/klnds/o2iblnd/o2iblnd.c +++ b/net/lnet/klnds/o2iblnd/o2iblnd.c @@ -2557,53 +2557,58 @@ static struct kib_dev *kiblnd_create_dev(char *ifname) { const struct in_ifaddr *ifa; struct net_device *netdev; - struct in_device *in_dev; - struct kib_dev *dev; + struct kib_dev *dev = NULL; int flags; int rc; rtnl_lock(); - netdev = dev_get_by_name(&init_net, ifname); - if (!netdev) { - CERROR("Can't find IPoIB interface %s\n", - ifname); - goto unlock; - } + for_each_netdev(&init_net, netdev) { + struct in_device *in_dev; - flags = dev_get_flags(netdev); - if (!(flags & IFF_UP)) { - CERROR("Can't query IPoIB interface %s: it's down\n", ifname); - goto unlock; - } + if (strcmp(netdev->name, "lo") == 0) /* skip the loopback IF */ + continue; - dev = kzalloc(sizeof(*dev), GFP_NOFS); - if (!dev) - goto unlock; + flags = dev_get_flags(netdev); + if (!(flags & IFF_UP)) { + CWARN("Can't query IPoIB interface %s: it's down\n", + netdev->name); + continue; + } - dev->ibd_can_failover = !!(flags & IFF_MASTER); + in_dev = __in_dev_get_rtnl(netdev); + if (!in_dev) { + CWARN("Interface %s has no IPv4 status.\n", + netdev->name); + continue; + } - INIT_LIST_HEAD(&dev->ibd_nets); - INIT_LIST_HEAD(&dev->ibd_list); /* not yet in kib_devs */ - INIT_LIST_HEAD(&dev->ibd_fail_list); + in_dev_for_each_ifa_rcu(ifa, in_dev) { + if (strcmp(ifname, ifa->ifa_label) == 0) { + dev = kzalloc(sizeof(*dev), GFP_NOFS); + if (!dev) + goto unlock; - in_dev = __in_dev_get_rtnl(netdev); - if (!in_dev) { - kfree(dev); - goto unlock; - } + dev->ibd_can_failover = !!(flags & IFF_MASTER); + dev->ibd_ifip = ntohl(ifa->ifa_local); - in_dev_for_each_ifa_rcu(ifa, in_dev) - if (!(ifa->ifa_flags & IFA_F_SECONDARY) && - strcmp(ifa->ifa_label, ifname) == 0) { - dev->ibd_ifip = ntohl(ifa->ifa_local); - break; + INIT_LIST_HEAD(&dev->ibd_nets); + /* not yet in kib_devs */ + INIT_LIST_HEAD(&dev->ibd_list); + INIT_LIST_HEAD(&dev->ibd_fail_list); + break; + } } + } rtnl_unlock(); + if (!dev) { + CERROR("Can't find any usable interfaces\n"); + return NULL; + } + if (dev->ibd_ifip == 0) { CERROR("Can't initialize device: no IP address\n"); - kfree(dev); - return NULL; + goto free_dev; } strcpy(&dev->ibd_ifname[0], ifname); @@ -2611,14 +2616,15 @@ static struct kib_dev *kiblnd_create_dev(char *ifname) rc = kiblnd_dev_failover(dev); if (rc) { CERROR("Can't initialize device: %d\n", rc); - kfree(dev); - return NULL; + goto free_dev; } list_add_tail(&dev->ibd_list, &kiblnd_data.kib_devs); return dev; unlock: rtnl_unlock(); +free_dev: + kfree(dev); return NULL; } @@ -2968,6 +2974,10 @@ static int kiblnd_startup(struct lnet_ni *ni) if (rc) goto net_failed; + /* ni_interfaces is only to support legacy pre Multi-Rail + * tcp bonding for ksocklnd. Multi-Rail wants each secondary + * IP to be treated as an unique 'struct ni' interfaces instead. + */ if (ni->ni_interfaces[0]) { /* Use the IPoIB interface specified in 'networks=' */ From patchwork Mon Sep 30 18:54:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167105 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5A1C716B1 for ; Mon, 30 Sep 2019 18:58:56 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 427DA224D5 for ; Mon, 30 Sep 2019 18:58:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 427DA224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8F86E5C3E3C; Mon, 30 Sep 2019 11:58:03 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id B783D5C338D for ; Mon, 30 Sep 2019 11:57:00 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 396DB100537D; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 2FD05BE; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:24 -0400 Message-Id: <1569869810-23848-6-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 005/151] lnet: consoldate secondary IP address handling X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: James Simmons , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" The last piece of code with broken secondary IP address support is lnet_parse_ip2nets(). We could fix it like o2iblnd or socklnd was done but since the LND drivers resolved those issues instead we can move the handling out of the LND drivers into one place in the LNet core. To do this we introduce struct lnet_inetdev which is a collection of data that the current LNet layer requires. The new function lnet_inet_enumerate() is used to collect this information. WC-bug-id: https://jira.whamcloud.com/browse/LU-11893 Linux-commit: b770d7117f35a ("LU-11893 lnet: consoldate secondary IP address handling") Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/34993 Reviewed-by: Olaf Weber Reviewed-by: Amir Shehata Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- include/linux/lnet/lib-lnet.h | 10 +++ net/lnet/klnds/o2iblnd/o2iblnd.c | 167 ++++++++++----------------------------- net/lnet/klnds/socklnd/socklnd.c | 121 +++++++++++----------------- net/lnet/lnet/config.c | 121 ++++++++++++++++------------ 4 files changed, 170 insertions(+), 249 deletions(-) diff --git a/include/linux/lnet/lib-lnet.h b/include/linux/lnet/lib-lnet.h index e60c446..776896a 100644 --- a/include/linux/lnet/lib-lnet.h +++ b/include/linux/lnet/lib-lnet.h @@ -36,6 +36,7 @@ #ifndef __LNET_LIB_LNET_H__ #define __LNET_LIB_LNET_H__ +#include #include #include #include @@ -649,6 +650,15 @@ void lnet_connect_console_error(int rc, lnet_nid_t peer_nid, int lnet_acceptor_start(void); void lnet_acceptor_stop(void); +struct lnet_inetdev { + u32 li_cpt; + u32 li_flags; + u32 li_ipaddr; + u32 li_netmask; + char li_name[IFNAMSIZ]; +}; + +int lnet_inet_enumerate(struct lnet_inetdev **dev_list); int lnet_sock_setbuf(struct socket *socket, int txbufsize, int rxbufsize); int lnet_sock_getbuf(struct socket *socket, int *txbufsize, int *rxbufsize); int lnet_sock_getaddr(struct socket *socket, bool remote, u32 *ip, int *port); diff --git a/net/lnet/klnds/o2iblnd/o2iblnd.c b/net/lnet/klnds/o2iblnd/o2iblnd.c index e952c0c..d780108 100644 --- a/net/lnet/klnds/o2iblnd/o2iblnd.c +++ b/net/lnet/klnds/o2iblnd/o2iblnd.c @@ -2553,81 +2553,6 @@ void kiblnd_destroy_dev(struct kib_dev *dev) kfree(dev); } -static struct kib_dev *kiblnd_create_dev(char *ifname) -{ - const struct in_ifaddr *ifa; - struct net_device *netdev; - struct kib_dev *dev = NULL; - int flags; - int rc; - - rtnl_lock(); - for_each_netdev(&init_net, netdev) { - struct in_device *in_dev; - - if (strcmp(netdev->name, "lo") == 0) /* skip the loopback IF */ - continue; - - flags = dev_get_flags(netdev); - if (!(flags & IFF_UP)) { - CWARN("Can't query IPoIB interface %s: it's down\n", - netdev->name); - continue; - } - - in_dev = __in_dev_get_rtnl(netdev); - if (!in_dev) { - CWARN("Interface %s has no IPv4 status.\n", - netdev->name); - continue; - } - - in_dev_for_each_ifa_rcu(ifa, in_dev) { - if (strcmp(ifname, ifa->ifa_label) == 0) { - dev = kzalloc(sizeof(*dev), GFP_NOFS); - if (!dev) - goto unlock; - - dev->ibd_can_failover = !!(flags & IFF_MASTER); - dev->ibd_ifip = ntohl(ifa->ifa_local); - - INIT_LIST_HEAD(&dev->ibd_nets); - /* not yet in kib_devs */ - INIT_LIST_HEAD(&dev->ibd_list); - INIT_LIST_HEAD(&dev->ibd_fail_list); - break; - } - } - } - rtnl_unlock(); - - if (!dev) { - CERROR("Can't find any usable interfaces\n"); - return NULL; - } - - if (dev->ibd_ifip == 0) { - CERROR("Can't initialize device: no IP address\n"); - goto free_dev; - } - strcpy(&dev->ibd_ifname[0], ifname); - - /* initialize the device */ - rc = kiblnd_dev_failover(dev); - if (rc) { - CERROR("Can't initialize device: %d\n", rc); - goto free_dev; - } - - list_add_tail(&dev->ibd_list, &kiblnd_data.kib_devs); - return dev; -unlock: - rtnl_unlock(); -free_dev: - kfree(dev); - return NULL; -} - static void kiblnd_base_shutdown(void) { struct kib_sched_info *sched; @@ -2887,8 +2812,7 @@ static int kiblnd_start_schedulers(struct kib_sched_info *sched) return rc; } -static int kiblnd_dev_start_threads(struct kib_dev *dev, int newdev, u32 *cpts, - int ncpts) +static int kiblnd_dev_start_threads(struct kib_dev *dev, u32 *cpts, int ncpts) { int cpt; int rc; @@ -2900,7 +2824,7 @@ static int kiblnd_dev_start_threads(struct kib_dev *dev, int newdev, u32 *cpts, cpt = !cpts ? i : cpts[i]; sched = kiblnd_data.kib_scheds[cpt]; - if (!newdev && sched->ibs_nthreads > 0) + if (sched->ibs_nthreads > 0) continue; rc = kiblnd_start_schedulers(kiblnd_data.kib_scheds[cpt]); @@ -2913,47 +2837,15 @@ static int kiblnd_dev_start_threads(struct kib_dev *dev, int newdev, u32 *cpts, return 0; } -static struct kib_dev *kiblnd_dev_search(char *ifname) -{ - struct kib_dev *alias = NULL; - struct kib_dev *dev; - char *colon; - char *colon2; - - colon = strchr(ifname, ':'); - list_for_each_entry(dev, &kiblnd_data.kib_devs, ibd_list) { - if (!strcmp(&dev->ibd_ifname[0], ifname)) - return dev; - - if (alias) - continue; - - colon2 = strchr(dev->ibd_ifname, ':'); - if (colon) - *colon = 0; - if (colon2) - *colon2 = 0; - - if (!strcmp(&dev->ibd_ifname[0], ifname)) - alias = dev; - - if (colon) - *colon = ':'; - if (colon2) - *colon2 = ':'; - } - return alias; -} - static int kiblnd_startup(struct lnet_ni *ni) { char *ifname; + struct lnet_inetdev *ifaces = NULL; struct kib_dev *ibdev = NULL; struct kib_net *net; unsigned long flags; int rc; - int newdev; - int node_id; + int i; LASSERT(ni->ni_net->net_lnd == &the_o2iblnd); @@ -2981,9 +2873,8 @@ static int kiblnd_startup(struct lnet_ni *ni) if (ni->ni_interfaces[0]) { /* Use the IPoIB interface specified in 'networks=' */ - BUILD_BUG_ON(LNET_INTERFACES_NUM <= 1); if (ni->ni_interfaces[1]) { - CERROR("Multiple interfaces not supported\n"); + CERROR("ko2iblnd: Multiple interfaces not supported\n"); goto failed; } @@ -2997,24 +2888,51 @@ static int kiblnd_startup(struct lnet_ni *ni) goto failed; } - ibdev = kiblnd_dev_search(ifname); + rc = lnet_inet_enumerate(&ifaces); + if (rc < 0) + goto failed; + + for (i = 0; i < rc; i++) { + if (strcmp(ifname, ifaces[i].li_name) == 0) + break; + } + + if (i == rc) { + CERROR("ko2iblnd: No matching interfaces\n"); + rc = -ENOENT; + goto failed; + } - newdev = !ibdev; - /* hmm...create kib_dev even for alias */ - if (!ibdev || strcmp(&ibdev->ibd_ifname[0], ifname)) - ibdev = kiblnd_create_dev(ifname); + ibdev = kzalloc(sizeof(*ibdev), GFP_KERNEL); + if (!ibdev) { + rc = -ENOMEM; + goto failed; + } + + ibdev->ibd_ifip = ifaces[i].li_ipaddr; + strlcpy(ibdev->ibd_ifname, ifaces[i].li_name, + sizeof(ibdev->ibd_ifname)); + ibdev->ibd_can_failover = !!(ifaces[i].li_flags & IFF_MASTER); - if (!ibdev) + INIT_LIST_HEAD(&ibdev->ibd_nets); + INIT_LIST_HEAD(&ibdev->ibd_list); /* not yet in kib_devs */ + INIT_LIST_HEAD(&ibdev->ibd_fail_list); + + /* initialize the device */ + rc = kiblnd_dev_failover(ibdev); + if (rc) { + CERROR("ko2iblnd: Can't initialize device: rc = %d\n", rc); goto failed; + } - node_id = dev_to_node(ibdev->ibd_hdev->ibh_ibdev->dma_device); - ni->ni_dev_cpt = cfs_cpt_of_node(lnet_cpt_table(), node_id); + list_add_tail(&ibdev->ibd_list, &kiblnd_data.kib_devs); net->ibn_dev = ibdev; ni->ni_nid = LNET_MKNID(LNET_NIDNET(ni->ni_nid), ibdev->ibd_ifip); - rc = kiblnd_dev_start_threads(ibdev, newdev, - ni->ni_cpts, ni->ni_ncpts); + ni->ni_dev_cpt = ifaces[i].li_cpt; + + rc = kiblnd_dev_start_threads(ibdev, ni->ni_cpts, ni->ni_ncpts); if (rc) goto failed; @@ -3037,6 +2955,7 @@ static int kiblnd_startup(struct lnet_ni *ni) if (!net->ibn_dev && ibdev) kiblnd_destroy_dev(ibdev); + kfree(ifaces); net_failed: kiblnd_shutdown(ni); diff --git a/net/lnet/klnds/socklnd/socklnd.c b/net/lnet/klnds/socklnd/socklnd.c index 1c957dc..d2c72b0 100644 --- a/net/lnet/klnds/socklnd/socklnd.c +++ b/net/lnet/klnds/socklnd/socklnd.c @@ -2602,63 +2602,6 @@ static int ksocknal_push(struct lnet_ni *ni, struct lnet_process_id id) } static int -ksocknal_enumerate_interfaces(struct ksock_net *net, char *iname) -{ - struct net_device *dev; - - rtnl_lock(); - for_each_netdev(&init_net, dev) { - /* The iname specified by a user land configuration can - * map to an ifa_label so always treat iname as an ifa_label. - * If iname is NULL then fall back to the net device name. - */ - const char *name = iname ? iname : dev->name; - const struct in_ifaddr *ifa; - struct in_device *in_dev; - - if (strcmp(dev->name, "lo") == 0) /* skip the loopback IF */ - continue; - - if (!(dev_get_flags(dev) & IFF_UP)) { - CWARN("Ignoring interface %s (down)\n", dev->name); - continue; - } - - in_dev = __in_dev_get_rtnl(dev); - if (!in_dev) { - CWARN("Interface %s has no IPv4 status.\n", dev->name); - continue; - } - - in_dev_for_each_ifa_rcu(ifa, in_dev) { - if (strcmp(name, ifa->ifa_label) == 0) { - int idx = net->ksnn_ninterfaces; - struct ksock_interface *ksi; - - if (idx >= ARRAY_SIZE(net->ksnn_interfaces)) { - rtnl_unlock(); - return -E2BIG; - } - - ksi = &net->ksnn_interfaces[idx]; - ksi->ksni_ipaddr = ntohl(ifa->ifa_local); - ksi->ksni_netmask = ifa->ifa_mask; - strlcpy(ksi->ksni_name, - name, sizeof(ksi->ksni_name)); - net->ksnn_ninterfaces++; - break; - } - } - } - rtnl_unlock(); - - if (net->ksnn_ninterfaces == 0) - CERROR("Can't find any usable interfaces\n"); - - return net->ksnn_ninterfaces > 0 ? 0 : -ENOENT; -} - -static int ksocknal_search_new_ipif(struct ksock_net *net) { int new_ipif = 0; @@ -2777,8 +2720,10 @@ static int ksocknal_push(struct lnet_ni *ni, struct lnet_process_id id) ksocknal_startup(struct lnet_ni *ni) { struct ksock_net *net; + struct ksock_interface *ksi = NULL; + struct lnet_inetdev *ifaces = NULL; + int i = 0; int rc; - int i; struct net_device *net_dev; int node_id; @@ -2809,11 +2754,20 @@ static int ksocknal_push(struct lnet_ni *ni, struct lnet_process_id id) ni->ni_net->net_tunables_set = true; } - net->ksnn_ninterfaces = 0; + rc = lnet_inet_enumerate(&ifaces); + if (rc < 0) + goto fail_1; + if (!ni->ni_interfaces[0]) { - rc = ksocknal_enumerate_interfaces(net, NULL); - if (rc <= 0) - goto fail_1; + ksi = &net->ksnn_interfaces[0]; + + /* Use the first discovered interface */ + net->ksnn_ninterfaces = 1; + ni->ni_dev_cpt = ifaces[0].li_cpt; + ksi->ksni_ipaddr = ifaces[0].li_ipaddr; + ksi->ksni_netmask = ifaces[0].li_netmask; + strlcpy(ksi->ksni_name, ifaces[0].li_name, + sizeof(ksi->ksni_name)); } else { /* Before Multi-Rail ksocklnd would manage * multiple interfaces with its own tcp bonding. @@ -2831,23 +2785,38 @@ static int ksocknal_push(struct lnet_ni *ni, struct lnet_process_id id) if (!ni->ni_interfaces[i]) break; - for (j = 0; j < net->ksnn_ninterfaces; j++) { - struct ksock_interface *ksi; - - ksi = &net->ksnn_interfaces[j]; - - if (strcmp(ni->ni_interfaces[i], - ksi->ksni_name) == 0) { - CERROR("found duplicate %s\n", - ksi->ksni_name); + for (j = 0; j < LNET_INTERFACES_NUM; j++) { + if (i != j && ni->ni_interfaces[j] && + strcmp(ni->ni_interfaces[i], + ni->ni_interfaces[j]) == 0) { rc = -EEXIST; + CERROR("ksocklnd: found duplicate %s at %d and %d, rc = %d\n", + ni->ni_interfaces[i], i, j, rc); goto fail_1; } } - rc = ksocknal_enumerate_interfaces(net, ni->ni_interfaces[i]); - if (rc <= 0) - goto fail_1; + for (j = 0; j < rc; j++) { + if (strcmp(ifaces[j].li_name, + ni->ni_interfaces[i]) != 0) + continue; + + ksi = &net->ksnn_interfaces[j]; + ni->ni_dev_cpt = ifaces[j].li_cpt; + ksi->ksni_ipaddr = ifaces[j].li_ipaddr; + ksi->ksni_netmask = ifaces[j].li_netmask; + strlcpy(ksi->ksni_name, ifaces[j].li_name, + sizeof(ksi->ksni_name)); + net->ksnn_ninterfaces++; + break; + } + } + + /* ni_interfaces don't map to all network interfaces */ + if (!ksi || net->ksnn_ninterfaces != i) { + CERROR("ksocklnd: requested %d but only %d interfaces found\n", + i, net->ksnn_ninterfaces); + goto fail_1; } } @@ -2866,8 +2835,8 @@ static int ksocknal_push(struct lnet_ni *ni, struct lnet_process_id id) if (rc) goto fail_1; - ni->ni_nid = LNET_MKNID(LNET_NIDNET(ni->ni_nid), - net->ksnn_interfaces[0].ksni_ipaddr); + LASSERT(ksi); + ni->ni_nid = LNET_MKNID(LNET_NIDNET(ni->ni_nid), ksi->ksni_ipaddr); list_add(&net->ksnn_list, &ksocknal_data.ksnd_nets); ksocknal_data.ksnd_nnets++; diff --git a/net/lnet/lnet/config.c b/net/lnet/lnet/config.c index 851b6c0..6dda6e5 100644 --- a/net/lnet/lnet/config.c +++ b/net/lnet/lnet/config.c @@ -1560,97 +1560,120 @@ struct lnet_ni * return count; } -static int -lnet_ipaddr_enumerate(u32 **ipaddrsp) +int lnet_inet_enumerate(struct lnet_inetdev **dev_list) { + struct lnet_inetdev *ifaces = NULL; struct net_device *dev; - u32 *ipaddrs; - int nalloc = 64; - int nip; - - ipaddrs = kcalloc(nalloc, sizeof(*ipaddrs), GFP_KERNEL); - if (!ipaddrs) { - CERROR("Can't allocate ipaddrs[%d]\n", nalloc); - return -ENOMEM; - } + int nalloc = 0; + int nip = 0; rtnl_lock(); for_each_netdev(&init_net, dev) { + int flags = dev_get_flags(dev); const struct in_ifaddr *ifa; struct in_device *in_dev; + int node_id; + int cpt; - if (strcmp(dev->name, "lo") == 0) + if (flags & IFF_LOOPBACK) /* skip the loopback IF */ continue; - if (!(dev_get_flags(dev) & IFF_UP)) { - CWARN("Ignoring interface %s: it's down\n", dev->name); + if (!(flags & IFF_UP)) { + CWARN("lnet: Ignoring interface %s: it's down\n", + dev->name); continue; } in_dev = __in_dev_get_rtnl(dev); if (!in_dev) { - CWARN("Interface %s has no IPv4 status.\n", dev->name); + CWARN("lnet: Interface %s has no IPv4 status.\n", + dev->name); continue; } - if (nip >= nalloc) { - u32 *ipaddrs2; - nalloc += nalloc; - ipaddrs2 = krealloc(ipaddrs, nalloc * sizeof(*ipaddrs2), - GFP_KERNEL); - if (ipaddrs2 == NULL) { - kfree(ipaddrs); - CERROR("Can't allocate ipaddrs[%d]\n", nalloc); - return -ENOMEM; + node_id = dev_to_node(&dev->dev); + cpt = cfs_cpt_of_node(lnet_cpt_table(), node_id); + in_dev_for_each_ifa_rtnl(ifa, in_dev) { + if (nip >= nalloc) { + struct lnet_inetdev *tmp; + + nalloc += LNET_INTERFACES_NUM; + tmp = krealloc(ifaces, nalloc * sizeof(*tmp), + GFP_KERNEL); + if (!tmp) { + kfree(ifaces); + ifaces = NULL; + nip = -ENOMEM; + goto unlock_rtnl; + } + ifaces = tmp; } - ipaddrs = ipaddrs2; - } - in_dev_for_each_ifa_rcu(ifa, in_dev) - if (!(ifa->ifa_flags & IFA_F_SECONDARY) && - strcmp(ifa->ifa_label, dev->name) == 0) { - ipaddrs[nip++] = ifa->ifa_local; - break; - } + ifaces[nip].li_cpt = cpt; + ifaces[nip].li_flags = flags; + ifaces[nip].li_ipaddr = ntohl(ifa->ifa_local); + ifaces[nip].li_netmask = ntohl(ifa->ifa_mask); + strlcpy(ifaces[nip].li_name, ifa->ifa_label, + sizeof(ifaces[nip].li_name)); + nip++; + } } +unlock_rtnl: rtnl_unlock(); - *ipaddrsp = ipaddrs; + if (nip == 0) { + CERROR("lnet: Can't find any usable interfaces, rc = -ENOENT\n"); + nip = -ENOENT; + } + + *dev_list = ifaces; return nip; } +EXPORT_SYMBOL(lnet_inet_enumerate); int lnet_parse_ip2nets(char **networksp, char *ip2nets) { + struct lnet_inetdev *ifaces = NULL; u32 *ipaddrs = NULL; - int nip = lnet_ipaddr_enumerate(&ipaddrs); + int nip; int rc; + int i; + nip = lnet_inet_enumerate(&ifaces); if (nip < 0) { - LCONSOLE_ERROR_MSG(0x117, - "Error %d enumerating local IP interfaces for ip2nets to match\n", - nip); + if (nip != -ENOENT) { + LCONSOLE_ERROR_MSG(0x117, + "Error %d enumerating local IP interfaces for ip2nets to match\n", + nip); + } else { + LCONSOLE_ERROR_MSG(0x118, + "No local IP interfaces for ip2nets to match\n"); + } return nip; } - if (!nip) { - LCONSOLE_ERROR_MSG(0x118, - "No local IP interfaces for ip2nets to match\n"); - return -ENOENT; + ipaddrs = kcalloc(nip, sizeof(*ipaddrs), GFP_KERNEL); + if (!ipaddrs) { + rc = -ENOMEM; + CERROR("lnet: Can't allocate ipaddrs[%d], rc = %d\n", + nip, rc); + goto out_free_addrs; } - rc = lnet_match_networks(networksp, ip2nets, ipaddrs, nip); - kfree(ipaddrs); + for (i = 0; i < nip; i++) + ipaddrs[i] = ifaces[i].li_ipaddr; + rc = lnet_match_networks(networksp, ip2nets, ipaddrs, nip); if (rc < 0) { LCONSOLE_ERROR_MSG(0x119, "Error %d parsing ip2nets\n", rc); - return rc; - } - - if (!rc) { + } else if (!rc) { LCONSOLE_ERROR_MSG(0x11a, "ip2nets does not match any local IP interfaces\n"); - return -ENOENT; + rc = -ENOENT; } - return 0; + kfree(ipaddrs); +out_free_addrs: + kfree(ifaces); + return rc > 0 ? 0 : rc; } From patchwork Mon Sep 30 18:54:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167107 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 91D4D16B1 for ; Mon, 30 Sep 2019 18:59:01 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7881C224D5 for ; Mon, 30 Sep 2019 18:59:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7881C224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 31E7F5C3E62; Mon, 30 Sep 2019 11:58:06 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 772B45C3165 for ; Mon, 30 Sep 2019 11:57:00 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 395D4100537C; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 32DA7BF; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:25 -0400 Message-Id: <1569869810-23848-7-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 006/151] lustre: support for gcc8 X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Alex Zhuravlev Building with gcc8 reports new warnings which are addressed in this patch. WC-bug-id: https://jira.whamcloud.com/browse/LU-11233 Lustre-commit: 6601661f9632 ("LU-11233 build: support for gcc8") Signed-off-by: Alex Zhuravlev Reviewed-on: https://review.whamcloud.com/34660 Reviewed-by: Andreas Dilger Reviewed-by: James Simmons Signed-off-by: James Simmons --- fs/lustre/lmv/lmv_obd.c | 14 ++++++++------ fs/lustre/obdclass/genops.c | 2 +- net/lnet/klnds/o2iblnd/o2iblnd_cb.c | 3 +-- 3 files changed, 10 insertions(+), 9 deletions(-) diff --git a/fs/lustre/lmv/lmv_obd.c b/fs/lustre/lmv/lmv_obd.c index 1c43022..acef43d 100644 --- a/fs/lustre/lmv/lmv_obd.c +++ b/fs/lustre/lmv/lmv_obd.c @@ -640,6 +640,7 @@ static int lmv_fid2path(struct obd_export *exp, int len, void *karg, if (remote_gf) { struct getinfo_fid2path *ori_gf; char *ptr; + int len; ori_gf = karg; if (strlen(ori_gf->gf_path) + 1 + @@ -650,12 +651,13 @@ static int lmv_fid2path(struct obd_export *exp, int len, void *karg, ptr = ori_gf->gf_path; - memmove(ptr + strlen(gf->gf_path) + 1, ptr, - strlen(ori_gf->gf_path)); - - strncpy(ptr, gf->gf_path, strlen(gf->gf_path)); - ptr += strlen(gf->gf_path); - *ptr = '/'; + len = strlen(gf->gf_path); + /* move the current path to the right to release space + * for closer-to-root part + */ + memmove(ptr + len + 1, ptr, strlen(ori_gf->gf_path)); + memcpy(ptr, gf->gf_path, len); + ptr[len] = '/'; } CDEBUG(D_INFO, "%s: get path %s " DFID " rec: %llu ln: %u\n", diff --git a/fs/lustre/obdclass/genops.c b/fs/lustre/obdclass/genops.c index 9a9de35..4b528a7 100644 --- a/fs/lustre/obdclass/genops.c +++ b/fs/lustre/obdclass/genops.c @@ -322,7 +322,7 @@ struct obd_device *class_newdev(const char *type_name, const char *name, newdev->obd_conn_inprogress = 0; - strncpy(newdev->obd_uuid.uuid, uuid, strlen(uuid)); + strncpy(newdev->obd_uuid.uuid, uuid, UUID_MAX); CDEBUG(D_IOCTL, "Allocate new device %s (%p)\n", newdev->obd_name, newdev); diff --git a/net/lnet/klnds/o2iblnd/o2iblnd_cb.c b/net/lnet/klnds/o2iblnd/o2iblnd_cb.c index b9585f6..871d4fb 100644 --- a/net/lnet/klnds/o2iblnd/o2iblnd_cb.c +++ b/net/lnet/klnds/o2iblnd/o2iblnd_cb.c @@ -1763,7 +1763,7 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, struct kib_tx *tx; int nob; int post_credit = IBLND_POSTRX_PEER_CREDIT; - u64 ibprm_cookie; + u64 ibprm_cookie = rxmsg->ibm_u.putreq.ibprm_cookie; int rc = 0; LASSERT(iov_iter_count(to) <= rlen); @@ -1796,7 +1796,6 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, break; case IBLND_MSG_PUT_REQ: { - u64 ibprm_cookie = rxmsg->ibm_u.putreq.ibprm_cookie; struct kib_msg *txmsg; struct kib_rdma_desc *rd; From patchwork Mon Sep 30 18:54:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167103 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D5FDD13BD for ; Mon, 30 Sep 2019 18:58:54 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BE7C8224D5 for ; Mon, 30 Sep 2019 18:58:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BE7C8224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 39F4E5C3E21; Mon, 30 Sep 2019 11:58:02 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 21DE25C3165 for ; Mon, 30 Sep 2019 11:57:00 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 39DB0100537E; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 36671A9; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:26 -0400 Message-Id: <1569869810-23848-8-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 007/151] lnet: Allocate MEs and small MDs in own kmem_caches X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Bruno Faccini As part of LU-3848 and LU-4330, it has been discovered that LNET MEs and small MDs (<=128 Bytes) are allocated in kmem_cache and thus can suffer quite frequent corruptions, from other modules or Kernel parts, that occur there. To avoid this, MEs and small-MDs specific kmem_cache have been created. Also, lnet_libmd struct fields have been re-ordered to optimize its memory foot-print. WC-bug-id: https://jira.whamcloud.com/browse/LU-4330 Lustre-commit: 9d9bb678d6b3 ("LU-4330 lnet: Allocate MEs and small MDs in own kmem_caches") Signed-off-by: Bruno Faccini Reviewed-on: http://review.whamcloud.com/18586 Reviewed-by: Andreas Dilger Reviewed-by: Doug Oucharek Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- include/linux/lnet/lib-lnet.h | 39 ++++++++++++++++++++++++++++++++++++++- include/linux/lnet/lib-types.h | 2 +- net/lnet/lnet/api-ni.c | 40 ++++++++++++++++++++++++++++++++++++++++ net/lnet/lnet/lib-md.c | 2 +- net/lnet/lnet/lib-me.c | 2 +- net/lnet/lnet/lib-ptl.c | 2 +- 6 files changed, 82 insertions(+), 5 deletions(-) diff --git a/include/linux/lnet/lib-lnet.h b/include/linux/lnet/lib-lnet.h index 776896a..538d457 100644 --- a/include/linux/lnet/lib-lnet.h +++ b/include/linux/lnet/lib-lnet.h @@ -80,6 +80,12 @@ /* default timeout */ #define DEFAULT_PEER_TIMEOUT 180 +#define LNET_SMALL_MD_SIZE offsetof(struct lnet_libmd, md_iov.iov[1]) +extern struct kmem_cache *lnet_mes_cachep; /* MEs kmem_cache */ +extern struct kmem_cache *lnet_small_mds_cachep; /* <= LNET_SMALL_MD_SIZE bytes + * MDs kmem_cache + */ + static inline int lnet_is_route_alive(struct lnet_route *route) { /* gateway is down */ @@ -151,6 +157,24 @@ static inline int lnet_md_unlinkable(struct lnet_libmd *md) cfs_percpt_unlock(the_lnet.ln_res_lock, cpt); } +static inline void +lnet_md_free(struct lnet_libmd *md) +{ + unsigned int size; + + if ((md->md_options & LNET_MD_KIOV) != 0) + size = offsetof(struct lnet_libmd, md_iov.kiov[md->md_niov]); + else + size = offsetof(struct lnet_libmd, md_iov.iov[md->md_niov]); + + if (size <= LNET_SMALL_MD_SIZE) { + CDEBUG(D_MALLOC, "slab-freed 'md' at %p.\n", md); + kmem_cache_free(lnet_small_mds_cachep, md); + } else { + kfree(md); + } +} + static inline int lnet_res_lock_current(void) { @@ -205,7 +229,20 @@ static inline int lnet_md_unlinkable(struct lnet_libmd *md) size = offsetof(struct lnet_libmd, md_iov.iov[niov]); } - md = kzalloc(size, GFP_NOFS); + if (size <= LNET_SMALL_MD_SIZE) { + md = kmem_cache_alloc(lnet_small_mds_cachep, + GFP_NOFS | __GFP_ZERO); + if (md) { + CDEBUG(D_MALLOC, + "slab-alloced 'md' of size %u at %p.\n", + size, md); + } else { + CDEBUG(D_MALLOC, "failed to allocate 'md' of size %u\n", + size); + } + } else { + md = kzalloc(size, GFP_NOFS); + } if (md) { /* Set here in case of early free */ md->md_options = umd->options; diff --git a/include/linux/lnet/lib-types.h b/include/linux/lnet/lib-types.h index 60f9c28..6d2d83a 100644 --- a/include/linux/lnet/lib-types.h +++ b/include/linux/lnet/lib-types.h @@ -164,9 +164,9 @@ struct lnet_libmd { int md_refcount; unsigned int md_options; unsigned int md_flags; + unsigned int md_niov; /* # frags at end of struct */ void *md_user_ptr; struct lnet_eq *md_eq; - unsigned int md_niov; /* # frags */ struct lnet_handle_md md_bulk_handle; union { struct kvec iov[LNET_MAX_IOV]; diff --git a/net/lnet/lnet/api-ni.c b/net/lnet/lnet/api-ni.c index 12b3c44..c33cd1f 100644 --- a/net/lnet/lnet/api-ni.c +++ b/net/lnet/lnet/api-ni.c @@ -216,6 +216,41 @@ static int lnet_discover(struct lnet_process_id id, u32 force, mutex_init(&the_lnet.ln_lnd_mutex); } +struct kmem_cache *lnet_mes_cachep; /* MEs kmem_cache */ +struct kmem_cache *lnet_small_mds_cachep; /* <= LNET_SMALL_MD_SIZE bytes + * MDs kmem_cache + */ + +static int +lnet_descriptor_setup(void) +{ + /* create specific kmem_cache for MEs and small MDs (i.e., originally + * allocated in kmem_cache). + */ + lnet_mes_cachep = kmem_cache_create("lnet_MEs", sizeof(struct lnet_me), + 0, 0, NULL); + if (!lnet_mes_cachep) + return -ENOMEM; + + lnet_small_mds_cachep = kmem_cache_create("lnet_small_MDs", + LNET_SMALL_MD_SIZE, 0, 0, + NULL); + if (!lnet_small_mds_cachep) + return -ENOMEM; + + return 0; +} + +static void +lnet_descriptor_cleanup(void) +{ + kmem_cache_destroy(lnet_small_mds_cachep); + lnet_small_mds_cachep = NULL; + + kmem_cache_destroy(lnet_mes_cachep); + lnet_mes_cachep = NULL; +} + static int lnet_create_remote_nets_table(void) { @@ -701,6 +736,10 @@ struct lnet_libhandle * INIT_LIST_HEAD(&the_lnet.ln_dc_expired); init_waitqueue_head(&the_lnet.ln_dc_waitq); + rc = lnet_descriptor_setup(); + if (rc != 0) + goto failed; + rc = lnet_create_remote_nets_table(); if (rc) goto failed; @@ -798,6 +837,7 @@ struct lnet_libhandle * the_lnet.ln_counters = NULL; } lnet_destroy_remote_nets_table(); + lnet_descriptor_cleanup(); return 0; } diff --git a/net/lnet/lnet/lib-md.c b/net/lnet/lnet/lib-md.c index 33a59fb..8fe3564 100644 --- a/net/lnet/lnet/lib-md.c +++ b/net/lnet/lnet/lib-md.c @@ -82,7 +82,7 @@ LASSERT(!list_empty(&md->md_list)); list_del_init(&md->md_list); - kfree(md); + lnet_md_free(md); } struct page *lnet_kvaddr_to_page(unsigned long vaddr) diff --git a/net/lnet/lnet/lib-me.c b/net/lnet/lnet/lib-me.c index f0365ea..4fe6991 100644 --- a/net/lnet/lnet/lib-me.c +++ b/net/lnet/lnet/lib-me.c @@ -91,7 +91,7 @@ if (!mtable) /* can't match portal type */ return -EPERM; - me = kzalloc(sizeof(*me), GFP_NOFS); + me = kmem_cache_alloc(lnet_mes_cachep, GFP_NOFS | __GFP_ZERO); if (!me) return -ENOMEM; diff --git a/net/lnet/lnet/lib-ptl.c b/net/lnet/lnet/lib-ptl.c index ae061e8..83548f5 100644 --- a/net/lnet/lnet/lib-ptl.c +++ b/net/lnet/lnet/lib-ptl.c @@ -773,7 +773,7 @@ struct list_head * != NULL) { CERROR("Active ME %p on exit\n", me); list_del(&me->me_list); - kfree(me); + kmem_cache_free(lnet_mes_cachep, me); } } /* the extra entry is for MEs with ignore bits */ From patchwork Mon Sep 30 18:54:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167049 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 44F4A13BD for ; Mon, 30 Sep 2019 18:57:12 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2D630224D5 for ; Mon, 30 Sep 2019 18:57:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2D630224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id B0D8A5C3A82; Mon, 30 Sep 2019 11:57:07 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 18D7E5C338D for ; Mon, 30 Sep 2019 11:57:01 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 3B430100537F; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 39BD4B4; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:27 -0400 Message-Id: <1569869810-23848-9-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 008/151] lustre: seq: make seq_proc_write_common() safer X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Andreas Dilger Don't allow seq_proc_write_common() to specify arbitrary ranges, since this can permanently corrupt the sequence controller and/or sequnece server. That would allow duplicate FID allocation, or possibly prevent any new files to be created or servers to be added to the filesystem. Instead, limit the sequence range that can be written via /proc to a subset of the sequence range currently allocated to that node. Add the "clear" keyword to allow dropping the entire local sequence and force a new one to be fetched from the sequence server. WC-bug-id: https://jira.whamcloud.com/browse/LU-3642 Lustre-commit: 05f69f5ee20e ("LU-3642 seq: make seq_proc_write_common() safer") Signed-off-by: Andreas Dilger Reviewed-on: http://review.whamcloud.com/7123 Reviewed-by: Jinshan Xiong Reviewed-by: Alex Zhuravlev Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/fid/lproc_fid.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/fs/lustre/fid/lproc_fid.c b/fs/lustre/fid/lproc_fid.c index 94869d4..e2e47df 100644 --- a/fs/lustre/fid/lproc_fid.c +++ b/fs/lustre/fid/lproc_fid.c @@ -52,14 +52,18 @@ /* Format: [0x64BIT_INT - 0x64BIT_INT] + 32 bytes just in case */ #define MAX_FID_RANGE_STRLEN (32 + 2 * 2 * sizeof(u64)) /* - * Note: this function is only used for testing, it is no safe for production - * use. + * Reduce the SEQ range allocated to a node to a strict subset of the range + * currently-allocated SEQ range. If the specified range is "clear", then + * drop all allocated sequences and request a new one from the master. + * + * Note: this function should only be used for testing, it is not necessarily + * safe for production use. */ static int ldebugfs_fid_write_common(const char __user *buffer, size_t count, struct lu_seq_range *range) { - struct lu_seq_range tmp; + struct lu_seq_range tmp = { 0, }; int rc; char kernbuf[MAX_FID_RANGE_STRLEN]; @@ -82,8 +86,6 @@ rc = sscanf(kernbuf, "[%llx - %llx]\n", (unsigned long long *)&tmp.lsr_start, (unsigned long long *)&tmp.lsr_end); - if (rc != 2) - return -EINVAL; if (!lu_seq_range_is_sane(&tmp) || lu_seq_range_is_zero(&tmp) || tmp.lsr_start < range->lsr_start || tmp.lsr_end > range->lsr_end) return -EINVAL; From patchwork Mon Sep 30 18:54:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167051 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9417B16B1 for ; Mon, 30 Sep 2019 18:57:13 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7C983224D5 for ; Mon, 30 Sep 2019 18:57:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7C983224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 69CA25C3AB3; Mon, 30 Sep 2019 11:57:08 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 5AF585C344B for ; Mon, 30 Sep 2019 11:57:01 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 3F38F1005380; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 3CC2FB5; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:28 -0400 Message-Id: <1569869810-23848-10-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 009/151] lustre: ptlrpc: Fix an rq_no_reply assertion failure X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Li Wei , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Li Wei An OSS had an assertion failure: LustreError: 5366:0:(ldlm_lib.c:2689:target_bulk_io()) @@@ timeout on bulk GET after 0+0s req@ffff88083a61b400 x1476486691018500/t0(4300509964) o4->8dda3382-83f8-6445-5eea-828fd59e4a06@192.168.1.116@o2ib1:0/0 lens 504/448 e 391470 to 0 dl 1408494729 ref 2 fl Complete:/4/0 rc 0/0 LustreError: 5432:0:(niobuf.c:550:ptlrpc_send_reply()) ASSERTION( req->rq_no_reply == 0 ) failed: Lustre: soaked-OST0000: Bulk IO write error with 8dda3382-83f8-6445-5eea-828fd59e4a06 (at 192.168.1.116@o2ib1), client will retry: rc -110 LustreError: 5432:0:(niobuf.c:550:ptlrpc_send_reply()) LBUG Pid: 5432, comm: ll_ost_io03_003 Call Trace: [] libcfs_debug_dumpstack+0x55/0x80 [libcfs] [] lbug_with_loc+0x47/0xb0 [libcfs] [] ptlrpc_send_reply+0x4ec/0x7f0 [ptlrpc] [] ? lustre_pack_reply_flags+0xae/0x1f0 [ptlrpc] [] ptlrpc_at_check_timed+0xcd5/0x1370 [ptlrpc] [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] [] ptlrpc_main+0x12e8/0x1990 [ptlrpc] [] ? pick_next_task_fair+0xd0/0x130 [] ? schedule+0x176/0x3b0 [] ? ptlrpc_main+0x0/0x1990 [ptlrpc] [] kthread+0x96/0xa0 [] child_rip+0xa/0x20 [] ? kthread+0x0/0xa0 [] ? child_rip+0x0/0x20 The thread in tgt_brw_write() had decided not to reply by setting rq_no_reply, right before another thread tried to send an early reply for the request. WC-bug-id: https://jira.whamcloud.com/browse/LU-5537 Lustre-commit: a8d448e4cd59 ("LU-5537 ptlrpc: Fix an rq_no_reply assertion failure") Signed-off-by: Li Wei Reviewed-on: http://review.whamcloud.com/11740 Reviewed-by: Andreas Dilger Reviewed-by: Johann Lombardi Signed-off-by: James Simmons --- fs/lustre/ptlrpc/service.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/fs/lustre/ptlrpc/service.c b/fs/lustre/ptlrpc/service.c index a40e964..c9ab9c3 100644 --- a/fs/lustre/ptlrpc/service.c +++ b/fs/lustre/ptlrpc/service.c @@ -1098,6 +1098,16 @@ static int ptlrpc_at_send_early_reply(struct ptlrpc_request *req) reqcopy->rq_reqmsg = reqmsg; memcpy(reqmsg, req->rq_reqmsg, req->rq_reqlen); + /* + * tgt_brw_read() and tgt_brw_write() may have decided not to reply. + * Without this check, we would fail the rq_no_reply assertion in + * ptlrpc_send_reply(). + */ + if (reqcopy->rq_no_reply) { + rc = -ETIMEDOUT; + goto out; + } + LASSERT(atomic_read(&req->rq_refcount)); /** if it is last refcount then early reply isn't needed */ if (atomic_read(&req->rq_refcount) == 1) { From patchwork Mon Sep 30 18:54:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167075 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EB3E516B1 for ; Mon, 30 Sep 2019 18:57:56 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D3555224D5 for ; Mon, 30 Sep 2019 18:57:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D3555224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 80E915C3B49; Mon, 30 Sep 2019 11:57:31 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id E3B9A5C34A5 for ; Mon, 30 Sep 2019 11:57:01 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 4437B1005381; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 3FBCCBB; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:29 -0400 Message-Id: <1569869810-23848-11-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 010/151] lustre: fld: resend seq lookup RPC if it is on LWP X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Wang Di , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Wang Di Because Light Weight connection might be evicted after restart, then cause inflight RPC fails, to avoid this, we need resend seq lookup RPC. remove "-f" from "stop mdt" in sanity 17m, so umount can keep the the connection, and otherwise the OSP might be evicted. WC-bug-id: https://jira.whamcloud.com/browse/LU-4571 Lustre-commit: cf7f66d87e52 ("LU-4571 fld: resend seq lookup RPC if it is on LWP") Signed-off-by: wang di Reviewed-on: http://review.whamcloud.com/9106 Reviewed-by: Andreas Dilger Reviewed-by: Jinshan Xiong Reviewed-by: Niu Yawei Signed-off-by: James Simmons --- fs/lustre/fld/fld_request.c | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/fs/lustre/fld/fld_request.c b/fs/lustre/fld/fld_request.c index 248fffa..4cb49cb 100644 --- a/fs/lustre/fld/fld_request.c +++ b/fs/lustre/fld/fld_request.c @@ -314,6 +314,7 @@ int fld_client_rpc(struct obd_export *exp, LASSERT(exp); +again: imp = class_exp2cliimp(exp); switch (fld_op) { case FLD_QUERY: @@ -329,8 +330,15 @@ int fld_client_rpc(struct obd_export *exp, op = req_capsule_client_get(&req->rq_pill, &RMF_FLD_OPC); *op = FLD_LOOKUP; - if (imp->imp_connect_flags_orig & OBD_CONNECT_MDS_MDS) + /* For MDS_MDS seq lookup, it will always use LWP connection, + * but LWP will be evicted after restart, so cause the error. + * so we will set no_delay for seq lookup request, once the + * request fails because of the eviction. always retry here + */ + if (imp->imp_connect_flags_orig & OBD_CONNECT_MDS_MDS) { req->rq_allow_replay = 1; + req->rq_no_delay = 1; + } break; case FLD_READ: req = ptlrpc_request_alloc_pack(imp, &RQF_FLD_READ, @@ -358,8 +366,19 @@ int fld_client_rpc(struct obd_export *exp, obd_get_request_slot(&exp->exp_obd->u.cli); rc = ptlrpc_queue_wait(req); obd_put_request_slot(&exp->exp_obd->u.cli); - if (rc) + if (rc != 0) { + if (rc == -EWOULDBLOCK) { + /* For no_delay req(see above), EWOULDBLOCK means the + * connection is being evicted, but this seq lookup + * should not return error, since it would cause + * unnecessary failure of the application, instead + * it should retry here + */ + ptlrpc_req_finished(req); + goto again; + } goto out_req; + } if (fld_op == FLD_QUERY) { prange = req_capsule_server_get(&req->rq_pill, &RMF_FLD_MDFLD); From patchwork Mon Sep 30 18:54:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167109 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8592A13BD for ; Mon, 30 Sep 2019 18:59:04 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6D9AE224D5 for ; Mon, 30 Sep 2019 18:59:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6D9AE224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 485D75C3E6D; Mon, 30 Sep 2019 11:58:07 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id A04705C344B for ; Mon, 30 Sep 2019 11:57:01 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 45C42100538B; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 42BBCBD; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:30 -0400 Message-Id: <1569869810-23848-12-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 011/151] lustre: fld: retry fld rpc even for ESHUTDOWN X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Wang Di , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Wang Di when LWP is being evicted, because it is not replayable, the request might return ESHUTDOWN or EWOULDBLOCK, instead of failed, which might cause application failure, fld client will retry RPC, until the connection is being setup again or the LWP is being closed. WC-bug-id: https://jira.whamcloud.com/browse/LU-4420 Lustre-commit: d335e310d4bf ("LU-4420 fld: retry fld rpc even for ESHUTDOWN") Signed-off-by: wang di Reviewed-on: http://review.whamcloud.com/10285 Reviewed-by: Fan Yong Reviewed-by: Johann Lombardi Signed-off-by: James Simmons --- fs/lustre/fld/fld_request.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/fs/lustre/fld/fld_request.c b/fs/lustre/fld/fld_request.c index 4cb49cb..b5b905f 100644 --- a/fs/lustre/fld/fld_request.c +++ b/fs/lustre/fld/fld_request.c @@ -367,12 +367,12 @@ int fld_client_rpc(struct obd_export *exp, rc = ptlrpc_queue_wait(req); obd_put_request_slot(&exp->exp_obd->u.cli); if (rc != 0) { - if (rc == -EWOULDBLOCK) { - /* For no_delay req(see above), EWOULDBLOCK means the - * connection is being evicted, but this seq lookup - * should not return error, since it would cause - * unnecessary failure of the application, instead - * it should retry here + if (rc == -EWOULDBLOCK || rc == -ESHUTDOWN) { + /* For no_delay req(see above), EWOULDBLOCK and + * ESHUTDOWN means the connection is being evicted, + * but this seq lookup should not return error, + * since it would cause unnecessary failure of the + * application, instead it should retry here */ ptlrpc_req_finished(req); goto again; From patchwork Mon Sep 30 18:54:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167065 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 12CAC16B1 for ; Mon, 30 Sep 2019 18:57:41 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EF077224D5 for ; Mon, 30 Sep 2019 18:57:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EF077224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 7E65F5C32FA; Mon, 30 Sep 2019 11:57:22 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 328FD5C34E7 for ; Mon, 30 Sep 2019 11:57:02 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 476A7100538C; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 45CA1A9; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:31 -0400 Message-Id: <1569869810-23848-13-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 012/151] lustre: fld: retry fld rpc until the import is closed X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Wang Di , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Wang Di Since LWP is not replayable, it should retry RPC until the import is closed, otherwise it would cause unnecessary failure of the application. WC-bug-id: https://jira.whamcloud.com/browse/LU-5312 Lustre-commit: 07d481957c16 ("LU-5312 fld: retry fld rpc until the import is closed") Signed-off-by: Wang Di Reviewed-on: http://review.whamcloud.com/11039 Reviewed-by: Andreas Dilger Reviewed-by: Fan Yong Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/fld/fld_request.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/fs/lustre/fld/fld_request.c b/fs/lustre/fld/fld_request.c index b5b905f..062f19f 100644 --- a/fs/lustre/fld/fld_request.c +++ b/fs/lustre/fld/fld_request.c @@ -367,12 +367,10 @@ int fld_client_rpc(struct obd_export *exp, rc = ptlrpc_queue_wait(req); obd_put_request_slot(&exp->exp_obd->u.cli); if (rc != 0) { - if (rc == -EWOULDBLOCK || rc == -ESHUTDOWN) { - /* For no_delay req(see above), EWOULDBLOCK and - * ESHUTDOWN means the connection is being evicted, - * but this seq lookup should not return error, - * since it would cause unnecessary failure of the - * application, instead it should retry here + if (imp->imp_state != LUSTRE_IMP_CLOSED) { + /* Since LWP is not replayable, so it will keep + * trying unless umount happens, otherwise it would + * cause unnecessary failure of the application. */ ptlrpc_req_finished(req); goto again; From patchwork Mon Sep 30 18:54:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167111 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9537A13BD for ; Mon, 30 Sep 2019 18:59:09 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7D4FE224D5 for ; Mon, 30 Sep 2019 18:59:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7D4FE224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 62B785C3E91; Mon, 30 Sep 2019 11:58:09 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 7310B5C3546 for ; Mon, 30 Sep 2019 11:57:02 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 4B922100538D; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 490ABB4; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:32 -0400 Message-Id: <1569869810-23848-14-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 013/151] lustre: fld: fld client lookup should retry X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Wang Di , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Wang Di If FLD client lookup fails because of the remote target is shutdown (or deactive), it should retry another target, otherwise it will cause the application failure. And FLD client should stop retry if the import has been deactive. WC-bug-id: https://jira.whamcloud.com/browse/LU-6419 Lustre-commit: 3ededde903c ("LU-6419 fld: fld client lookup should retry") Signed-off-by: wang di Reviewed-on: http://review.whamcloud.com/14313 Reviewed-by: Lai Siyao Reviewed-by: Fan Yong Reviewed-by: Alex Zhuravlev Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/fld/fld_request.c | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/fs/lustre/fld/fld_request.c b/fs/lustre/fld/fld_request.c index 062f19f..75cba18 100644 --- a/fs/lustre/fld/fld_request.c +++ b/fs/lustre/fld/fld_request.c @@ -367,7 +367,7 @@ int fld_client_rpc(struct obd_export *exp, rc = ptlrpc_queue_wait(req); obd_put_request_slot(&exp->exp_obd->u.cli); if (rc != 0) { - if (imp->imp_state != LUSTRE_IMP_CLOSED) { + if (imp->imp_state != LUSTRE_IMP_CLOSED && !imp->imp_deactive) { /* Since LWP is not replayable, so it will keep * trying unless umount happens, otherwise it would * cause unnecessary failure of the application. @@ -404,6 +404,7 @@ int fld_client_lookup(struct lu_client_fld *fld, u64 seq, u32 *mds, { struct lu_seq_range res = { 0 }; struct lu_fld_target *target; + struct lu_fld_target *origin; int rc; rc = fld_cache_lookup(fld->lcf_cache, seq, &res); @@ -415,7 +416,8 @@ int fld_client_lookup(struct lu_client_fld *fld, u64 seq, u32 *mds, /* Can not find it in the cache */ target = fld_client_get_target(fld, seq); LASSERT(target); - + origin = target; +again: CDEBUG(D_INFO, "%s: Lookup fld entry (seq: %#llx) on target %s (idx %llu)\n", fld->lcf_name, seq, fld_target_name(target), target->ft_idx); @@ -424,6 +426,23 @@ int fld_client_lookup(struct lu_client_fld *fld, u64 seq, u32 *mds, fld_range_set_type(&res, flags); rc = fld_client_rpc(target->ft_exp, &res, FLD_QUERY, NULL); + if (rc == -ESHUTDOWN) { + /* If fld lookup failed because the target has been shutdown, + * then try next target in the list, until trying all targets + * or fld lookup succeeds + */ + spin_lock(&fld->lcf_lock); + if (target->ft_chain.next == fld->lcf_targets.prev) + target = list_entry(fld->lcf_targets.next, + struct lu_fld_target, ft_chain); + else + target = list_entry(target->ft_chain.next, + struct lu_fld_target, + ft_chain); + spin_unlock(&fld->lcf_lock); + if (target != origin) + goto again; + } if (rc == 0) { *mds = res.lsr_index; From patchwork Mon Sep 30 18:54:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167055 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 56FA513BD for ; Mon, 30 Sep 2019 18:57:22 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3F5E8224D5 for ; Mon, 30 Sep 2019 18:57:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3F5E8224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id ADC535C3B3B; Mon, 30 Sep 2019 11:57:12 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id B3ABA5C3546 for ; Mon, 30 Sep 2019 11:57:02 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 4EF90100538F; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 4C12AB5; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:33 -0400 Message-Id: <1569869810-23848-15-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 014/151] lustre: ldlm: testcases for multiple modify RPCs feature X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Gregoire Pichon This patch creates new testcases in the Auster test suite to test the support of multiple modify RPCs in flight feature. Two new OBD_FAIL codes are added to allow several failure occurences of the reint requests or replies. This is needed because the current fail checks impose the OBD_FAIL_ONCE flag. WC-bug-id: https://jira.whamcloud.com/browse/LU-5319 Lustre-commit: c2d27a0f1268 ("LU-5319 tests: testcases for multiple modify RPCs feature") Signed-off-by: Gregoire Pichon Reviewed-on: http://review.whamcloud.com/14861 Reviewed-by: Jian Yu Reviewed-by: Andreas Dilger Signed-off-by: James Simmons --- fs/lustre/include/obd_support.h | 2 ++ fs/lustre/ldlm/ldlm_lib.c | 3 +++ 2 files changed, 5 insertions(+) diff --git a/fs/lustre/include/obd_support.h b/fs/lustre/include/obd_support.h index 3e15cac..7999ac6 100644 --- a/fs/lustre/include/obd_support.h +++ b/fs/lustre/include/obd_support.h @@ -189,6 +189,8 @@ #define OBD_FAIL_MDS_SWAP_LAYOUTS_NET 0x14f #define OBD_FAIL_MDS_HSM_ACTION_NET 0x150 #define OBD_FAIL_MDS_CHANGELOG_INIT 0x151 +#define OBD_FAIL_MDS_REINT_MULTI_NET 0x159 +#define OBD_FAIL_MDS_REINT_MULTI_NET_REP 0x15a /* layout lock */ #define OBD_FAIL_MDS_NO_LL_GETATTR 0x170 diff --git a/fs/lustre/ldlm/ldlm_lib.c b/fs/lustre/ldlm/ldlm_lib.c index 887507d..a3b8df4 100644 --- a/fs/lustre/ldlm/ldlm_lib.c +++ b/fs/lustre/ldlm/ldlm_lib.c @@ -689,6 +689,9 @@ int target_pack_pool_reply(struct ptlrpc_request *req) DEBUG_REQ(D_ERROR, req, "dropping reply"); return -ECOMM; } + if (unlikely(lustre_msg_get_opc(req->rq_reqmsg) == MDS_REINT && + OBD_FAIL_CHECK(OBD_FAIL_MDS_REINT_MULTI_NET_REP))) + return -ECOMM; if (unlikely(rc)) { DEBUG_REQ(D_NET, req, "processing error (%d)", rc); From patchwork Mon Sep 30 18:54:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167079 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 15A1016B1 for ; Mon, 30 Sep 2019 18:58:05 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F237F224D5 for ; Mon, 30 Sep 2019 18:58:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F237F224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id C58675C3950; Mon, 30 Sep 2019 11:57:35 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 008EB5C3546 for ; Mon, 30 Sep 2019 11:57:02 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 506CC1005393; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 4F49ABE; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:34 -0400 Message-Id: <1569869810-23848-16-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 015/151] lustre: ldlm: Don't check opcode with NULL rq_reqmsg X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jeremy Filizetti , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Jeremy Filizetti When GSS is enabled it's possible to have a NULL rq_reqmsg if a bad signature or no context is returned during the unwrap of the request. Don't check the opcode in this case. WC-bug-id: https://jira.whamcloud.com/browse/LU-7508 Lustre-commit: 3f4572caef5f ("LU-7508 ldlm: Don't check opcode with NULL rq_reqmsg") Signed-off-by: Jeremy Filizetti Reviewed-on: http://review.whamcloud.com/17414 Reviewed-by: Andreas Dilger Reviewed-by: Sebastien Buisson Reviewed-by: John L. Hammond Signed-off-by: James Simmons --- fs/lustre/ldlm/ldlm_lib.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/fs/lustre/ldlm/ldlm_lib.c b/fs/lustre/ldlm/ldlm_lib.c index a3b8df4..04982b8 100644 --- a/fs/lustre/ldlm/ldlm_lib.c +++ b/fs/lustre/ldlm/ldlm_lib.c @@ -689,8 +689,12 @@ int target_pack_pool_reply(struct ptlrpc_request *req) DEBUG_REQ(D_ERROR, req, "dropping reply"); return -ECOMM; } - if (unlikely(lustre_msg_get_opc(req->rq_reqmsg) == MDS_REINT && - OBD_FAIL_CHECK(OBD_FAIL_MDS_REINT_MULTI_NET_REP))) + /* We can have a null rq_reqmsg in the event of bad signature or + * no context when unwrapping + */ + if (req->rq_reqmsg && + unlikely(lustre_msg_get_opc(req->rq_reqmsg) == MDS_REINT && + OBD_FAIL_CHECK(OBD_FAIL_MDS_REINT_MULTI_NET_REP))) return -ECOMM; if (unlikely(rc)) { From patchwork Mon Sep 30 18:54:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167053 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ACCB116B1 for ; Mon, 30 Sep 2019 18:57:19 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9261B224D5 for ; Mon, 30 Sep 2019 18:57:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9261B224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 731885C33E7; Mon, 30 Sep 2019 11:57:11 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 41A475C340D for ; Mon, 30 Sep 2019 11:57:03 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 5361B1005394; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 525ECA9; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:35 -0400 Message-Id: <1569869810-23848-17-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 016/151] lustre: all: remove all Sun license and URL references X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: James Simmons , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" The link to the GPL license at Sun is no longer there. Also Sun Inc is gone so lets remove all references to it in our source code. WC-bug-id: https://jira.whamcloud.com/browse/LU-8648 Lustre-commit: 72057a3af19e ("LU-8648 all: remove all Sun license and URL references") Signed-off-by: James Simmons Reviewed-on: http://review.whamcloud.com/22800 Reviewed-by: Andreas Dilger Reviewed-by: John L. Hammond Reviewed-by: Frank Zago Signed-off-by: James Simmons --- fs/lustre/include/lustre_lmv.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/lustre/include/lustre_lmv.h b/fs/lustre/include/lustre_lmv.h index c4f05d2..5e15c62 100644 --- a/fs/lustre/include/lustre_lmv.h +++ b/fs/lustre/include/lustre_lmv.h @@ -16,7 +16,7 @@ * * You should have received a copy of the GNU General Public License * version 2 along with this program; If not, see - * http://www.sun.com/software/products/lustre/docs/GPLv2.pdf + * http://www.gnu.org/licenses/gpl-2.0.html * * GPL HEADER END */ From patchwork Mon Sep 30 18:54:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167115 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CA9A516B1 for ; Mon, 30 Sep 2019 18:59:16 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B320F224D5 for ; Mon, 30 Sep 2019 18:59:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B320F224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id ED9715C3B49; Mon, 30 Sep 2019 11:58:12 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 848A05C340D for ; Mon, 30 Sep 2019 11:57:03 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 5709E1005395; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 557FABB; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:36 -0400 Message-Id: <1569869810-23848-18-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 017/151] lustre: ldlm: Use interval tree to update kms X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Patrick Farrell Currently, ldlm_extent_shift_kms does a linear search of the list of granted locks on the resource when looking for the lock to use to update the kms. This can be avoided by using the interval trees which store the extents of granted locks. We change the interval tree to store locks in reverse order, with lowest end first, and highest end last. Then we can easily find the last lock-end that doesn't ignore kms, and use that to choose a new kms. NeilBrown: modified from original to work with linux interval-tree WC-bug-id: https://jira.whamcloud.com/browse/LU-8272 Lustre-commit: e7cf1b060ba3 ("LU-8272 ldlm: Use interval tree to update kms") Signed-off-by: Patrick Farrell Signed-off-by: NeilBrown Reviewed-on: https://review.whamcloud.com/20779 Reviewed-by: Vitaly Fertman Reviewed-by: Jinshan Xiong Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/ldlm/ldlm_extent.c | 66 ++++++++++++++++++++++++++++++++++---------- 1 file changed, 51 insertions(+), 15 deletions(-) diff --git a/fs/lustre/ldlm/ldlm_extent.c b/fs/lustre/ldlm/ldlm_extent.c index 0695f7e..66616b0 100644 --- a/fs/lustre/ldlm/ldlm_extent.c +++ b/fs/lustre/ldlm/ldlm_extent.c @@ -55,13 +55,22 @@ #include "ldlm_internal.h" #include -#define START(node) ((node)->l_policy_data.l_extent.start) -#define LAST(node) ((node)->l_policy_data.l_extent.end) +/* We sort the interval tree in reverse order, because we sometimes + * and to find the interval with the highest end, and the first/next + * iteration only allows is to walk in increasing order of start. + */ +#define ISTART(end) (U64_MAX - (end)) +#define IEND(start) (U64_MAX - (start)) + +#define START(node) ISTART((node)->l_policy_data.l_extent.end) +#define LAST(node) IEND((node)->l_policy_data.l_extent.start) INTERVAL_TREE_DEFINE(struct ldlm_lock, l_rb, u64, __subtree_last, START, LAST, static, extent); /* When a lock is cancelled by a client, the KMS may undergo change if this - * is the "highest lock". This function returns the new KMS value. + * is the "highest lock". This function returns the new KMS value, updating + * it only if we were the highest lock. + * * Caller must hold lr_lock already. * * NB: A lock on [x,y] protects a KMS of up to y + 1 bytes! @@ -69,8 +78,11 @@ u64 ldlm_extent_shift_kms(struct ldlm_lock *lock, u64 old_kms) { struct ldlm_resource *res = lock->l_resource; + struct ldlm_interval_tree *tree; struct ldlm_lock *lck; u64 kms = 0; + int idx = 0; + bool complete; /* don't let another thread in ldlm_extent_shift_kms race in * just after we finish and take our lock into account in its @@ -78,20 +90,44 @@ u64 ldlm_extent_shift_kms(struct ldlm_lock *lock, u64 old_kms) */ ldlm_set_kms_ignore(lock); - list_for_each_entry(lck, &res->lr_granted, l_res_link) { - if (ldlm_is_kms_ignore(lck)) - continue; + /* We iterate over the lock trees, looking for the largest kms + * smaller than the current one. Note that each tree is + * iterated starting a largest end, because the interval tree + * is stored last-to-first order. + */ + for (idx = 0; idx < LCK_MODE_NUM; idx++) { + tree = &res->lr_itree[idx]; - if (lck->l_policy_data.l_extent.end >= old_kms) - return old_kms; + for (lck = extent_iter_first(&tree->lit_root, 0, U64_MAX); + lck; + lck = extent_iter_next(lck, 0, U64_MAX)) { + if (ldlm_is_kms_ignore(lck)) + continue; - /* This extent _has_ to be smaller than old_kms (checked above) - * so kms can only ever be smaller or the same as old_kms. + /* This is the last lock-end that doesn't ignore + * kms. + * If it has a greater or equal kms, we are not + * the highest lock (or we share that distinction + * with another lock), and don't need to update KMS. + * Record old_kms and stop looking. + */ + if (lck->l_policy_data.l_extent.end >= old_kms) { + kms = old_kms; + complete = true; + } else + kms = lck->l_policy_data.l_extent.end + 1; + break; + } + + /* this tells us we're not the highest lock, so we don't need + * to check the remaining trees */ - if (lck->l_policy_data.l_extent.end + 1 > kms) - kms = lck->l_policy_data.l_extent.end + 1; + if (complete) + break; } - LASSERTF(kms <= old_kms, "kms %llu old_kms %llu\n", kms, old_kms); + + LASSERTF(kms <= old_kms, "kms %llu old_kms %llu\n", kms, + old_kms); return kms; } @@ -197,9 +233,9 @@ void ldlm_extent_search(struct rb_root_cached *root, { struct ldlm_lock *lock; - for (lock = extent_iter_first(root, start, end); + for (lock = extent_iter_first(root, ISTART(end), IEND(start)); lock; - lock = extent_iter_next(lock, start, end)) + lock = extent_iter_next(lock, ISTART(end), IEND(start))) if (matches(lock, data)) break; } From patchwork Mon Sep 30 18:54:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167057 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8E3B913BD for ; Mon, 30 Sep 2019 18:57:26 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 75BBC224D5 for ; Mon, 30 Sep 2019 18:57:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 75BBC224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id E5F815C3A8A; Mon, 30 Sep 2019 11:57:14 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id DADC05C340D for ; Mon, 30 Sep 2019 11:57:03 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 5A9FA1005396; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 58D90B4; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:37 -0400 Message-Id: <1569869810-23848-19-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 018/151] lustre: osc: prepare OSC code to be used from MDC X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mikhal Pershin , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mikhal Pershin Move osc/osc_cl_include.h to include/lustre_osc.h to be included from both OSC and MDC. WC-bug-id: https://jira.whamcloud.com/browse/LU-3285 Lustre-commit: 036d987c6d71 ("LU-3285 osc: prepare OSC code to be used from MDC") Signed-off-by: Mikhal Pershin Reviewed-on: https://review.whamcloud.com/22996 Reviewed-by: Andreas Dilger Reviewed-by: Jinshan Xiong Signed-off-by: James Simmons --- .../osc_cl_internal.h => include/lustre_osc.h} | 151 ++++++++++++++++----- fs/lustre/osc/lproc_osc.c | 2 + fs/lustre/osc/osc_cache.c | 3 +- fs/lustre/osc/osc_dev.c | 14 +- fs/lustre/osc/osc_internal.h | 107 +++------------ fs/lustre/osc/osc_io.c | 18 +-- fs/lustre/osc/osc_lock.c | 3 +- fs/lustre/osc/osc_object.c | 20 +-- fs/lustre/osc/osc_page.c | 4 +- fs/lustre/osc/osc_quota.c | 2 + fs/lustre/osc/osc_request.c | 3 +- 11 files changed, 157 insertions(+), 170 deletions(-) rename fs/lustre/{osc/osc_cl_internal.h => include/lustre_osc.h} (86%) diff --git a/fs/lustre/osc/osc_cl_internal.h b/fs/lustre/include/lustre_osc.h similarity index 86% rename from fs/lustre/osc/osc_cl_internal.h rename to fs/lustre/include/lustre_osc.h index 5fec6f6..f96583a 100644 --- a/fs/lustre/osc/osc_cl_internal.h +++ b/fs/lustre/include/lustre_osc.h @@ -31,22 +31,97 @@ * Lustre is a trademark of Sun Microsystems, Inc. */ /* - * This file is part of Lustre, http://www.lustre.org/ - * Lustre is a trademark of Sun Microsystems, Inc. + * fs/lustre/include/lustre_osc.h * - * Internal interfaces of OSC layer. + * OSC layer structures and methods common for both OSC and MDC. + * + * This file contains OSC interfaces used by OSC and MDC. Most of them + * were just moved from fs/lustre/osc/osc_cl_internal.h for Data-on-MDT + * purposes * * Author: Nikita Danilov - * Author: Jinshan Xiong + * Author: Jinshan Xiong + * Author: Mikhail Pershin */ -#ifndef OSC_CL_INTERNAL_H -#define OSC_CL_INTERNAL_H +#ifndef LUSTRE_OSC_H +#define LUSTRE_OSC_H + +#include #include -/* osc_build_res_name() */ #include -#include "osc_internal.h" + +struct osc_quota_info { + /* linkage for quota hash table */ + struct rhash_head oqi_hash; + u32 oqi_id; + struct rcu_head rcu; +}; + +enum async_flags { + ASYNC_READY = 0x1, /* osc_make_ready will not be called + * before this page is added to an rpc + */ + ASYNC_URGENT = 0x2, /* page must be put into an RPC before + * return + */ + ASYNC_COUNT_STABLE = 0x4, /* osc_refresh_count will not be called + * to give the caller a chance to update + * or cancel the size of the io + */ + ASYNC_HP = 0x10, +}; + +struct osc_async_page { + int oap_magic; + unsigned short oap_cmd; + + struct list_head oap_pending_item; + struct list_head oap_rpc_item; + + u64 oap_obj_off; + unsigned int oap_page_off; + enum async_flags oap_async_flags; + + struct brw_page oap_brw_page; + + struct ptlrpc_request *oap_request; + struct client_obd *oap_cli; + struct osc_object *oap_obj; + + spinlock_t oap_lock; +}; + +#define oap_page oap_brw_page.pg +#define oap_count oap_brw_page.count +#define oap_brw_flags oap_brw_page.flag + +static inline struct osc_async_page *brw_page2oap(struct brw_page *pga) +{ + return container_of(pga, struct osc_async_page, oap_brw_page); +} + +static inline void osc_wake_cache_waiters(struct client_obd *cli) +{ + wake_up(&cli->cl_cache_waiters); +} + +struct osc_device { + struct cl_device od_cl; + struct obd_export *od_exp; + + /* Write stats is actually protected by client_obd's lock. */ + struct osc_stats { + u64 os_lockless_writes; /* by bytes */ + u64 os_lockless_reads; /* by bytes */ + u64 os_lockless_truncates; /* by times */ + } od_stats; + + /* configuration item(s) */ + int od_contention_time; + int od_lockless_truncate; +}; /* \defgroup osc osc * @{ @@ -166,6 +241,8 @@ struct osc_object { /* number of active IOs of this object */ atomic_t oo_nr_ios; wait_queue_head_t oo_io_waitq; + + bool oo_initialized; }; static inline void osc_object_lock(struct osc_object *obj) @@ -353,21 +430,13 @@ struct osc_page { extern struct kmem_cache *osc_thread_kmem; extern struct kmem_cache *osc_session_kmem; extern struct kmem_cache *osc_extent_kmem; +extern struct kmem_cache *osc_quota_kmem; -extern struct lu_device_type osc_device_type; extern struct lu_context_key osc_key; extern struct lu_context_key osc_session_key; #define OSC_FLAGS (ASYNC_URGENT | ASYNC_READY) -int osc_lock_init(const struct lu_env *env, - struct cl_object *obj, struct cl_lock *lock, - const struct cl_io *io); -int osc_io_init(const struct lu_env *env, - struct cl_object *obj, struct cl_io *io); -struct lu_object *osc_object_alloc(const struct lu_env *env, - const struct lu_object_header *hdr, - struct lu_device *dev); int osc_page_init(const struct lu_env *env, struct cl_object *obj, struct cl_page *page, pgoff_t ind); @@ -413,7 +482,7 @@ void osc_io_unplug(const struct lu_env *env, struct client_obd *cli, /**************************************************************************** * - * Accessors. + * Accessors and type conversions. * */ @@ -440,14 +509,8 @@ static inline struct osc_io *osc_env_io(const struct lu_env *env) return &osc_env_session(env)->os_io; } -static inline int osc_is_object(const struct lu_object *obj) -{ - return obj->lo_dev->ld_type == &osc_device_type; -} - static inline struct osc_device *lu2osc_dev(const struct lu_device *d) { - LINVRNT(d->ld_type == &osc_device_type); return container_of(d, struct osc_device, od_cl.cd_lu_dev); } @@ -463,7 +526,6 @@ static inline struct client_obd *osc_cli(const struct osc_object *obj) static inline struct osc_object *cl2osc(const struct cl_object *obj) { - LINVRNT(osc_is_object(&obj->co_lu)); return container_of(obj, struct osc_object, oo_cl); } @@ -472,6 +534,36 @@ static inline struct cl_object *osc2cl(const struct osc_object *obj) return (struct cl_object *)&obj->oo_cl; } +static inline struct osc_device *obd2osc_dev(const struct obd_device *d) +{ + return container_of_safe(d->obd_lu_dev, struct osc_device, + od_cl.cd_lu_dev); +} + +static inline struct lu_device *osc2lu_dev(struct osc_device *osc) +{ + return &osc->od_cl.cd_lu_dev; +} + +static inline struct lu_object *osc2lu(struct osc_object *osc) +{ + return &osc->oo_cl.co_lu; +} + +static inline struct osc_object *lu2osc(const struct lu_object *obj) +{ + return container_of_safe(obj, struct osc_object, oo_cl.co_lu); +} + +static inline struct osc_io *cl2osc_io(const struct lu_env *env, + const struct cl_io_slice *slice) +{ + struct osc_io *oio = container_of_safe(slice, struct osc_io, oi_cl); + + LINVRNT(oio == osc_env_io(env)); + return oio; +} + static inline enum ldlm_mode osc_cl_lock2ldlm(enum cl_lock_mode mode) { LASSERT(mode == CLM_READ || mode == CLM_WRITE || mode == CLM_GROUP); @@ -496,7 +588,6 @@ static inline enum cl_lock_mode osc_ldlm2cl_lock(enum ldlm_mode mode) static inline struct osc_page *cl2osc_page(const struct cl_page_slice *slice) { - LINVRNT(osc_is_object(&slice->cpl_obj->co_lu)); return container_of(slice, struct osc_page, ops_cl); } @@ -532,15 +623,9 @@ static inline struct osc_page *oap2osc_page(struct osc_async_page *oap) static inline struct osc_lock *cl2osc_lock(const struct cl_lock_slice *slice) { - LINVRNT(osc_is_object(&slice->cls_obj->co_lu)); return container_of(slice, struct osc_lock, ols_cl); } -static inline struct osc_lock *osc_lock_at(const struct cl_lock *lock) -{ - return cl2osc_lock(cl_lock_at(lock, &osc_device_type)); -} - static inline int osc_io_srvlock(struct osc_io *oio) { return (oio->oi_lockless && !oio->oi_cl.cis_io->ci_no_srvlock); @@ -667,4 +752,4 @@ bool osc_page_gang_lookup(const struct lu_env *env, struct cl_io *io, osc_page_gang_cbt cb, void *cbdata); /* @} osc */ -#endif /* OSC_CL_INTERNAL_H */ +#endif /* LUSTRE_OSC_H */ diff --git a/fs/lustre/osc/lproc_osc.c b/fs/lustre/osc/lproc_osc.c index eb724f4..8e85863 100644 --- a/fs/lustre/osc/lproc_osc.c +++ b/fs/lustre/osc/lproc_osc.c @@ -38,6 +38,8 @@ #include #include #include +#include + #include "osc_internal.h" static ssize_t active_show(struct kobject *kobj, struct attribute *attr, diff --git a/fs/lustre/osc/osc_cache.c b/fs/lustre/osc/osc_cache.c index 46d578e..1d06654 100644 --- a/fs/lustre/osc/osc_cache.c +++ b/fs/lustre/osc/osc_cache.c @@ -38,7 +38,8 @@ #define DEBUG_SUBSYSTEM S_OSC -#include "osc_cl_internal.h" +#include + #include "osc_internal.h" static int extent_debug; /* set it to be true for more debug */ diff --git a/fs/lustre/osc/osc_dev.c b/fs/lustre/osc/osc_dev.c index 997fa35..080c32c 100644 --- a/fs/lustre/osc/osc_dev.c +++ b/fs/lustre/osc/osc_dev.c @@ -39,8 +39,9 @@ /* class_name2obd() */ #include +#include -#include "osc_cl_internal.h" +#include "osc_internal.h" /** \addtogroup osc * @{ @@ -91,17 +92,6 @@ struct lu_kmem_descr osc_caches[] = { /***************************************************************************** * - * Type conversions. - * - */ - -static struct lu_device *osc2lu_dev(struct osc_device *osc) -{ - return &osc->od_cl.cd_lu_dev; -} - -/***************************************************************************** - * * Osc device and device type functions. * */ diff --git a/fs/lustre/osc/osc_internal.h b/fs/lustre/osc/osc_internal.h index 95f5b93..af2f237 100644 --- a/fs/lustre/osc/osc_internal.h +++ b/fs/lustre/osc/osc_internal.h @@ -36,70 +36,15 @@ #define OAP_MAGIC 8675309 +#include + extern atomic_t osc_pool_req_count; extern unsigned int osc_reqpool_maxreqcount; extern struct ptlrpc_request_pool *osc_rq_pool; -struct lu_env; - -enum async_flags { - ASYNC_READY = 0x1, /* ap_make_ready will not be - * called before this page is - * added to an rpc - */ - ASYNC_URGENT = 0x2, /* page must be put into an RPC - * before return - */ - ASYNC_COUNT_STABLE = 0x4, /* ap_refresh_count will not be - * called to give the caller a - * chance to update or cancel - * the size of the io - */ - ASYNC_HP = 0x10, -}; - -struct osc_async_page { - int oap_magic; - unsigned short oap_cmd; - - struct list_head oap_pending_item; - struct list_head oap_rpc_item; - - u64 oap_obj_off; - unsigned int oap_page_off; - enum async_flags oap_async_flags; - - struct brw_page oap_brw_page; - - struct ptlrpc_request *oap_request; - struct client_obd *oap_cli; - struct osc_object *oap_obj; - - spinlock_t oap_lock; -}; - -#define oap_page oap_brw_page.pg -#define oap_count oap_brw_page.count -#define oap_brw_flags oap_brw_page.flag - -static inline struct osc_async_page *brw_page2oap(struct brw_page *pga) -{ - return container_of(pga, struct osc_async_page, oap_brw_page); -} - -static inline void osc_wake_cache_waiters(struct client_obd *cli) -{ - wake_up(&cli->cl_cache_waiters); -} - int osc_shrink_grant_to_target(struct client_obd *cli, u64 target_bytes); void osc_update_next_shrink(struct client_obd *cli); -/* - * cl integration. - */ -#include - typedef int (*osc_enqueue_upcall_f)(void *cookie, struct lustre_handle *lockh, int rc); @@ -145,6 +90,24 @@ long osc_lru_shrink(const struct lu_env *env, struct client_obd *cli, extern struct lu_device_type osc_device_type; +static inline int osc_is_object(const struct lu_object *obj) +{ + return obj->lo_dev->ld_type == &osc_device_type; +} + +static inline struct osc_lock *osc_lock_at(const struct cl_lock *lock) +{ + return cl2osc_lock(cl_lock_at(lock, &osc_device_type)); +} + +int osc_lock_init(const struct lu_env *env, struct cl_object *obj, + struct cl_lock *lock, const struct cl_io *io); +int osc_io_init(const struct lu_env *env, struct cl_object *obj, + struct cl_io *io); +struct lu_object *osc_object_alloc(const struct lu_env *env, + const struct lu_object_header *hdr, + struct lu_device *dev); + static inline int osc_recoverable_error(int rc) { return (rc == -EIO || rc == -EROFS || rc == -ENOMEM || @@ -161,38 +124,8 @@ static inline char *cli_name(struct client_obd *cli) return cli->cl_import->imp_obd->obd_name; } -struct osc_device { - struct cl_device od_cl; - struct obd_export *od_exp; - - /* Write stats is actually protected by client_obd's lock. */ - struct osc_stats { - u64 os_lockless_writes; /* by bytes */ - u64 os_lockless_reads; /* by bytes */ - u64 os_lockless_truncates; /* by times */ - } od_stats; - - /* configuration item(s) */ - int od_contention_time; - int od_lockless_truncate; -}; - -static inline struct osc_device *obd2osc_dev(const struct obd_device *d) -{ - return container_of_safe(d->obd_lu_dev, struct osc_device, - od_cl.cd_lu_dev); -} - extern struct lu_kmem_descr osc_caches[]; -extern struct kmem_cache *osc_quota_kmem; -struct osc_quota_info { - /* linkage for quota hash table */ - struct rhash_head oqi_hash; - u32 oqi_id; - struct rcu_head rcu; -}; - int osc_quota_setup(struct obd_device *obd); int osc_quota_cleanup(struct obd_device *obd); int osc_quota_setdq(struct client_obd *cli, const unsigned int qid[], diff --git a/fs/lustre/osc/osc_io.c b/fs/lustre/osc/osc_io.c index 0b9ed01..f7169e9 100644 --- a/fs/lustre/osc/osc_io.c +++ b/fs/lustre/osc/osc_io.c @@ -39,8 +39,9 @@ #define DEBUG_SUBSYSTEM S_OSC #include +#include -#include "osc_cl_internal.h" +#include "osc_internal.h" /** \addtogroup osc * @{ @@ -48,21 +49,6 @@ /***************************************************************************** * - * Type conversions. - * - */ - -static struct osc_io *cl2osc_io(const struct lu_env *env, - const struct cl_io_slice *slice) -{ - struct osc_io *oio = container_of_safe(slice, struct osc_io, oi_cl); - - LINVRNT(oio == osc_env_io(env)); - return oio; -} - -/***************************************************************************** - * * io operations. * */ diff --git a/fs/lustre/osc/osc_lock.c b/fs/lustre/osc/osc_lock.c index 65b734ea..917ae71 100644 --- a/fs/lustre/osc/osc_lock.c +++ b/fs/lustre/osc/osc_lock.c @@ -40,8 +40,9 @@ /* fid_build_reg_res_name() */ #include +#include -#include "osc_cl_internal.h" +#include "osc_internal.h" /** \addtogroup osc * @{ diff --git a/fs/lustre/osc/osc_object.c b/fs/lustre/osc/osc_object.c index 7468729..ffa31d5 100644 --- a/fs/lustre/osc/osc_object.c +++ b/fs/lustre/osc/osc_object.c @@ -37,8 +37,9 @@ */ #define DEBUG_SUBSYSTEM S_OSC +#include -#include "osc_cl_internal.h" +#include "osc_internal.h" /** \addtogroup osc * @{ @@ -46,23 +47,6 @@ /***************************************************************************** * - * Type conversions. - * - */ - -static struct lu_object *osc2lu(struct osc_object *osc) -{ - return &osc->oo_cl.co_lu; -} - -static struct osc_object *lu2osc(const struct lu_object *obj) -{ - LINVRNT(osc_is_object(obj)); - return container_of(obj, struct osc_object, oo_cl.co_lu); -} - -/***************************************************************************** - * * Object operations. * */ diff --git a/fs/lustre/osc/osc_page.c b/fs/lustre/osc/osc_page.c index 8d541df..3576b34 100644 --- a/fs/lustre/osc/osc_page.c +++ b/fs/lustre/osc/osc_page.c @@ -39,7 +39,9 @@ #define DEBUG_SUBSYSTEM S_OSC #include -#include "osc_cl_internal.h" +#include + +#include "osc_internal.h" static void osc_lru_del(struct client_obd *cli, struct osc_page *opg); static void osc_lru_use(struct client_obd *cli, struct osc_page *opg); diff --git a/fs/lustre/osc/osc_quota.c b/fs/lustre/osc/osc_quota.c index 5fa85c6..3909e11 100644 --- a/fs/lustre/osc/osc_quota.c +++ b/fs/lustre/osc/osc_quota.c @@ -25,6 +25,8 @@ */ #include +#include + #include "osc_internal.h" static const struct rhashtable_params quota_hash_params = { diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index a64b8ab..64c4fd1 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -50,8 +50,9 @@ #include #include #include +#include + #include "osc_internal.h" -#include "osc_cl_internal.h" atomic_t osc_pool_req_count; unsigned int osc_reqpool_maxreqcount; From patchwork Mon Sep 30 18:54:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167201 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2B07116C1 for ; Mon, 30 Sep 2019 19:03:11 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 12F66224F1 for ; Mon, 30 Sep 2019 19:03:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 12F66224F1 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 0F56B5E45B2; Mon, 30 Sep 2019 11:59:36 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 3C7C65C340D for ; Mon, 30 Sep 2019 11:57:04 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 5EBFA100539C; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 5C7FBB5; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:38 -0400 Message-Id: <1569869810-23848-20-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 019/151] lustre: statahead: support striped directory X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lai Siyao , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Lai Siyao Current statahead will getattr by fid, but for striped directory, it needs to fix its parent fid to stripe fid, which needs file name, so pack file name in md_op_data. Remove check for remote dir check in lmv_intent_getattr_async(), which is not necessary, and not checking it won't cause anything wrong, while the code is simpler. WC-bug-id: https://jira.whamcloud.com/browse/LU-5106 Lustre-commit: 9e7952c045a3 ("LU-5106 statahead: support striped directory") Signed-off-by: Lai Siyao Reviewed-on: https://review.whamcloud.com/27683 Reviewed-by: Andreas Dilger Reviewed-by: Fan Yong Reviewed-by: wangdi Signed-off-by: James Simmons --- fs/lustre/llite/statahead.c | 3 ++- fs/lustre/lmv/lmv_obd.c | 27 +++++++++++---------------- 2 files changed, 13 insertions(+), 17 deletions(-) diff --git a/fs/lustre/llite/statahead.c b/fs/lustre/llite/statahead.c index 797ad06..8fc4471 100644 --- a/fs/lustre/llite/statahead.c +++ b/fs/lustre/llite/statahead.c @@ -749,7 +749,8 @@ static void sa_fini_data(struct md_enqueue_info *minfo) if (!minfo) return ERR_PTR(-ENOMEM); - op_data = ll_prep_md_op_data(&minfo->mi_data, dir, child, NULL, 0, 0, + op_data = ll_prep_md_op_data(&minfo->mi_data, dir, child, + entry->se_qstr.name, entry->se_qstr.len, 0, LUSTRE_OPC_ANY, NULL); if (IS_ERR(op_data)) { kfree(minfo); diff --git a/fs/lustre/lmv/lmv_obd.c b/fs/lustre/lmv/lmv_obd.c index acef43d..946c757 100644 --- a/fs/lustre/lmv/lmv_obd.c +++ b/fs/lustre/lmv/lmv_obd.c @@ -2904,30 +2904,25 @@ static int lmv_intent_getattr_async(struct obd_export *exp, struct md_op_data *op_data = &minfo->mi_data; struct obd_device *obd = exp->exp_obd; struct lmv_obd *lmv = &obd->u.lmv; - struct lmv_tgt_desc *ptgt = NULL; - struct lmv_tgt_desc *ctgt = NULL; + struct lmv_tgt_desc *tgt = NULL; if (!fid_is_sane(&op_data->op_fid2)) return -EINVAL; - ptgt = lmv_locate_mds(lmv, op_data, &op_data->op_fid1); - if (IS_ERR(ptgt)) - return PTR_ERR(ptgt); - - ctgt = lmv_locate_mds(lmv, op_data, &op_data->op_fid2); - if (IS_ERR(ctgt)) - return PTR_ERR(ctgt); + tgt = lmv_locate_mds(lmv, op_data, &op_data->op_fid1); + if (IS_ERR(tgt)) + return PTR_ERR(tgt); /* - * if child is on remote MDT, we need 2 async RPCs to fetch both LOOKUP - * lock on parent, and UPDATE lock on child MDT, which makes all - * complicated. Considering remote dir is rare case, and not supporting - * it in statahead won't cause any issue, drop its support for now. + * no special handle for remote dir, which needs to fetch both LOOKUP + * lock on parent, and then UPDATE lock on child MDT, which makes all + * complicated because this is done async. So only LOOKUP lock is + * fetched for remote dir, but considering remote dir is rare case, + * and not supporting it in statahead won't cause any issue, just leave + * it as is. */ - if (ptgt != ctgt) - return -ENOTSUPP; - return md_intent_getattr_async(ptgt->ltd_exp, minfo); + return md_intent_getattr_async(tgt->ltd_exp, minfo); } static int lmv_revalidate_lock(struct obd_export *exp, struct lookup_intent *it, From patchwork Mon Sep 30 18:54:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167061 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D08E13BD for ; Mon, 30 Sep 2019 18:57:33 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 74D9F224D5 for ; Mon, 30 Sep 2019 18:57:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 74D9F224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 153785C3B91; Mon, 30 Sep 2019 11:57:19 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 935295C340D for ; Mon, 30 Sep 2019 11:57:04 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 61A2B100539D; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 5FB54A9; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:39 -0400 Message-Id: <1569869810-23848-21-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 020/151] lustre: readdir: improve striped readdir X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lai Siyao , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Lai Siyao Striped directory needs to build its directory page from sub-stripe directory pages, current code iterate sub stripe directory pages to search dirent for each hash, this is inefficient, as may cause statahead fail because statahead thread is slow in readdir, while 'ls' is faster and can't find cached statahead entries, and finally cause statahead fail. This patch introduces a struct lmv_dir_ctxt which saves dir page and current dirent for all stripes, to find the dirent which has the closest hash value it only needs to compare the dirent of all stripes, and then pop this dirent from its stripe, until all stripes reache the end. This patch contains another fix: previously LDP_COLLIDE is set by default, change to set dir page end hash 'ldp_hash_end' to hash of next dirent, and only set LDP_COLLIDE when 'ldp_hash_end' equals last dirent hash 'lde_hash'. We should avoid dir hash collision. WC-bug-id: https://jira.whamcloud.com/browse/LU-5106 Lustre-commit: 98fc9a77446a ("LU-5106 readdir: improve striped readdir") Signed-off-by: Lai Siyao Reviewed-on: https://review.whamcloud.com/27663 Reviewed-by: Fan Yong Reviewed-by: wangdi Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/lmv/lmv_obd.c | 423 ++++++++++++++++++++++++++---------------------- 1 file changed, 231 insertions(+), 192 deletions(-) diff --git a/fs/lustre/lmv/lmv_obd.c b/fs/lustre/lmv/lmv_obd.c index 946c757..d71d077 100644 --- a/fs/lustre/lmv/lmv_obd.c +++ b/fs/lustre/lmv/lmv_obd.c @@ -2001,60 +2001,79 @@ static int lmv_fsync(struct obd_export *exp, const struct lu_fid *fid, return md_fsync(tgt->ltd_exp, fid, request); } -/** - * Get current minimum entry from striped directory - * - * This function will search the dir entry, whose hash value is the - * closest(>=) to @hash_offset, from all of sub-stripes, and it is - * only being called for striped directory. - * - * @exp: export of LMV - * @op_data: parameters transferred beween client MD stack - * stripe_information will be included in this - * parameter - * @cb_op: ldlm callback being used in enqueue in - * mdc_read_page - * @hash_offset: the hash value, which is used to locate - * minum(closet) dir entry - * @stripe_offset: the caller use this to indicate the stripe - * index of last entry, so to avoid hash conflict - * between stripes. It will also be used to - * return the stripe index of current dir entry. - * @entp: the minum entry and it also is being used - * to input the last dir entry to resolve the - * hash conflict - * - * @ppage: the page which holds the minum entry - * - * Return: = 0 get the entry successfully - * negative errno (< 0) does not get the entry - */ -static int lmv_get_min_striped_entry(struct obd_export *exp, - struct md_op_data *op_data, - struct md_callback *cb_op, - u64 hash_offset, int *stripe_offset, - struct lu_dirent **entp, - struct page **ppage) +struct stripe_dirent { + struct page *sd_page; + struct lu_dirpage *sd_dp; + struct lu_dirent *sd_ent; + bool sd_eof; +}; + +struct lmv_dir_ctxt { + struct lmv_obd *ldc_lmv; + struct md_op_data *ldc_op_data; + struct md_callback *ldc_cb_op; + u64 ldc_hash; + int ldc_count; + struct stripe_dirent ldc_stripes[0]; +}; + +static inline void put_stripe_dirent(struct stripe_dirent *stripe) +{ + if (stripe->sd_page) { + kunmap(stripe->sd_page); + put_page(stripe->sd_page); + stripe->sd_page = NULL; + } +} + +static inline void put_lmv_dir_ctxt(struct lmv_dir_ctxt *ctxt) { - struct lmv_stripe_md *lsm = op_data->op_mea1; - struct obd_device *obd = exp->exp_obd; - struct lmv_obd *lmv = &obd->u.lmv; - struct lu_dirent *min_ent = NULL; - struct page *min_page = NULL; - struct lmv_tgt_desc *tgt; - int stripe_count; - int min_idx = 0; - int rc = 0; int i; - stripe_count = lsm->lsm_md_stripe_count; - for (i = 0; i < stripe_count; i++) { - u64 stripe_hash = hash_offset; - struct lu_dirent *ent = NULL; - struct page *page = NULL; - struct lu_dirpage *dp; + for (i = 0; i < ctxt->ldc_count; i++) + put_stripe_dirent(&ctxt->ldc_stripes[i]); +} + +static struct lu_dirent *stripe_dirent_next(struct lmv_dir_ctxt *ctxt, + struct stripe_dirent *stripe, + int stripe_index) +{ + struct lu_dirent *ent = stripe->sd_ent; + u64 hash = ctxt->ldc_hash; + u64 end; + int rc = 0; + + LASSERT(stripe == &ctxt->ldc_stripes[stripe_index]); + + if (ent) { + ent = lu_dirent_next(ent); + if (!ent) { +check_eof: + end = le64_to_cpu(stripe->sd_dp->ldp_hash_end); + + put_stripe_dirent(stripe); + + if (end == MDS_DIR_END_OFF) { + stripe->sd_ent = NULL; + stripe->sd_eof = true; + return NULL; + } + LASSERT(hash <= end); + hash = end; + } + } + + if (!ent) { + struct md_op_data *op_data = ctxt->ldc_op_data; + struct lmv_oinfo *oinfo; + struct lu_fid fid = op_data->op_fid1; + struct inode *inode = op_data->op_data; + struct lmv_tgt_desc *tgt; - tgt = lmv_get_target(lmv, lsm->lsm_md_oinfo[i].lmo_mds, NULL); + LASSERT(!stripe->sd_page); + + oinfo = &op_data->op_mea1->lsm_md_oinfo[stripe_index]; + tgt = lmv_get_target(ctxt->ldc_lmv, oinfo->lmo_mds, NULL); if (IS_ERR(tgt)) { rc = PTR_ERR(tgt); goto out; @@ -2064,87 +2083,117 @@ static int lmv_get_min_striped_entry(struct obd_export *exp, * op_data will be shared by each stripe, so we need * reset these value for each stripe */ - op_data->op_fid1 = lsm->lsm_md_oinfo[i].lmo_fid; - op_data->op_fid2 = lsm->lsm_md_oinfo[i].lmo_fid; - op_data->op_data = lsm->lsm_md_oinfo[i].lmo_root; -next: - rc = md_read_page(tgt->ltd_exp, op_data, cb_op, stripe_hash, - &page); + op_data->op_fid1 = oinfo->lmo_fid; + op_data->op_fid2 = oinfo->lmo_fid; + op_data->op_data = oinfo->lmo_root; + + rc = md_read_page(tgt->ltd_exp, op_data, ctxt->ldc_cb_op, hash, + &stripe->sd_page); + + op_data->op_fid1 = fid; + op_data->op_fid2 = fid; + op_data->op_data = inode; + if (rc) goto out; - dp = page_address(page); - for (ent = lu_dirent_start(dp); ent; - ent = lu_dirent_next(ent)) { - /* Skip dummy entry */ - if (!le16_to_cpu(ent->lde_namelen)) - continue; + stripe->sd_dp = page_address(stripe->sd_page); + ent = lu_dirent_start(stripe->sd_dp); + } - if (le64_to_cpu(ent->lde_hash) < hash_offset) - continue; + for (; ent; ent = lu_dirent_next(ent)) { + /* Skip dummy entry */ + if (!le16_to_cpu(ent->lde_namelen)) + continue; - if (le64_to_cpu(ent->lde_hash) == hash_offset && - (*entp == ent || i < *stripe_offset)) - continue; + /* skip . and .. for other stripes */ + if (stripe_index && + (strncmp(ent->lde_name, ".", + le16_to_cpu(ent->lde_namelen)) == 0 || + strncmp(ent->lde_name, "..", + le16_to_cpu(ent->lde_namelen)) == 0)) + continue; - /* skip . and .. for other stripes */ - if (i && (!strncmp(ent->lde_name, ".", - le16_to_cpu(ent->lde_namelen)) || - !strncmp(ent->lde_name, "..", - le16_to_cpu(ent->lde_namelen)))) - continue; - break; - } + if (le64_to_cpu(ent->lde_hash) < hash) + continue; - if (!ent) { - stripe_hash = le64_to_cpu(dp->ldp_hash_end); + break; + } - kunmap(page); - put_page(page); - page = NULL; + if (!ent) + goto check_eof; - /* - * reach the end of current stripe, go to next stripe - */ - if (stripe_hash == MDS_DIR_END_OFF) +out: + stripe->sd_ent = ent; + /* treat error as eof, so dir can be partially accessed */ + if (rc) { + put_stripe_dirent(stripe); + stripe->sd_eof = true; + LCONSOLE_WARN("dir " DFID " stripe %d readdir failed: %d, directory is partially accessed!\n", + PFID(&ctxt->ldc_op_data->op_fid1), stripe_index, + rc); + } + return ent; +} + +/** + * Get dirent with the closest hash for striped directory + * + * This function will search the dir entry, whose hash value is the + * closest(>=) to hash from all of sub-stripes, and it is only being called + * for striped directory. + * + * @param ctxt dir read context + * + * Return: dirent get the entry successfully NULL does not get + * the entry, normally it means it reaches the end of + * the directory, while read stripe dirent error is + * ignored to allow partial access. + */ +static struct lu_dirent *lmv_dirent_next(struct lmv_dir_ctxt *ctxt) +{ + struct stripe_dirent *stripe; + struct lu_dirent *ent = NULL; + int i; + int min = -1; + + /* TODO: optimize with k-way merge sort */ + for (i = 0; i < ctxt->ldc_count; i++) { + stripe = &ctxt->ldc_stripes[i]; + if (stripe->sd_eof) + continue; + + if (!stripe->sd_ent) { + /* locate starting entry */ + stripe_dirent_next(ctxt, stripe, i); + if (!stripe->sd_ent) { + LASSERT(stripe->sd_eof); continue; - else - goto next; + } } - if (min_ent) { - if (le64_to_cpu(min_ent->lde_hash) > - le64_to_cpu(ent->lde_hash)) { - min_ent = ent; - kunmap(min_page); - put_page(min_page); - min_idx = i; - min_page = page; - } else { - kunmap(page); - put_page(page); - page = NULL; - } - } else { - min_ent = ent; - min_page = page; - min_idx = i; + if (min == -1 || + le64_to_cpu(ctxt->ldc_stripes[min].sd_ent->lde_hash) > + le64_to_cpu(stripe->sd_ent->lde_hash)) { + min = i; + if (le64_to_cpu(stripe->sd_ent->lde_hash) == + ctxt->ldc_hash) + break; } } -out: - if (*ppage) { - kunmap(*ppage); - put_page(*ppage); + if (min != -1) { + stripe = &ctxt->ldc_stripes[min]; + ent = stripe->sd_ent; + /* pop found dirent */ + stripe_dirent_next(ctxt, stripe, min); } - *stripe_offset = min_idx; - *entp = min_ent; - *ppage = min_page; - return rc; + + return ent; } /** - * Build dir entry page from a striped directory + * Build dir entry page for striped directory * * This function gets one entry by @offset from a striped directory. It will * read entries from all of stripes, and choose one closest to the required @@ -2153,12 +2202,11 @@ static int lmv_get_min_striped_entry(struct obd_export *exp, * and .. in a directory. * 2. op_data will be shared by all of stripes, instead of allocating new * one, so need to restore before reusing. - * 3. release the entry page if that is not being chosen. * * @exp: obd export refer to LMV * @op_data: hold those MD parameters of read_entry * @cb_op: ldlm callback being used in enqueue in mdc_read_entry - * @ldp: the entry being read + * @offset: the entry being read * @ppage: the page holding the entry. Note: because the entry * will be accessed in upper layer, so we need hold the * page until the usages of entry is finished, see @@ -2167,81 +2215,80 @@ static int lmv_get_min_striped_entry(struct obd_export *exp, * Returns: =0 if get entry successfully * <0 cannot get entry */ -static int lmv_read_striped_page(struct obd_export *exp, +static int lmv_striped_read_page(struct obd_export *exp, struct md_op_data *op_data, struct md_callback *cb_op, u64 offset, struct page **ppage) { - struct inode *master_inode = op_data->op_data; - struct lu_fid master_fid = op_data->op_fid1; - u64 hash_offset = offset; - u32 ldp_flags; - struct page *min_ent_page = NULL; - struct page *ent_page = NULL; - struct lu_dirent *min_ent = NULL; - struct lu_dirent *last_ent; - struct lu_dirent *ent; + struct page *page = NULL; struct lu_dirpage *dp; + void *start; + struct lu_dirent *ent; + struct lu_dirent *last_ent; + int stripe_count; + struct lmv_dir_ctxt *ctxt; + struct lu_dirent *next = NULL; + u16 ent_size; size_t left_bytes; - int ent_idx = 0; - void *area; - int rc; + int rc = 0; /* * Allocate a page and read entries from all of stripes and fill * the page by hash order */ - ent_page = alloc_page(GFP_KERNEL); - if (!ent_page) + page = alloc_page(GFP_KERNEL); + if (!page) return -ENOMEM; /* Initialize the entry page */ - dp = kmap(ent_page); + dp = kmap(page); memset(dp, 0, sizeof(*dp)); dp->ldp_hash_start = cpu_to_le64(offset); - ldp_flags = LDF_COLLIDE; - area = dp + 1; + start = dp + 1; left_bytes = PAGE_SIZE - sizeof(*dp); - ent = area; + ent = start; last_ent = ent; - do { - u16 ent_size; - - /* Find the minum entry from all sub-stripes */ - rc = lmv_get_min_striped_entry(exp, op_data, cb_op, hash_offset, - &ent_idx, &min_ent, - &min_ent_page); - if (rc) - goto out; - /* - * If it can not get minum entry, it means it already reaches - * the end of this directory - */ - if (!min_ent) { - last_ent->lde_reclen = 0; - hash_offset = MDS_DIR_END_OFF; - goto out; + /* initialize dir read context */ + stripe_count = op_data->op_mea1->lsm_md_stripe_count; + ctxt = kzalloc(offsetof(typeof(*ctxt), ldc_stripes[stripe_count]), + GFP_NOFS); + if (!ctxt) { + rc = -ENOMEM; + goto free_page; + } + ctxt->ldc_lmv = &exp->exp_obd->u.lmv; + ctxt->ldc_op_data = op_data; + ctxt->ldc_cb_op = cb_op; + ctxt->ldc_hash = offset; + ctxt->ldc_count = stripe_count; + + while (1) { + next = lmv_dirent_next(ctxt); + + /* end of directory */ + if (!next) { + ctxt->ldc_hash = MDS_DIR_END_OFF; + break; } + ctxt->ldc_hash = le64_to_cpu(next->lde_hash); - ent_size = le16_to_cpu(min_ent->lde_reclen); + ent_size = le16_to_cpu(next->lde_reclen); /* - * the last entry lde_reclen is 0, but it might not - * the end of this entry of this temporay entry + * the last entry lde_reclen is 0, but it might not be the last + * one of this temporay dir page */ if (!ent_size) ent_size = lu_dirent_calc_size( - le16_to_cpu(min_ent->lde_namelen), - le32_to_cpu(min_ent->lde_attrs)); - if (ent_size > left_bytes) { - last_ent->lde_reclen = cpu_to_le16(0); - hash_offset = le64_to_cpu(min_ent->lde_hash); - goto out; - } + le16_to_cpu(next->lde_namelen), + le32_to_cpu(next->lde_attrs)); + /* page full */ + if (ent_size > left_bytes) + break; - memcpy(ent, min_ent, ent_size); + memcpy(ent, next, ent_size); /* * Replace . with master FID and Replace .. with the parent FID @@ -2250,49 +2297,41 @@ static int lmv_read_striped_page(struct obd_export *exp, if (!strncmp(ent->lde_name, ".", le16_to_cpu(ent->lde_namelen)) && le16_to_cpu(ent->lde_namelen) == 1) - fid_cpu_to_le(&ent->lde_fid, &master_fid); + fid_cpu_to_le(&ent->lde_fid, &op_data->op_fid1); else if (!strncmp(ent->lde_name, "..", le16_to_cpu(ent->lde_namelen)) && le16_to_cpu(ent->lde_namelen) == 2) fid_cpu_to_le(&ent->lde_fid, &op_data->op_fid3); + CDEBUG(D_INODE, "entry %.*s hash %#llx\n", + le16_to_cpu(ent->lde_namelen), ent->lde_name, + le64_to_cpu(ent->lde_hash)); + left_bytes -= ent_size; ent->lde_reclen = cpu_to_le16(ent_size); last_ent = ent; ent = (void *)ent + ent_size; - hash_offset = le64_to_cpu(min_ent->lde_hash); - if (hash_offset == MDS_DIR_END_OFF) { - last_ent->lde_reclen = 0; - break; - } - } while (1); -out: - if (min_ent_page) { - kunmap(min_ent_page); - put_page(min_ent_page); } - if (unlikely(rc)) { - __free_page(ent_page); - ent_page = NULL; - } else { - if (ent == area) - ldp_flags |= LDF_EMPTY; - dp->ldp_flags |= cpu_to_le32(ldp_flags); - dp->ldp_hash_end = cpu_to_le64(hash_offset); - } + last_ent->lde_reclen = 0; - /* - * We do not want to allocate md_op_data during each - * dir entry reading, so op_data will be shared by every stripe, - * then we need to restore it back to original value before - * return to the upper layer - */ - op_data->op_fid1 = master_fid; - op_data->op_fid2 = master_fid; - op_data->op_data = master_inode; + if (ent == start) + dp->ldp_flags |= LDF_EMPTY; + else if (ctxt->ldc_hash == le64_to_cpu(last_ent->lde_hash)) + dp->ldp_flags |= LDF_COLLIDE; + dp->ldp_flags = cpu_to_le32(dp->ldp_flags); + dp->ldp_hash_end = cpu_to_le64(ctxt->ldc_hash); + + put_lmv_dir_ctxt(ctxt); + kfree(ctxt); + + *ppage = page; + + return 0; - *ppage = ent_page; +free_page: + kunmap(page); + __free_page(page); return rc; } @@ -2307,7 +2346,7 @@ static int lmv_read_page(struct obd_export *exp, struct md_op_data *op_data, struct lmv_tgt_desc *tgt; if (unlikely(lsm)) { - return lmv_read_striped_page(exp, op_data, cb_op, + return lmv_striped_read_page(exp, op_data, cb_op, offset, ppage); } From patchwork Mon Sep 30 18:54:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167113 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3460A16B1 for ; Mon, 30 Sep 2019 18:59:12 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1C9BD224D5 for ; Mon, 30 Sep 2019 18:59:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1C9BD224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 904945C3C7F; Mon, 30 Sep 2019 11:58:10 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id E91725C336D for ; Mon, 30 Sep 2019 11:57:04 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 63980100539E; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 62A4CBB; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:40 -0400 Message-Id: <1569869810-23848-22-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 021/151] lustre: llog: consolidate common error checking X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Andreas Dilger Consolidate error checking for corrupted and empty llog files detected when the llog file is being opened into a single routine llog_cat_process_common(). WC-bug-id: https://jira.whamcloud.com/browse/LU-9153 Lustre-commit: b522f7b4b1de ("LU-9153 llog: consolidate common error checking") Signed-off-by: Andreas Dilger Reviewed-on: https://review.whamcloud.com/26211 Reviewed-by: Faccini Bruno Reviewed-by: Mike Pershin Reviewed-by: Oleg Drokin Signed-off-by: James Simmons Acked-by: James Simmons --- fs/lustre/obdclass/llog_cat.c | 48 ++++++++++++++++++++++++++++++++----------- 1 file changed, 36 insertions(+), 12 deletions(-) diff --git a/fs/lustre/obdclass/llog_cat.c b/fs/lustre/obdclass/llog_cat.c index b61c858..4536f54 100644 --- a/fs/lustre/obdclass/llog_cat.c +++ b/fs/lustre/obdclass/llog_cat.c @@ -140,32 +140,55 @@ int llog_cat_close(const struct lu_env *env, struct llog_handle *cathandle) } EXPORT_SYMBOL(llog_cat_close); -static int llog_cat_process_cb(const struct lu_env *env, - struct llog_handle *cat_llh, - struct llog_rec_hdr *rec, void *data) +static int llog_cat_process_common(const struct lu_env *env, + struct llog_handle *cat_llh, + struct llog_rec_hdr *rec, + struct llog_handle **llhp) { - struct llog_process_data *d = data; - struct llog_logid_rec *lir = (struct llog_logid_rec *)rec; - struct llog_handle *llh; + struct llog_logid_rec *lir = container_of(rec, typeof(*lir), lid_hdr); int rc; - if (rec->lrh_type != LLOG_LOGID_MAGIC) { + if (rec->lrh_type != le32_to_cpu(LLOG_LOGID_MAGIC)) { + rc = -EINVAL; + CWARN("%s: invalid record in catalog " DFID ":%x: rc = %d\n", + cat_llh->lgh_ctxt->loc_obd->obd_name, + PFID(&cat_llh->lgh_id.lgl_oi.oi_fid), + cat_llh->lgh_id.lgl_ogen, rc); + CERROR("invalid record in catalog\n"); return -EINVAL; } CDEBUG(D_HA, "processing log " DFID ":%x at index %u of catalog " DFID "\n", PFID(&lir->lid_id.lgl_oi.oi_fid), lir->lid_id.lgl_ogen, - rec->lrh_index, PFID(&cat_llh->lgh_id.lgl_oi.oi_fid)); + le32_to_cpu(rec->lrh_index), + PFID(&cat_llh->lgh_id.lgl_oi.oi_fid)); - rc = llog_cat_id2handle(env, cat_llh, &llh, &lir->lid_id); + rc = llog_cat_id2handle(env, cat_llh, llhp, &lir->lid_id); if (rc) { - CERROR("%s: cannot find handle for llog " DFID ": %d\n", - cat_llh->lgh_ctxt->loc_obd->obd_name, - PFID(&lir->lid_id.lgl_oi.oi_fid), rc); + CWARN("%s: can't find llog handle " DFID ":%x: rc = %d\n", + cat_llh->lgh_ctxt->loc_obd->obd_name, + PFID(&lir->lid_id.lgl_oi.oi_fid), + lir->lid_id.lgl_ogen, rc); + return rc; } + return rc; +} + +static int llog_cat_process_cb(const struct lu_env *env, + struct llog_handle *cat_llh, + struct llog_rec_hdr *rec, void *data) +{ + struct llog_process_data *d = data; + struct llog_handle *llh = NULL; + int rc; + + rc = llog_cat_process_common(env, cat_llh, rec, &llh); + if (rc) + goto out; + if (rec->lrh_index < d->lpd_startcat) /* Skip processing of the logs until startcat */ rc = 0; @@ -183,6 +206,7 @@ static int llog_cat_process_cb(const struct lu_env *env, NULL, false); } +out: llog_handle_put(llh); return rc; From patchwork Mon Sep 30 18:54:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167117 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AAFF613BD for ; Mon, 30 Sep 2019 18:59:19 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 93AAC224D5 for ; Mon, 30 Sep 2019 18:59:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 93AAC224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id F11065C3EDB; Mon, 30 Sep 2019 11:58:13 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 4AEE35C336D for ; Mon, 30 Sep 2019 11:57:05 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 680DC100539F; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 66CC0B4; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:41 -0400 Message-Id: <1569869810-23848-23-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 022/151] lustre: llite: NULL pointer dereference in cl_object_top() X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andriy Skulysh , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Andriy Skulysh During race condition iget5_locked() returns inode without I_NEW bit for the second thread. If cl_file_inode_init() failed for the first thread lli_clob isn't valid for corresponding fid. But failed inode is marked bad, check it. WC-bug-id: https://jira.whamcloud.com/browse/LU-6436 Seagate-bug-id: MRP-4035 Lustre-commit: 13c8d5e4bebf ("LU-6436 llite: NULL pointer dereference in cl_object_top()") Signed-off-by: Andriy Skulysh Reviewed-on: https://review.whamcloud.com/27777 Reviewed-by: Bobi Jam Reviewed-by: Jinshan Xiong Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/namei.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/lustre/llite/namei.c b/fs/lustre/llite/namei.c index caa759f..9c9acf2 100644 --- a/fs/lustre/llite/namei.c +++ b/fs/lustre/llite/namei.c @@ -133,6 +133,9 @@ struct inode *ll_iget(struct super_block *sb, ino_t hash, inode_has_no_xattr(inode); unlock_new_inode(inode); } + } else if (is_bad_inode(inode)) { + iput(inode); + inode = ERR_PTR(-ESTALE); } else if (!(inode->i_state & (I_FREEING | I_CLEAR))) { rc = ll_update_inode(inode, md); CDEBUG(D_VFSTRACE, "got inode: " DFID "(%p): rc = %d\n", From patchwork Mon Sep 30 18:54:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167121 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B383816B1 for ; Mon, 30 Sep 2019 18:59:26 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9BC84224D5 for ; Mon, 30 Sep 2019 18:59:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9BC84224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 9A9215C3F0C; Mon, 30 Sep 2019 11:58:17 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8B5675C39A1 for ; Mon, 30 Sep 2019 11:57:05 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 6B5A21005406; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 6A184B5; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:42 -0400 Message-Id: <1569869810-23848-24-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 023/151] lustre: ptlrpc: remove incorrect pid printing X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Sebastien Buisson Remove incorrect pid printing in ptlrpc_send_new_req(). It was always printing 0, and current pid is part of the message header anyway. WC-bug-id: https://jira.whamcloud.com/browse/LU-7760 Lustre-commit: 3283170eeb05 ("LU-7760 ptlrpc: remove incorrect pid printing") Signed-off-by: Sebastien Buisson Reviewed-on: https://review.whamcloud.com/27336 Reviewed-by: Andreas Dilger Reviewed-by: James Simmons Reviewed-by: Fan Yong Signed-off-by: James Simmons --- fs/lustre/ptlrpc/client.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/fs/lustre/ptlrpc/client.c b/fs/lustre/ptlrpc/client.c index f7f3678..ba894c0 100644 --- a/fs/lustre/ptlrpc/client.c +++ b/fs/lustre/ptlrpc/client.c @@ -1551,8 +1551,7 @@ static int ptlrpc_send_new_req(struct ptlrpc_request *req) spin_unlock(&req->rq_lock); DEBUG_REQ(D_HA, req, - "req from PID %d waiting for recovery: (%s != %s)", - lustre_msg_get_status(req->rq_reqmsg), + "req waiting for recovery: (%s != %s)", ptlrpc_import_state_name(req->rq_send_state), ptlrpc_import_state_name(imp->imp_state)); LASSERT(list_empty(&req->rq_list)); From patchwork Mon Sep 30 18:54:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167071 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 53B2B13BD for ; Mon, 30 Sep 2019 18:57:50 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3BF07224F0 for ; Mon, 30 Sep 2019 18:57:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3BF07224F0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 172EE5C356A; Mon, 30 Sep 2019 11:57:27 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id CC6945C39AE for ; Mon, 30 Sep 2019 11:57:05 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 6E07E1005407; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 6D0CBA9; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:43 -0400 Message-Id: <1569869810-23848-25-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 024/151] lnet: Fix lost lock X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Dmitry Eremin Unlock lnet_net_lock in case of error in function lnet_dyn_del_net(). WC-bug-id: https://jira.whamcloud.com/browse/LU-9769 Lustre-commit: 9761b5c63103 ("LU-9769 lnet: Fix lost lock") Signed-off-by: Dmitry Eremin Reviewed-on: https://review.whamcloud.com/28039 Reviewed-by: Amir Shehata Reviewed-by: Andreas Dilger Reviewed-by: James Simmons Signed-off-by: James Simmons --- net/lnet/lnet/api-ni.c | 1 + 1 file changed, 1 insertion(+) diff --git a/net/lnet/lnet/api-ni.c b/net/lnet/lnet/api-ni.c index c33cd1f..47f4f2a 100644 --- a/net/lnet/lnet/api-ni.c +++ b/net/lnet/lnet/api-ni.c @@ -2906,6 +2906,7 @@ int lnet_dyn_del_ni(struct lnet_ioctl_config_ni *conf) net = lnet_get_net_locked(net_id); if (!net) { + lnet_net_unlock(0); rc = -EINVAL; goto out; } From patchwork Mon Sep 30 18:54:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167125 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8003313BD for ; Mon, 30 Sep 2019 18:59:33 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 685EA224D5 for ; Mon, 30 Sep 2019 18:59:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 685EA224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id BEE985C3F39; Mon, 30 Sep 2019 11:58:20 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 1908E5C39AE for ; Mon, 30 Sep 2019 11:57:06 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 7112C1005408; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 7000BBB; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:44 -0400 Message-Id: <1569869810-23848-26-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 025/151] lustre: llite: Reduce overhead for ll_do_fast_read X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Patrick Farrell In ll_do_fast_read, looking up a cl_env adds some overhead, and can also cause spinlock contention on older kernels. Fast read can safely use the preallocated percpu cl_env, so do that to reduce overhead. SLES numbers on recent Xeon, CentOS numbers on VMs on older hardware. SLES has queued spinlocks and scales perfectly with multiple threads, with or without this patch. CentOS scales poorly at small I/O sizes without this patch. SLES is SLES12SP2, CentOS is CentOS 7.3. SLES: 1 thread 8b 1K 1M Without: 23 2200 6800 With: 27.5 2500 7200 4 threads 8b 1K 1M Without: 90 8700 27000 With: 108 10000 28000 Earlier kernel (CentOS 7.3): 1 thread 8b 1K 1M Without: 9 1000 5100 with: 12 1300 5800 4 threads 8b 1K 1M Without: 22 2400 17000 With: 48 4900 20000 WC-bug-id: https://jira.whamcloud.com/browse/LU-9749 Lustre-commit: c084c6215851 ("LU-9749 llite: Reduce overhead for ll_do_fast_read") Signed-off-by: Patrick Farrell Reviewed-on: https://review.whamcloud.com/27970 Reviewed-by: Andreas Dilger Reviewed-by: Jinshan Xiong Reviewed-by: Dmitry Eremin Signed-off-by: James Simmons --- fs/lustre/llite/file.c | 16 ++++++---------- fs/lustre/llite/rw.c | 29 ++++++++++++++++++++--------- 2 files changed, 26 insertions(+), 19 deletions(-) diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index bc5eebb..53b4620 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -1270,15 +1270,13 @@ static void ll_io_init(struct cl_io *io, const struct file *file, int write) * doesn't make the situation worse on single node but it may interleave write * results from multiple nodes due to short read handling in ll_file_aio_read(). * - * @env: lu_env * @iocb: kiocb from kernel * @iter: user space buffers where the data will be copied * * Returns: number of bytes have been read, or error code if error occurred. */ static ssize_t -ll_do_fast_read(const struct lu_env *env, struct kiocb *iocb, - struct iov_iter *iter) +ll_do_fast_read(struct kiocb *iocb, struct iov_iter *iter) { ssize_t result; @@ -1292,9 +1290,7 @@ static void ll_io_init(struct cl_io *io, const struct file *file, int write) if (iocb->ki_filp->f_flags & O_DIRECT) return 0; - ll_cl_add(iocb->ki_filp, env, NULL, LCC_RW); result = generic_file_read_iter(iocb, iter); - ll_cl_remove(iocb->ki_filp, env); /* * If the first page is not in cache, generic_file_aio_read() will be @@ -1319,14 +1315,14 @@ static ssize_t ll_file_read_iter(struct kiocb *iocb, struct iov_iter *to) u16 refcheck; ssize_t rc2; + result = ll_do_fast_read(iocb, to); + if (result < 0 || iov_iter_count(to) == 0) + goto out; + env = cl_env_get(&refcheck); if (IS_ERR(env)) return PTR_ERR(env); - result = ll_do_fast_read(env, iocb, to); - if (result < 0 || iov_iter_count(to) == 0) - goto out; - args = ll_env_args(env); args->u.normal.via_iter = to; args->u.normal.via_iocb = iocb; @@ -1338,8 +1334,8 @@ static ssize_t ll_file_read_iter(struct kiocb *iocb, struct iov_iter *to) else if (result == 0) result = rc2; -out: cl_env_put(env, &refcheck); +out: return result; } diff --git a/fs/lustre/llite/rw.c b/fs/lustre/llite/rw.c index e66aa67..32f028db 100644 --- a/fs/lustre/llite/rw.c +++ b/fs/lustre/llite/rw.c @@ -1158,22 +1158,21 @@ int ll_readpage(struct file *file, struct page *vmpage) { struct cl_object *clob = ll_i2info(file_inode(file))->lli_clob; struct ll_cl_context *lcc; - const struct lu_env *env; - struct cl_io *io; + const struct lu_env *env = NULL; + struct cl_io *io = NULL; struct cl_page *page; int result; lcc = ll_cl_find(file); - if (!lcc) { - unlock_page(vmpage); - return -EIO; + if (lcc) { + env = lcc->lcc_env; + io = lcc->lcc_io; } - env = lcc->lcc_env; - io = lcc->lcc_io; if (!io) { /* fast read */ struct ll_file_data *fd = LUSTRE_FPRIVATE(file); struct ll_readahead_state *ras = &fd->fd_ras; + struct lu_env *local_env = NULL; struct inode *inode = file_inode(file); struct vvp_page *vpg; @@ -1189,11 +1188,16 @@ int ll_readpage(struct file *file, struct page *vmpage) return result; } + if (!env) { + local_env = cl_env_percpu_get(); + env = local_env; + } + vpg = cl2vvp_page(cl_object_page_slice(page->cp_obj, page)); if (vpg->vpg_defer_uptodate) { enum ras_update_flags flags = LL_RAS_HIT; - if (lcc->lcc_type == LCC_MMAP) + if (lcc && lcc->lcc_type == LCC_MMAP) flags |= LL_RAS_MMAP; /* @@ -1220,8 +1224,15 @@ int ll_readpage(struct file *file, struct page *vmpage) } } - unlock_page(vmpage); + /* release page refcount before unlocking the page to ensure + * the object won't be destroyed in the calling path of + * cl_page_put(). Please see comment in ll_releasepage(). + */ cl_page_put(env, page); + unlock_page(vmpage); + if (local_env) + cl_env_percpu_put(local_env); + return result; } From patchwork Mon Sep 30 18:54:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167129 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 70CF216C1 for ; Mon, 30 Sep 2019 18:59:40 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 55FDC224F0 for ; Mon, 30 Sep 2019 18:59:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 55FDC224F0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 11DE05C3F7D; Mon, 30 Sep 2019 11:58:24 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 72DB45C3456 for ; Mon, 30 Sep 2019 11:57:06 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 758D41005409; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 7336DB4; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:45 -0400 Message-Id: <1569869810-23848-27-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 026/151] lustre: ptlrpc: change cr_sent_tv from timespec to ktime X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alex Zhuravlev , James Simmons , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" Instead of calculating timediff by manually subtracting the current timespec64 from cr_sent_tv we can convert cr_sent_tv to ktime_t and simplify the code. WC-id: https://jira.hpdd.intel.com/browse/LU-4423 Lustre-commit: de7a14b54637 ("LU-4423 ptlrpc: use 64-bit times for request times") Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/24977 WC-id: https://jira.hpdd.intel.com/browse/LU-9683 Lustre-commit: 61c48e79fdfb ("LU-9683 ptlrpc: fix argument misorder") Reviewed-on: https://review.whamcloud.com/28027 Signed-off-by: Alex Zhuravlev Reviewed-by: Bob Glossman Reviewed-by: Andreas Dilger Reviewed-by: Niu Yawei Reviewed-by: Mike Pershin Signed-off-by: James Simmons --- fs/lustre/include/lustre_net.h | 6 +++--- fs/lustre/ptlrpc/client.c | 11 +++++------ fs/lustre/ptlrpc/niobuf.c | 2 +- fs/lustre/ptlrpc/service.c | 33 +++++++++++++-------------------- 4 files changed, 22 insertions(+), 30 deletions(-) diff --git a/fs/lustre/include/lustre_net.h b/fs/lustre/include/lustre_net.h index fde59df..20ce196 100644 --- a/fs/lustre/include/lustre_net.h +++ b/fs/lustre/include/lustre_net.h @@ -537,8 +537,8 @@ struct ptlrpc_cli_req { long cr_delay_limit; /** time request was first queued */ unsigned long cr_queued_time; - /** request sent timeval */ - struct timespec64 cr_sent_tv; + /** request sent in nanosseconds */ + ktime_t cr_sent_ns; /** time for request really sent out */ time64_t cr_sent_out; /** when req reply unlink must finish. */ @@ -600,7 +600,7 @@ struct ptlrpc_cli_req { #define rq_bulk rq_cli.cr_bulk #define rq_delay_limit rq_cli.cr_delay_limit #define rq_queued_time rq_cli.cr_queued_time -#define rq_sent_tv rq_cli.cr_sent_tv +#define rq_sent_ns rq_cli.cr_sent_ns #define rq_real_sent rq_cli.cr_sent_out #define rq_reply_deadline rq_cli.cr_reply_deadline #define rq_bulk_deadline rq_cli.cr_bulk_deadline diff --git a/fs/lustre/ptlrpc/client.c b/fs/lustre/ptlrpc/client.c index ba894c0..4888578 100644 --- a/fs/lustre/ptlrpc/client.c +++ b/fs/lustre/ptlrpc/client.c @@ -1325,8 +1325,8 @@ static int after_reply(struct ptlrpc_request *req) struct obd_import *imp = req->rq_import; struct obd_device *obd = req->rq_import->imp_obd; int rc; - struct timespec64 work_start; - long timediff; + ktime_t work_start; + s64 timediff; u64 committed; LASSERT(obd); @@ -1355,10 +1355,9 @@ static int after_reply(struct ptlrpc_request *req) return 0; } - ktime_get_real_ts64(&work_start); - timediff = (work_start.tv_sec - req->rq_sent_tv.tv_sec) * USEC_PER_SEC + - (work_start.tv_nsec - req->rq_sent_tv.tv_nsec) / - NSEC_PER_USEC; + work_start = ktime_get_real(); + timediff = ktime_us_delta(work_start, req->rq_sent_ns); + /* * NB Until this point, the whole of the incoming message, * including buflens, status etc is in the sender's byte order. diff --git a/fs/lustre/ptlrpc/niobuf.c b/fs/lustre/ptlrpc/niobuf.c index 033d628..e7aff51 100644 --- a/fs/lustre/ptlrpc/niobuf.c +++ b/fs/lustre/ptlrpc/niobuf.c @@ -684,7 +684,7 @@ int ptl_send_rpc(struct ptlrpc_request *request, int noreply) OBD_FAIL_TIMEOUT(OBD_FAIL_PTLRPC_DELAY_SEND, request->rq_timeout + 5); - ktime_get_real_ts64(&request->rq_sent_tv); + request->rq_sent_ns = ktime_get_real(); request->rq_sent = ktime_get_real_seconds(); /* We give the server rq_timeout secs to process the req, and * add the network latency for our local timeout. diff --git a/fs/lustre/ptlrpc/service.c b/fs/lustre/ptlrpc/service.c index c9ab9c3..b2a1b4a 100644 --- a/fs/lustre/ptlrpc/service.c +++ b/fs/lustre/ptlrpc/service.c @@ -1649,12 +1649,11 @@ static bool ptlrpc_server_normal_pending(struct ptlrpc_service_part *svcpt, { struct ptlrpc_service *svc = svcpt->scp_service; struct ptlrpc_request *request; - struct timespec64 work_start; - struct timespec64 work_end; - struct timespec64 timediff; - struct timespec64 arrived; - unsigned long timediff_usecs; - unsigned long arrived_usecs; + ktime_t work_start; + ktime_t work_end; + ktime_t arrived; + s64 timediff_usecs; + s64 arrived_usecs; int fail_opc = 0; request = ptlrpc_server_request_get(svcpt, false); @@ -1676,10 +1675,9 @@ static bool ptlrpc_server_normal_pending(struct ptlrpc_service_part *svcpt, if (OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_DUMP_LOG)) libcfs_debug_dumplog(); - ktime_get_real_ts64(&work_start); - timediff = timespec64_sub(work_start, request->rq_arrival_time); - timediff_usecs = timediff.tv_sec * USEC_PER_SEC + - timediff.tv_nsec / NSEC_PER_USEC; + work_start = ktime_get_real(); + arrived = timespec64_to_ktime(request->rq_arrival_time); + timediff_usecs = ktime_us_delta(work_start, arrived); if (likely(svc->srv_stats)) { lprocfs_counter_add(svc->srv_stats, PTLRPC_REQWAIT_CNTR, timediff_usecs); @@ -1746,15 +1744,11 @@ static bool ptlrpc_server_normal_pending(struct ptlrpc_service_part *svcpt, (s64)ktime_get_real_seconds() - request->rq_deadline); } - ktime_get_real_ts64(&work_end); - timediff = timespec64_sub(work_end, work_start); - timediff_usecs = timediff.tv_sec * USEC_PER_SEC + - timediff.tv_nsec / NSEC_PER_USEC; - arrived = timespec64_sub(work_end, request->rq_arrival_time); - arrived_usecs = arrived.tv_sec * USEC_PER_SEC + - arrived.tv_nsec / NSEC_PER_USEC; + work_end = ktime_get_real(); + timediff_usecs = ktime_us_delta(work_end, work_start); + arrived_usecs = ktime_us_delta(work_end, arrived); CDEBUG(D_RPCTRACE, - "Handled RPC pname:cluuid+ref:pid:xid:nid:opc %s:%s+%d:%d:x%llu:%s:%d Request processed in %ldus (%ldus total) trans %llu rc %d/%d\n", + "Handled RPC pname:cluuid+ref:pid:xid:nid:opc %s:%s+%d:%d:x%llu:%s:%d Request processed in %lldus (%lldus total) trans %llu rc %d/%d\n", current->comm, (request->rq_export ? (char *)request->rq_export->exp_client_uuid.uuid : "0"), @@ -1787,8 +1781,7 @@ static bool ptlrpc_server_normal_pending(struct ptlrpc_service_part *svcpt, DEBUG_REQ(D_ADAPTTO, request, "sent %d early replies before finishing in %llds", request->rq_early_count, - (s64)work_end.tv_sec - - request->rq_arrival_time.tv_sec); + arrived_usecs / USEC_PER_SEC); } ptlrpc_server_finish_active_request(svcpt, request); From patchwork Mon Sep 30 18:54:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167119 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7AF1F16B1 for ; Mon, 30 Sep 2019 18:59:24 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 63956224D5 for ; Mon, 30 Sep 2019 18:59:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 63956224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id B4D355C3EFE; Mon, 30 Sep 2019 11:58:16 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id C83345C3A40 for ; Mon, 30 Sep 2019 11:57:06 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 787B2100540A; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 7647DB5; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:46 -0400 Message-Id: <1569869810-23848-28-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 027/151] lustre: ptlrpc: Use C99 initializer in ptlrpc_register_rqbd() X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Steve Guminski , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Steve Guminski This patch makes no functional changes. The struct initializer in ptlrpc_register_rqbd() is updated to C99 syntax. C89 positional initializers require values to be placed in the correct order. This will cause errors if the fields of the struct definition are reordered or fields are added or removed. C99 named initializers avoid this problem, and also automatically clear any values that are not explicitly set. WC-bug-id: https://jira.whamcloud.com/browse/LU-6210 Lustre-commit: 3af10d510078 ("LU-6210 ptlrpc: Use C99 initializer in ptlrpc_register_rqbd()") Reviewed-on: https://review.whamcloud.com/28479 Signed-off-by: Steve Guminski Reviewed-by: James Simmons Reviewed-by: John L. Hammond Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/ptlrpc/niobuf.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/fs/lustre/ptlrpc/niobuf.c b/fs/lustre/ptlrpc/niobuf.c index e7aff51..f2b7902 100644 --- a/fs/lustre/ptlrpc/niobuf.c +++ b/fs/lustre/ptlrpc/niobuf.c @@ -747,7 +747,10 @@ int ptl_send_rpc(struct ptlrpc_request *request, int noreply) int ptlrpc_register_rqbd(struct ptlrpc_request_buffer_desc *rqbd) { struct ptlrpc_service *service = rqbd->rqbd_svcpt->scp_service; - static struct lnet_process_id match_id = {LNET_NID_ANY, LNET_PID_ANY}; + static struct lnet_process_id match_id = { + .nid = LNET_NID_ANY, + .pid = LNET_PID_ANY + }; int rc; struct lnet_md md; struct lnet_handle_me me_h; From patchwork Mon Sep 30 18:54:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167073 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EFBFA16C1 for ; Mon, 30 Sep 2019 18:57:56 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D6F79224F0 for ; Mon, 30 Sep 2019 18:57:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D6F79224F0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8197B5C3B61; Mon, 30 Sep 2019 11:57:31 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 166485C357A for ; Mon, 30 Sep 2019 11:57:07 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 7A711100541D; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 7936EA9; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:47 -0400 Message-Id: <1569869810-23848-29-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 028/151] lustre: lmv: stripe dir page may be released mistakenly X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lai Siyao , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Lai Siyao stripe_dirent_next() may put_stripe_dirent() while its dirent is still in use, e.g. lmv_dirent_next() popped stripe last dirent, when it can't point sd_ent to next, but it shouldn't release stripe dir page. stripe_dirent->sd_ent should be set NULL when its dir page is released, which can avoid misuse. WC-bug-id: https://jira.whamcloud.com/browse/LU-9857 Lustre-commit: beadbad429ae ("LU-9857 lmv: stripe dir page may be released mistakenly") Signed-off-by: Lai Siyao Reviewed-on: https://review.whamcloud.com/28548 Reviewed-by: Fan Yong Reviewed-by: John L. Hammond Tested-by: John L. Hammond Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/lmv/lmv_obd.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/fs/lustre/lmv/lmv_obd.c b/fs/lustre/lmv/lmv_obd.c index d71d077..bcbda30 100644 --- a/fs/lustre/lmv/lmv_obd.c +++ b/fs/lustre/lmv/lmv_obd.c @@ -2023,6 +2023,7 @@ static inline void put_stripe_dirent(struct stripe_dirent *stripe) kunmap(stripe->sd_page); put_page(stripe->sd_page); stripe->sd_page = NULL; + stripe->sd_ent = NULL; } } @@ -2045,20 +2046,24 @@ static struct lu_dirent *stripe_dirent_next(struct lmv_dir_ctxt *ctxt, LASSERT(stripe == &ctxt->ldc_stripes[stripe_index]); + if (stripe->sd_eof) + return NULL; + if (ent) { ent = lu_dirent_next(ent); if (!ent) { check_eof: end = le64_to_cpu(stripe->sd_dp->ldp_hash_end); - put_stripe_dirent(stripe); - + LASSERTF(hash <= end, "hash %llx end %llx\n", + hash, end); if (end == MDS_DIR_END_OFF) { stripe->sd_ent = NULL; stripe->sd_eof = true; return NULL; } - LASSERT(hash <= end); + + put_stripe_dirent(stripe); hash = end; } } @@ -2114,10 +2119,8 @@ static struct lu_dirent *stripe_dirent_next(struct lmv_dir_ctxt *ctxt, le16_to_cpu(ent->lde_namelen)) == 0)) continue; - if (le64_to_cpu(ent->lde_hash) < hash) - continue; - - break; + if (le64_to_cpu(ent->lde_hash) >= hash) + break; } if (!ent) From patchwork Mon Sep 30 18:54:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167123 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2F05713BD for ; Mon, 30 Sep 2019 18:59:32 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 166BA224D5 for ; Mon, 30 Sep 2019 18:59:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 166BA224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 022025C3F31; Mon, 30 Sep 2019 11:58:20 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 576D25C3A66 for ; Mon, 30 Sep 2019 11:57:07 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 7E5D3100541E; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 7C5C0BB; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:48 -0400 Message-Id: <1569869810-23848-30-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 029/151] lnet: selftest: Use C99 struct initializer in framework.c X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Steve Guminski , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Steve Guminski This patch makes no functional changes. The struct initializer in framework.c is updated to C99 syntax. C89 positional initializers require values to be placed in the correct order. This will cause errors if the fields of the struct definition are reordered or fields are added or removed. C99 named initializers avoid this problem, and also automatically clear any values that are not explicitly set. WC-bug-id: https://jira.whamcloud.com/browse/LU-6210 Lustre-commit: 60c9a2d351d4 ("LU-6210 lnet: Use C99 struct initializer in framework.c") Signed-off-by: Steve Guminski Reviewed-on: https://review.whamcloud.com/28436 Reviewed-by: James Simmons Reviewed-by: Andreas Dilger Signed-off-by: James Simmons --- net/lnet/selftest/framework.c | 42 +++++++----------------------------------- 1 file changed, 7 insertions(+), 35 deletions(-) diff --git a/net/lnet/selftest/framework.c b/net/lnet/selftest/framework.c index 958f627..72892c1 100644 --- a/net/lnet/selftest/framework.c +++ b/net/lnet/selftest/framework.c @@ -1584,41 +1584,13 @@ struct srpc_client_rpc * } static struct srpc_service sfw_services[] = { - { - /* sv_id */ SRPC_SERVICE_DEBUG, - /* sv_name */ "debug", - 0 - }, - { - /* sv_id */ SRPC_SERVICE_QUERY_STAT, - /* sv_name */ "query stats", - 0 - }, - { - /* sv_id */ SRPC_SERVICE_MAKE_SESSION, - /* sv_name */ "make session", - 0 - }, - { - /* sv_id */ SRPC_SERVICE_REMOVE_SESSION, - /* sv_name */ "remove session", - 0 - }, - { - /* sv_id */ SRPC_SERVICE_BATCH, - /* sv_name */ "batch service", - 0 - }, - { - /* sv_id */ SRPC_SERVICE_TEST, - /* sv_name */ "test service", - 0 - }, - { - /* sv_id */ 0, - /* sv_name */ NULL, - 0 - } + { .sv_id = SRPC_SERVICE_DEBUG, .sv_name = "debug", }, + { .sv_id = SRPC_SERVICE_QUERY_STAT, .sv_name = "query stats", }, + { .sv_id = SRPC_SERVICE_MAKE_SESSION, .sv_name = "make session", }, + { .sv_id = SRPC_SERVICE_REMOVE_SESSION, .sv_name = "remove session", }, + { .sv_id = SRPC_SERVICE_BATCH, .sv_name = "batch service", }, + { .sv_id = SRPC_SERVICE_TEST, .sv_name = "test service", }, + { .sv_id = 0, } }; int From patchwork Mon Sep 30 18:54:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167127 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 599E716B1 for ; Mon, 30 Sep 2019 18:59:40 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 412AA224D5 for ; Mon, 30 Sep 2019 18:59:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 412AA224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id CFCDA5C3B7D; Mon, 30 Sep 2019 11:58:23 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 9A0425C3A7A for ; Mon, 30 Sep 2019 11:57:07 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 80475100541F; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 7F434BD; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:49 -0400 Message-Id: <1569869810-23848-31-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 030/151] lnet: fix memory leak and lnet_interfaces_max X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Amir Shehata , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Amir Shehata Free buffer allocated for discover command. Set lnet_interfaces_max to LNET_INTERFACES_MAX_DEFAULT if it's not defined or if it's being set to something below LNET_INTERFACES_MIN. For lnet_ping() and lnet_discover() if the provided space can fit more NIDs than lnet_interfaces_max then ensure only lnet_interfaces_max is copied into the buffer. WC-bug-id: https://jira.whamcloud.com/browse/LU-9909 Lustre-commit: 81d4f7a25319 ("LU-9909 lnet: fix memory leak and lnet_interfaces_max") Signed-off-by: Amir Shehata Reviewed-on: https://review.whamcloud.com/28702 Reviewed-by: Sonia Sharma Reviewed-by: Olaf Weber Reviewed-by: John L. Hammond Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/lnet/api-ni.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/net/lnet/lnet/api-ni.c b/net/lnet/lnet/api-ni.c index 47f4f2a..78fd05f 100644 --- a/net/lnet/lnet/api-ni.c +++ b/net/lnet/lnet/api-ni.c @@ -170,8 +170,8 @@ static int lnet_discover(struct lnet_process_id id, u32 force, if (value < LNET_INTERFACES_MIN) { CWARN("max interfaces provided are too small, setting to %d\n", - LNET_INTERFACES_MIN); - value = LNET_INTERFACES_MIN; + LNET_INTERFACES_MAX_DEFAULT); + value = LNET_INTERFACES_MAX_DEFAULT; } *(int *)kp->arg = value; @@ -3407,9 +3407,15 @@ static int lnet_ping(struct lnet_process_id id, signed long timeout, int rc2; /* n_ids limit is arbitrary */ - if (n_ids <= 0 || n_ids > lnet_interfaces_max || id.nid == LNET_NID_ANY) + if (n_ids <= 0 || id.nid == LNET_NID_ANY) return -EINVAL; + /* if the user buffer has more space than the lnet_interfaces_max + * then only fill it up to lnet_interfaces_max + */ + if (n_ids > lnet_interfaces_max) + n_ids = lnet_interfaces_max; + if (id.pid == LNET_PID_ANY) id.pid = LNET_PID_LUSTRE; @@ -3575,13 +3581,18 @@ static int lnet_ping(struct lnet_process_id id, signed long timeout, int max_intf = lnet_interfaces_max; if (n_ids <= 0 || - id.nid == LNET_NID_ANY || - n_ids > max_intf) + id.nid == LNET_NID_ANY) return -EINVAL; if (id.pid == LNET_PID_ANY) id.pid = LNET_PID_LUSTRE; + /* if the user buffer has more space than the max_intf + * then only fill it up to max_intf + */ + if (n_ids > max_intf) + n_ids = max_intf; + buf = kcalloc(n_ids, sizeof(*buf), GFP_KERNEL); if (!buf) return -ENOMEM; From patchwork Mon Sep 30 18:54:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167131 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E271113BD for ; Mon, 30 Sep 2019 18:59:47 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CB82A224F0 for ; Mon, 30 Sep 2019 18:59:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CB82A224F0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 6B2CE5C38F8; Mon, 30 Sep 2019 11:58:27 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id DD90A5C3A9B for ; Mon, 30 Sep 2019 11:57:07 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 83263100542F; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 8226FB4; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:50 -0400 Message-Id: <1569869810-23848-32-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 031/151] lnet: decref on peer after use X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Amir Shehata , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Amir Shehata After looking up the peer for both ping and discover we need to decref the peer so we don't lose a reference on it. This needs to be done while the mutex_lock is held to ensure the peer list remains stable. WC-bug-id: https://jira.whamcloud.com/browse/LU-9918 Lustre-commit: 440c1c03fe08 ("LU-9918 lnet: decref on peer after use") Signed-off-by: Amir Shehata Reviewed-on: https://review.whamcloud.com/28722 Reviewed-by: John L. Hammond Reviewed-by: Sonia Sharma Reviewed-by: Olaf Weber Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/lnet/api-ni.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/net/lnet/lnet/api-ni.c b/net/lnet/lnet/api-ni.c index 78fd05f..bc3f808 100644 --- a/net/lnet/lnet/api-ni.c +++ b/net/lnet/lnet/api-ni.c @@ -3297,11 +3297,15 @@ u32 lnet_get_dlc_seq_locked(void) if (rc < 0) return rc; + mutex_lock(&the_lnet.ln_api_mutex); lp = lnet_find_peer(ping->ping_id.nid); if (lp) { ping->ping_id.nid = lp->lp_primary_nid; ping->mr_info = lnet_peer_is_multi_rail(lp); + lnet_peer_decref_locked(lp); } + mutex_unlock(&the_lnet.ln_api_mutex); + ping->ping_count = rc; return 0; } @@ -3315,11 +3319,15 @@ u32 lnet_get_dlc_seq_locked(void) discover->ping_count); if (rc < 0) return rc; + + mutex_lock(&the_lnet.ln_api_mutex); lp = lnet_find_peer(discover->ping_id.nid); if (lp) { discover->ping_id.nid = lp->lp_primary_nid; discover->mr_info = lnet_peer_is_multi_rail(lp); + lnet_peer_decref_locked(lp); } + mutex_unlock(&the_lnet.ln_api_mutex); discover->ping_count = rc; return 0; From patchwork Mon Sep 30 18:54:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167137 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A10E913BD for ; Mon, 30 Sep 2019 18:59:55 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 89825224D5 for ; Mon, 30 Sep 2019 18:59:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 89825224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 1E9805C3FDF; Mon, 30 Sep 2019 11:58:31 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 2A4DB5C3AA7 for ; Mon, 30 Sep 2019 11:57:08 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 86B3F1005437; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 854DEA9; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:51 -0400 Message-Id: <1569869810-23848-33-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 032/151] lnet: rediscover peer if it changed X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Amir Shehata , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Amir Shehata If the peer has changed after we unlocked the cpt then we'll need to discover the new peer. WC-bug-id: https://jira.whamcloud.com/browse/LU-9917 Lustre-commit: 1fc4ed3ac40a ("LU-9917 lnet: rediscover peer if it changed") Signed-off-by: Amir Shehata Reviewed-on: https://review.whamcloud.com/28772 Reviewed-by: Olaf Weber Reviewed-by: John L. Hammond Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/lnet/peer.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/net/lnet/lnet/peer.c b/net/lnet/lnet/peer.c index ade7f23..34e61f8 100644 --- a/net/lnet/lnet/peer.c +++ b/net/lnet/lnet/peer.c @@ -2034,6 +2034,13 @@ static void lnet_peer_clear_discovery_error(struct lnet_peer *lp) lnet_net_unlock(LNET_LOCK_EX); lnet_net_lock(cpt); + /* If the peer has changed after we've discovered the older peer, + * then we need to discovery the new peer to make sure the + * interface information is up to date + */ + if (lp != lpni->lpni_peer_net->lpn_peer) + goto again; + if (signal_pending(current)) rc = -EINTR; else if (the_lnet.ln_dc_state != LNET_DC_STATE_RUNNING) From patchwork Mon Sep 30 18:54:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167133 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E4F4D16C1 for ; Mon, 30 Sep 2019 18:59:47 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CADEC224D5 for ; Mon, 30 Sep 2019 18:59:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CADEC224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 6E9CB5C3A82; Mon, 30 Sep 2019 11:58:27 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 6BF035C3AB5 for ; Mon, 30 Sep 2019 11:57:08 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 8964E1005438; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 8803FB5; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:52 -0400 Message-Id: <1569869810-23848-34-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 033/151] lnet: resolve unsafe list access X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Amir Shehata , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Amir Shehata Use list_for_each_entry_safe() when accessing messages on pending queue. Remove the message from the list before calling lnet_finalize() or lnet_send(). When destroying the peer make sure to queue all pending messages on a global list. We can not resend them at this point because the cpt lock is held. Unlocking the cpt lock could lead to an inconsistent state. Use the discovery thread to check if the global list is not empty and if so resend all messages on the list. Use a new spin lock to protect the resend message list. I steered clear from reusing an existing lock because LNet locking is complex and reusing a lock will add to this complexity. Using a new lock makes the code easier to understand. Verify that all lists are empty before destroying the peer_ni WC-bug-id: https://jira.whamcloud.com/browse/LU-9921 Lustre-commit: 62c3c8d14856 ("LU-9921 lnet: resolve unsafe list access") Signed-off-by: Amir Shehata Reviewed-on: https://review.whamcloud.com/28723 Reviewed-by: Olaf Weber Reviewed-by: John L. Hammond Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- include/linux/lnet/lib-types.h | 4 +++ net/lnet/lnet/api-ni.c | 15 ++++++++++ net/lnet/lnet/peer.c | 68 ++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 85 insertions(+), 2 deletions(-) diff --git a/include/linux/lnet/lib-types.h b/include/linux/lnet/lib-types.h index 6d2d83a..a5cbf07 100644 --- a/include/linux/lnet/lib-types.h +++ b/include/linux/lnet/lib-types.h @@ -945,6 +945,10 @@ struct lnet { struct list_head ln_net_zombie; /* the loopback NI */ struct lnet_ni *ln_loni; + /* resend messages list */ + struct list_head ln_msg_resend; + /* spin lock to protect the msg resend list */ + spinlock_t ln_msg_resend_lock; /* remote networks with routes to them */ struct list_head *ln_remote_nets_hash; diff --git a/net/lnet/lnet/api-ni.c b/net/lnet/lnet/api-ni.c index bc3f808..b038010 100644 --- a/net/lnet/lnet/api-ni.c +++ b/net/lnet/lnet/api-ni.c @@ -211,6 +211,7 @@ static int lnet_discover(struct lnet_process_id id, u32 force, lnet_init_locks(void) { spin_lock_init(&the_lnet.ln_eq_wait_lock); + spin_lock_init(&the_lnet.ln_msg_resend_lock); init_waitqueue_head(&the_lnet.ln_eq_waitq); init_waitqueue_head(&the_lnet.ln_rc_waitq); mutex_init(&the_lnet.ln_lnd_mutex); @@ -1652,6 +1653,10 @@ static void lnet_push_target_fini(void) lnet_shutdown_lndnets(void) { struct lnet_net *net; + struct list_head resend; + struct lnet_msg *msg, *tmp; + + INIT_LIST_HEAD(&resend); /* NB called holding the global mutex */ @@ -1682,6 +1687,15 @@ static void lnet_push_target_fini(void) net_list)) != NULL) lnet_shutdown_lndnet(net); + spin_lock(&the_lnet.ln_msg_resend_lock); + list_splice(&the_lnet.ln_msg_resend, &resend); + spin_unlock(&the_lnet.ln_msg_resend_lock); + + list_for_each_entry_safe(msg, tmp, &resend, msg_list) { + list_del_init(&msg->msg_list); + lnet_finalize(msg, -ECANCELED); + } + lnet_net_lock(LNET_LOCK_EX); the_lnet.ln_state = LNET_STATE_SHUTDOWN; lnet_net_unlock(LNET_LOCK_EX); @@ -2025,6 +2039,7 @@ int lnet_lib_init(void) INIT_LIST_HEAD(&the_lnet.ln_lnds); INIT_LIST_HEAD(&the_lnet.ln_net_zombie); INIT_LIST_HEAD(&the_lnet.ln_rcd_zombie); + INIT_LIST_HEAD(&the_lnet.ln_msg_resend); INIT_LIST_HEAD(&the_lnet.ln_rcd_deathrow); /* diff --git a/net/lnet/lnet/peer.c b/net/lnet/lnet/peer.c index 34e61f8..d7a2e3b 100644 --- a/net/lnet/lnet/peer.c +++ b/net/lnet/lnet/peer.c @@ -231,6 +231,22 @@ if (lp->lp_data) lnet_ping_buffer_decref(lp->lp_data); + /* if there are messages still on the pending queue, then make + * sure to queue them on the ln_msg_resend list so they can be + * resent at a later point if the discovery thread is still + * running. + * If the discovery thread has stopped, then the wakeup will be a + * no-op, and it is expected the lnet_shutdown_lndnets() will + * eventually be called, which will traverse this list and + * finalize the messages on the list. + * We can not resend them now because we're holding the cpt lock. + * Releasing the lock can cause an inconsistent state + */ + spin_lock(&the_lnet.ln_msg_resend_lock); + list_splice(&lp->lp_dc_pendq, &the_lnet.ln_msg_resend); + spin_unlock(&the_lnet.ln_msg_resend_lock); + wake_up(&the_lnet.ln_dc_waitq); + kfree(lp); } @@ -1532,6 +1548,8 @@ struct lnet_peer_net * LASSERT(lpni->lpni_rtr_refcount == 0); LASSERT(list_empty(&lpni->lpni_txq)); LASSERT(lpni->lpni_txqnob == 0); + LASSERT(list_empty(&lpni->lpni_peer_nis)); + LASSERT(list_empty(&lpni->lpni_on_remote_peer_ni_list)); lpn = lpni->lpni_peer_net; lpni->lpni_peer_net = NULL; @@ -1730,7 +1748,7 @@ static int lnet_peer_queue_for_discovery(struct lnet_peer *lp) */ static void lnet_peer_discovery_complete(struct lnet_peer *lp) { - struct lnet_msg *msg = NULL; + struct lnet_msg *msg, *tmp; int rc = 0; struct list_head pending_msgs; @@ -1746,7 +1764,8 @@ static void lnet_peer_discovery_complete(struct lnet_peer *lp) lnet_net_unlock(LNET_LOCK_EX); /* iterate through all pending messages and send them again */ - list_for_each_entry(msg, &pending_msgs, msg_list) { + list_for_each_entry_safe(msg, tmp, &pending_msgs, msg_list) { + list_del_init(&msg->msg_list); if (lp->lp_dc_error) { lnet_finalize(msg, lp->lp_dc_error); continue; @@ -2970,6 +2989,8 @@ static int lnet_peer_discovery_wait_for_work(void) break; if (!list_empty(&the_lnet.ln_dc_request)) break; + if (!list_empty(&the_lnet.ln_msg_resend)) + break; if (lnet_peer_dc_timed_out(ktime_get_real_seconds())) break; lnet_net_unlock(cpt); @@ -2995,6 +3016,47 @@ static int lnet_peer_discovery_wait_for_work(void) return rc; } +/* Messages that were pending on a destroyed peer will be put on a global + * resend list. The message resend list will be checked by + * the discovery thread when it wakes up, and will resend messages. These + * messages can still be sendable in the case the lpni which was the initial + * cause of the message re-queue was transferred to another peer. + * + * It is possible that LNet could be shutdown while we're iterating + * through the list. lnet_shudown_lndnets() will attempt to access the + * resend list, but will have to wait until the spinlock is released, by + * which time there shouldn't be any more messages on the resend list. + * During shutdown lnet_send() will fail and lnet_finalize() will be called + * for the messages so they can be released. The other case is that + * lnet_shudown_lndnets() can finalize all the messages before this + * function can visit the resend list, in which case this function will be + * a no-op. + */ +static void lnet_resend_msgs(void) +{ + struct lnet_msg *msg, *tmp; + struct list_head resend; + int rc; + + INIT_LIST_HEAD(&resend); + + spin_lock(&the_lnet.ln_msg_resend_lock); + list_splice(&the_lnet.ln_msg_resend, &resend); + spin_unlock(&the_lnet.ln_msg_resend_lock); + + list_for_each_entry_safe(msg, tmp, &resend, msg_list) { + list_del_init(&msg->msg_list); + rc = lnet_send(msg->msg_src_nid_param, msg, + msg->msg_rtr_nid_param); + if (rc < 0) { + CNETERR("Error sending %s to %s: %d\n", + lnet_msgtyp2str(msg->msg_type), + libcfs_id2str(msg->msg_target), rc); + lnet_finalize(msg, rc); + } + } +} + /* The discovery thread. */ static int lnet_peer_discovery(void *arg) { @@ -3008,6 +3070,8 @@ static int lnet_peer_discovery(void *arg) if (lnet_peer_discovery_wait_for_work()) break; + lnet_resend_msgs(); + if (lnet_push_target_resize_needed()) lnet_push_target_resize(); From patchwork Mon Sep 30 18:54:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167063 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 14CB713BD for ; Mon, 30 Sep 2019 18:57:36 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F0B6D224D5 for ; Mon, 30 Sep 2019 18:57:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F0B6D224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 3BB9C5C3BA4; Mon, 30 Sep 2019 11:57:20 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id C14DB5C3AC0 for ; Mon, 30 Sep 2019 11:57:08 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 8E6101005439; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 8BB3FBB; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:53 -0400 Message-Id: <1569869810-23848-35-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 034/151] lustre: llite: Implement ladvise lockahead X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Patrick Farrell Ladvise lockahead is a new feature allowing userspace to request extent locks in advance of the IO which will use them. These locks are not expanded beyond the size requested by userspace. They are intended to make it possible to address lock contention between multiple clients resulting from lock expansion. They should allow optimizing various IO patterns, notably strided writing. (Further information in LU-6179) Asynchronous glimpse locks are a speculative version of glimpse locks, and already implement the required behavior. Lockahead requests share this behavior. Additionally, lockahead creates extent locks in advance of IO, and so breaks the assumption that the holder of the highest lock knows the current file size. So we also modify the ofd_intent_policy code to glimpse PW locks until it finds one it knows to be in use, taking care to send only one glimpse to each client. The current patch allows asynchronous non-blocking lock ahead requests and synchronous blocking requests. We cannot do asynchronous blocking requests, because of deadlocks that occur in having ptlrpcd threads handle blocking lock requests. Finally, this patch also adds another advice to disable lock expansion, setting a per-file descriptor flag. This allows user space to control whether or not lock requests on this file descriptor will undergo lock expansion. This means if lockahead locks are not created ahead of IO (due to inherent raciness) or are cancelled by a competing IO request, the IO requests that should have used the manually requested locks will not result in expanded locks. This avoids lock ping-pong, and because the resulting locks will not extend to the end of the file, future lockahead requests can be granted. Effectively, this means that if lockahead usage for strided IO is interrupted by a competing request, it can re-assert itself. lockahead is implented via the ladvise interface from userspace. As lockahead results in a DLM lock request rather than file advice, we do not use the lower levels of the ladvise implementation. Note this patch has one oddity: Cray released an earlier version of lockahead without FL_SPECULATIVE support. That version uses OBD_CONNECT_LOCKAHEAD_OLD, this new one uses OBD_CONNECT_LOCKAHEAD. The client code in this patch is interoperable with that version, so it also advertises OBD_CONNECT_LOCKAHEAD_OLD support, but the server version is not, so the server advertises only OBD_CONNECT_LOCKAHEAD support. Client support for the original lockahead is slated for removal after the release of 2.12. This is enforced with a compile time version test that will remove support. WC-bug-id: https://jira.whamcloud.com/browse/LU-6179 Lustre-commit: a8dcf372f430 ("LU-6179 llite: Implement ladvise lockahead") Signed-off-by: Patrick Farrell Reviewed-on: https://review.whamcloud.com/13564 Reviewed-by: Andreas Dilger Reviewed-by: Jinshan Xiong Reviewed-by: Frank Zago Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- Documentation/lustre/ladvise_lockahead.txt | 304 +++++++++++++++++++++++++++++ fs/lustre/include/cl_object.h | 36 +++- fs/lustre/include/lustre_dlm.h | 4 +- fs/lustre/include/lustre_dlm_flags.h | 41 +++- fs/lustre/include/lustre_export.h | 20 ++ fs/lustre/include/lustre_osc.h | 11 +- fs/lustre/ldlm/ldlm_lib.c | 1 + fs/lustre/ldlm/ldlm_lock.c | 7 +- fs/lustre/llite/file.c | 277 ++++++++++++++++++++++++-- fs/lustre/llite/glimpse.c | 27 ++- fs/lustre/llite/llite_internal.h | 9 + fs/lustre/llite/llite_lib.c | 23 ++- fs/lustre/llite/vvp_io.c | 3 + fs/lustre/lov/lov_io.c | 1 + fs/lustre/obdclass/cl_lock.c | 2 +- fs/lustre/obdclass/lprocfs_status.c | 3 +- fs/lustre/osc/osc_internal.h | 3 +- fs/lustre/osc/osc_lock.c | 71 +++++-- fs/lustre/osc/osc_request.c | 50 +++-- fs/lustre/ptlrpc/wiretest.c | 8 +- include/uapi/linux/lustre/lustre_idl.h | 12 +- include/uapi/linux/lustre/lustre_user.h | 40 +++- 22 files changed, 857 insertions(+), 96 deletions(-) create mode 100644 Documentation/lustre/ladvise_lockahead.txt diff --git a/Documentation/lustre/ladvise_lockahead.txt b/Documentation/lustre/ladvise_lockahead.txt new file mode 100644 index 0000000..91dffcd --- /dev/null +++ b/Documentation/lustre/ladvise_lockahead.txt @@ -0,0 +1,304 @@ +Ladvise Lock Ahead design + +Lock ahead is a new Lustre feature aimed at solving a long standing problem +with shared file write performance in Lustre. It requires client and server +support. It will be used primarily via the MPI-I/O library, not directly from +user applications. + +The first part of this document (sections 1 and 2) is an overview of the +problem and high level description of the solution. Section 3 explains how the +library will make use of this feature, and sections 4 and 5 describe the design +of the Lustre changes. + +1. Overview: Purpose & Interface +Lock ahead is intended to allow optimization of certain I/O patterns which +would otherwise suffer LDLM* lock contention. It allows applications to +manually request locks on specific extents of a file, avoiding the usual +server side optimizations. This applications which know their I/O pattern to +use that information to avoid false conflicts due to server side optimizations. + +*Lustre distributed lock manager. This is the locking layer shared between +clients and servers, to manage access between clients. + +Normally, clients get locks automatically as the first step of an I/O. +The client asks for a lock which covers exactly the area of interest (ie, a +read or write lock of n bytes at offset x), but the server attempts to optimize +this by expanding the lock to cover as much of the file as possible. This is +useful for a single client, but can be trouble for multiple clients. + +In cases where multiple clients wish to write to the same file, this +optimization can result in locks that conflict when the actual I/O operations +do not. This requires clients to wait for one another to complete I/O, even +when there is no conflict between actual I/O requests. This can significantly +reduce performance (Anywhere from 40-90%, depending on system specs) for some +workloads. + +The lockahead feature makes it possible to avoid this problem by acquiring the +necessary locks in advance, by explicit requests with server side extent +changes disabled. We add a new lfs advice type, LU_LADVISE_LOCKAHEAD, +which allows lock requests from userspace on the client, specifying the extent +and the I/O mode (read/write) for the lock. These lock requests explicitly +disable server side changes to the lock extent, so the lock returned to the +client covers only the extent requested. + +When using this feature, clients which intend to write to a file can request +locks to cover their I/O pattern, wait a moment for the locks to be granted, +then write or read the file. + +In this way, a set of clients which knows their I/O pattern in advance can +force the LDLM layer to grant locks appropriate for that I/O pattern. This +allows applications which are poorly handled by the default lock optimization +behavior to significantly improve their performance. + +2. I/O Pattern & Locking problems +2. A. Strided writing and MPI-I/O +There is a thorough explanation and overview of strided writing and the +benefits of this functionality in the slides from the lock ahead presentation +at LUG 2015. It is highly recommended to read that first, as the graphics are +much clearer than the prose here. + +See slides 1-13: +http://wiki.lustre.org/images/f/f9/Shared-File-Performance-in-Lustre_Farrell.pdf + +MPI-I/O uses strided writing when doing I/O from a large job to a single file. +I/O is aggregated from all the nodes running a particular application to a +small number of I/O aggregator nodes which then write out the data, in a +strided manner. + +In strided writing, different clients take turns writing different blocks of a +file (A block is some arbitrary number of bytes). Client 1 is responsible for +writes to block 0, block 2, block 4, etc., client 2 is responsible for block 1, +block 3, etc. + +Without the ability to manually request locks, strided writing is set up in +concert with Lustre file striping so each client writes to one OST. (IE, for a +file striped to three OSTs, we would write from three clients.) + +The particular case of interest is when we want to use more than one client +per OST. This is important, because an OST typically has much more bandwidth +than one client. Strided writes are non-overlapping, so they should be able to +proceed in parallel with more than one client per OST. In practice, on Lustre, +they do not, due to lock expansion. + +2. B. Locking problems +We will now describe locking when there is more than one client per OST. This +behavior is the same on a per OST basis in a file striped across multiple OSTs. +When the first client asks to write block 0, it asks for the required lock from +the server. When it receives this request, the server sees that there are no +other locks on the file. Since it assumes the client will want to write to the +file again, the server expands the lock as far as possible. In this case, it +expands the lock to the maximum file size (effectively, to infinity), then +grants it to client 1. + +When client 2 wants to write block 1, it conflicts with the expanded lock +granted to client 1. The server then must revoke (In Lustre terms, +'call back') the lock granted to client 1 so it can grant a lock to client 2. +After the lock granted to client is revoked, there are no locks on the file. +The server sees this when processing the lock request from client 2, and +expands that lock to cover the whole file. + +Client 1 then wishes to write block 3 of the file... And the cycle continues. +The two clients exchange the extended lock throughout the write, allowing only +one client to write at a time, plus latency to exchange the lock. The effect is +dramatic: Two clients are actually slower than one. (Similar behavior is seen +with more than two clients.) + +The solution is to use this new advice type to acquire locks before they are +needed. In effect, before it starts writing to the file, client 1 requests +locks on block 0, block 2, etc. It locks 'ahead' a certain (tunable) number of +locks. Client 2 does the same. Then they both begin to write, and are able to +do so in parallel. A description of the actual library implementation follows. + +3. Library implementation +Actually implementing this in the library carries a number of wrinkles. +The basic pattern is this: +Before writing, an I/O aggregator requests a certain number of locks on blocks +that it is responsible for. It may or may not ever write to these blocks, but +it takes locks knowing it might. It then begins to write, tracking how many of +the locks it has used. When the number of locks 'ahead' of the I/O is low +enough, it requests more locks in advance of the I/O. + +For technical reasons which are explained in the implementation section, these +lock requests are either asynchronous and non-blocking or synchronous and +blocking. In Lustre terms, non-blocking means if there is already a lock on +the relevant extent of the file, the manual lock request is not granted. This +means that if there is already a lock on the file (quite common; imagine +writing to a file which was previously read by another process), these lock +requests will be denied. However, once the first 'real' write arrives that +was hoping to use a lockahead lock, that write will cause the blocking lock to +be cancelled, so this interference is not fatal. + +It is of course possible for another process to get in the way by immediately +asking for a lock on the file. This is something users should try to avoid. +When writing out a file, repeatedly trying to read it will impact performance +even without this feature. + +These interfering locks can also happen if a manually requested lock is, for +some reason, not available in time for the write which intended to use it. +The lock which results from this write request is expanded using the +normal rules. So it's possible for that lock (depending on the position of +other locks at the time) to be extended to cover the rest of the file. That +will block future lockahead locks. + +The expanded lock will be revoked when a write happens (from another client) +in the range covered by that lock, but the lock for that write will be expanded +as well - And then we return to handing the lock back and forth between +clients. These expanded locks will still block future lockahead locks, +rendering them useless. + +The way to avoid this is to turn off lock expansion for I/Os which are +supposed to be using these manually requested locks. That way, if the +manually requested lock is not available, the lock request for the I/O will not +be expanded. Instead, that request (which is blocking, unlike a lockahead +request) will cancel any interfering locks, but the resulting lock will not be +expanded. This leaves the later parts of the file open, allowing future +manual lock requests to succeed. This means that if an interfering lock blocks +some manual requests, those are lost, but the next set of manual requests can +proceed as normal. + +In effect, the 'locking ahead of I/O' is interrupted, but then is able to +re-assert itself. The feature used here is referred to as 'no expansion' +locking (as only the extent required by the actual I/O operation is locked) +and is turned on with another new ladvise advice, LU_LADVISE_NOEXPAND. This +feature is added as part of the lockahead patch. The strided writing library +will use this advice on the file descriptor it uses for writing. + +4. Client side design +4. A. Ladvise lockahead +Requestlock uses the existing asynchronous lock request functionality +implemented for asynchronous glimpse locks (AGLs), a long standing Lustre +feature. AGLs are locks which are requested by statahead, which are used to +get file size information before it's requested. The key thing about an +asynchronous lock request is that it does not have a specific I/O operation +waiting for the lock. + +This means two key things: + +1. There is no OSC lock (lock layer above LDLM for data locking) associated +with the LDLM lock +2. There is no thread waiting for the LDLM lock, so lock grant processing +must be handled by the ptlrpc daemon thread which received the reply + +Since both of these issues are addressed by the asynchronous lock request code +which lockahead shares with AGL, we will not explore them in depth here. + +Finally, lockahead requests set the CEF_LOCK_NO_EXPAND flag, which tells the +OSC (the per OST layer of the client) to set LDLM_FL_NO_EXPANSION on any lock +requests. LDLM_FL_NO_EXPANSION is a new LDLM lock flag which tells the server +not to expand the lock extent. + +This leaves the user facing interface. Requestlock is implemented as a new +ladvise advice, and it uses the ladvise feature of multiple advices in one API +call to put many lock requests in to an array of advices. + +The arguments required for this advice are a mode (read or write), range (start +and end), and flags. + +The client will then make lock requests on these extents, one at a time. +Because the lock requests are asynchronous (replies are handled by ptlrpcd), +many requests can be made quickly by overlapping them, rather than waiting for +each one to complete. (This requires that they be non-blocking, as the +ptlrpcd threads must not wait in the ldlm layer.) + +4. B. LU_LADVISE_LOCKNOEXPAND +The lock no expand ladvise advice sets a boolean in a Lustre data structure +associated with a file descriptor. When an I/O is done to this file +descriptor, the flag is picked up and passed through to the ldlm layer, where +it sets LDLM_FL_NO_EXPANSION on lock requests made for that I/O. + +5. Server side changes +Implementing lockahead requires server support for LDLM_FL_NO_EXPANSION, but +it also required an additional pair of server side changes to fix issues which +came up because of lockahead. These changes are not part of the core design +instead, they are separate fixes which are required for it to work. + +5. A. Support LDLM_FL_NO_EXPANSION + +Disabling server side lock expansion is done with a new LDLM flag. This is +done with a simple check for that flag on the server before attempting to +expand the lock. If the flag is found, lock expansion is skipped. + +5. B. Implement LDLM_FL_SPECULATIVE + +As described above, lock ahead locks are non-blocking. The BLOCK_NOWAIT LDLM +flag is used now to implement some nonblocking behavior, but it only considers +group locks blocking. But, for asynchronous lock requests to work correctly, +they cannot wait for any other locks. For this purpose, we add +LDLM_FL_SPECULATIVE. This new flag is used for asynchronous lock requests, +and implements the broader non-blocking behavior they require. + +5. C. File size & ofd_intent_policy changes + +Knowing the current file size during writes is tricky on a distributed file +system, because multiple clients can be writing to a file at any time. When +writes are in progress, the server must identify which client is currently +responsible for growing the file size, and ask that client what the file size +is. + +To do this, the server uses glimpse locking (in ofd_intent_policy) to get the +current file size from the clients. This code uses the assumption that the +holder of the highest write lock (PW lock) knows the current file size. A +client learns the (then current) file size when a lock is granted. Because +only the holder of the highest lock can grow a file, either the size hasn't +changed, or that client knows the new size; so the server only has to contact +the client which holds this lock, and it knows the current file size. + +Note that the above is actually racy. When the server asks, the client can +still be writing, or another client could acquire a higher lock during this +time. The goal is a good approximation while the file is being written, and a +correct answer once all the clients are done writing. This is achieved because +once writes to a file are complete, the holder of that highest lock is +guaranteed to know the current file size. This is where manually requested +locks cause trouble. + +By creating write locks in advance of an actual I/O, lockahead breaks the +assumption that the holder of the highest lock knows the file size. + +This assumption is normally true because locks which are created as part of +IO - rather than in advance of it - are guaranteed to be 'active', IE, +involved in IO, and the holder of the highest 'active' lock always knows the +current file size, because the size is either not changing or the holder of +that lock is responsible for updating it. + +Consider: Two clients, A and B, strided writing. Each client requests, for +example, 2 manually requested locks. (Real numbers are much higher.) Client A +holds locks on segments 0 and 2, client B holds locks on segments 1 and 3. + +The request comes to write 3 segments of data. Client A writes to segment 0, +client B writes to segment 1, and client A also writes to segment 2. No data +is written to segment 3. At this point, the server checks the file size, by +glimpsing the highest lock . The lock on segment 3. Client B does not know +about the writing done by client A to segment 2, so it gives an incorrect file +size. + +This would be OK if client B had pending writes to segment 3, but it does not. +In this situation, the server will never get the correct file size while this +lock exists. + +The solution is relatively straightforward: The server needs to glimpse every +client holding a write lock (starting from the top) until we find one holding +an 'active' lock (because the size is known to be at least the size returned +from an 'active' lock), and take the largest size returned. This avoids asking +only a client which may not know the correct file size. + +Unfortunately, there is no way to know if a manually requested lock is active +from the server side. So when we see such a lock, we must send a glimpse to +the holder (unless we have already sent a glimpse to that client*). However, +because locks without LDLM_FL_NO_EXPANSION set are guaranteed to be 'active', +once we reach the first such lock, we can stop glimpsing. + +*This is because when we glimpse a specific lock, the client holding it returns +its best idea of the size information, so we only need to send one glimpse to +each client. + +This is less efficient than the standard "glimpse only the top lock" +methodology, but since we only need to glimpse one lock per client (and the +number of clients writing to the part of a file on a given OST is fairly +limited), the cost is restrained. + +Additionally, lock cancellation methods such as early lock cancel aggressively +clean up older locks, particularly when the LRU limit is exceeded, so the +total lock count should also remain manageable. + +In the end, the final verdict here is performance. Requestlock testing for the +strided I/O case has shown good performance results. diff --git a/fs/lustre/include/cl_object.h b/fs/lustre/include/cl_object.h index be30ce8..c04f6c9 100644 --- a/fs/lustre/include/cl_object.h +++ b/fs/lustre/include/cl_object.h @@ -1597,10 +1597,14 @@ enum cl_enq_flags { */ CEF_NONBLOCK = 0x00000001, /** - * take lock asynchronously (out of order), as it cannot - * deadlock. This is for LDLM_FL_HAS_INTENT locks used for glimpsing. + * Tell lower layers this is a glimpse request, translated to + * LDLM_FL_HAS_INTENT at LDLM layer. + * + * Also, because glimpse locks never block other locks, we count this + * as automatically compatible with other osc locks. + * (see osc_lock_compatible) */ - CEF_ASYNC = 0x00000002, + CEF_GLIMPSE = 0x00000002, /** * tell the server to instruct (though a flag in the blocking ast) an * owner of the conflicting lock, that it can drop dirty pages @@ -1609,8 +1613,9 @@ enum cl_enq_flags { CEF_DISCARD_DATA = 0x00000004, /** * tell the sub layers that it must be a `real' lock. This is used for - * mmapped-buffer locks and glimpse locks that must be never converted - * into lockless mode. + * mmapped-buffer locks and glimpse locks, manually requested locks + * (LU_LADVISE_LOCKAHEAD) that must be never converted into lockless + * mode. * * \see vvp_mmap_locks(), cl_glimpse_lock(). */ @@ -1627,9 +1632,16 @@ enum cl_enq_flags { */ CEF_NEVER = 0x00000010, /** - * for async glimpse lock. + * tell the dlm layer this is a speculative lock request + * speculative lock requests are locks which are not requested as part + * of an I/O operation. Instead, they are requested because we expect + * to use them in the future. They are requested asynchronously at the + * ptlrpc layer. + * + * Currently used for asynchronous glimpse locks and manually requested + * locks (LU_LADVISE_LOCKAHEAD). */ - CEF_AGL = 0x00000020, + CEF_SPECULATIVE = 0x00000020, /** * enqueue a lock to test DLM lock existence. */ @@ -1640,9 +1652,13 @@ enum cl_enq_flags { */ CEF_LOCK_MATCH = BIT(7), /** + * tell the DLM layer to lock only the requested range + */ + CEF_LOCK_NO_EXPAND = BIT(8), + /** * mask of enq_flags. */ - CEF_MASK = 0x000000ff, + CEF_MASK = 0x000001ff, }; /** @@ -1849,7 +1865,9 @@ struct cl_io { /** * O_NOATIME */ - ci_noatime:1; + ci_noatime:1, + /* Tell sublayers not to expand LDLM locks requested for this IO */ + ci_lock_no_expand:1; /** * Number of pages owned by this IO. For invariant checking. */ diff --git a/fs/lustre/include/lustre_dlm.h b/fs/lustre/include/lustre_dlm.h index a1ff852..1c286eb 100644 --- a/fs/lustre/include/lustre_dlm.h +++ b/fs/lustre/include/lustre_dlm.h @@ -508,8 +508,8 @@ struct ldlm_glimpse_work { */ }; -/** The ldlm_glimpse_work is allocated on the stack and should not be freed. */ -#define LDLM_GL_WORK_NOFREE 0x1 +/* The ldlm_glimpse_work was slab allocated & must be freed accordingly.*/ +#define LDLM_GL_WORK_SLAB_ALLOCATED 0x1 /** * Interval tree for extent locks. diff --git a/fs/lustre/include/lustre_dlm_flags.h b/fs/lustre/include/lustre_dlm_flags.h index abeb651..87cda36 100644 --- a/fs/lustre/include/lustre_dlm_flags.h +++ b/fs/lustre/include/lustre_dlm_flags.h @@ -62,6 +62,15 @@ #define ldlm_set_block_wait(_l) LDLM_SET_FLAG((_l), 1ULL << 3) #define ldlm_clear_block_wait(_l) LDLM_CLEAR_FLAG((_l), 1ULL << 3) +/** + * Lock request is speculative/asynchronous, and cannot wait for any reason. + * Fail the lock request if any blocking locks are encountered. + */ +#define LDLM_FL_SPECULATIVE 0x0000000000000010ULL /* bit 4 */ +#define ldlm_is_speculative(_l) LDLM_TEST_FLAG((_l), 1ULL << 4) +#define ldlm_set_speculative(_l) LDLM_SET_FLAG((_l), 1ULL << 4) +#define ldlm_clear_specualtive_(_l) LDLM_CLEAR_FLAG((_l), 1ULL << 4) + /** blocking or cancel packet was queued for sending. */ #define LDLM_FL_AST_SENT 0x0000000000000020ULL /* bit 5 */ #define ldlm_is_ast_sent(_l) LDLM_TEST_FLAG((_l), 1ULL << 5) @@ -137,6 +146,25 @@ #define ldlm_clear_cancel_on_block(_l) LDLM_CLEAR_FLAG((_l), 1ULL << 23) /** + * Part of original lockahead implementation, OBD_CONNECT_LOCKAHEAD_OLD. + * Reserved temporarily to allow those implementations to keep working. + * Will be removed after 2.12 release. + */ +#define LDLM_FL_LOCKAHEAD_OLD_RESERVED 0x0000000010000000ULL /* bit 28 */ +#define ldlm_is_do_not_expand_io(_l) LDLM_TEST_FLAG((_l), 1ULL << 28) +#define ldlm_set_do_not_expand_io(_l) LDLM_SET_FLAG((_l), 1ULL << 28) +#define ldlm_clear_do_not_expand_io(_l) LDLM_CLEAR_FLAG((_l), 1ULL << 28) + +/** + * Do not expand this lock. Grant it only on the extent requested. + * Used for manually requested locks from the client (LU_LADVISE_LOCKAHEAD). + */ +#define LDLM_FL_NO_EXPANSION 0x0000000020000000ULL /* bit 29 */ +#define ldlm_is_do_not_expand(_l) LDLM_TEST_FLAG((_l), 1ULL << 29) +#define ldlm_set_do_not_expand(_l) LDLM_SET_FLAG((_l), 1ULL << 29) +#define ldlm_clear_do_not_expand(_l) LDLM_CLEAR_FLAG((_l), 1ULL << 29) + +/** * measure lock contention and return -EUSERS if locking contention is high */ #define LDLM_FL_DENY_ON_CONTENTION 0x0000000040000000ULL /* bit 30 */ @@ -376,13 +404,16 @@ #define LDLM_FL_GONE_MASK (LDLM_FL_DESTROYED |\ LDLM_FL_FAILED) -/** l_flags bits marked as "inherit" bits */ -/* Flags inherited from wire on enqueue/reply between client/server. */ -/* NO_TIMEOUT flag to force ldlm_lock_match() to wait with no timeout. */ -/* TEST_LOCK flag to not let TEST lock to be granted. */ +/** l_flags bits marked as "inherit" bits + * Flags inherited from wire on enqueue/reply between client/server. + * NO_TIMEOUT flag to force ldlm_lock_match() to wait with no timeout. + * TEST_LOCK flag to not let TEST lock to be granted. + * NO_EXPANSION to tell server not to expand extent of lock request + */ #define LDLM_FL_INHERIT_MASK (LDLM_FL_CANCEL_ON_BLOCK |\ LDLM_FL_NO_TIMEOUT |\ - LDLM_FL_TEST_LOCK) + LDLM_FL_TEST_LOCK |\ + LDLM_FL_NO_EXPANSION) /** test for ldlm_lock flag bit set */ #define LDLM_TEST_FLAG(_l, _b) (((_l)->l_flags & (_b)) != 0) diff --git a/fs/lustre/include/lustre_export.h b/fs/lustre/include/lustre_export.h index fb34e0b..ed0664b 100644 --- a/fs/lustre/include/lustre_export.h +++ b/fs/lustre/include/lustre_export.h @@ -149,6 +149,16 @@ static inline u64 exp_connect_flags(struct obd_export *exp) return *exp_connect_flags_ptr(exp); } +static inline u64 *exp_connect_flags2_ptr(struct obd_export *exp) +{ + return &exp->exp_connect_data.ocd_connect_flags2; +} + +static inline u64 exp_connect_flags2(struct obd_export *exp) +{ + return *exp_connect_flags2_ptr(exp); +} + static inline int exp_max_brw_size(struct obd_export *exp) { if (exp_connect_flags(exp) & OBD_CONNECT_BRW_SIZE) @@ -235,6 +245,16 @@ static inline bool imp_connect_disp_stripe(struct obd_import *imp) return ocd->ocd_connect_flags & OBD_CONNECT_DISP_STRIPE; } +static inline int exp_connect_lockahead_old(struct obd_export *exp) +{ + return !!(exp_connect_flags(exp) & OBD_CONNECT_LOCKAHEAD_OLD); +} + +static inline int exp_connect_lockahead(struct obd_export *exp) +{ + return !!(exp_connect_flags2(exp) & OBD_CONNECT2_LOCKAHEAD); +} + struct obd_export *class_conn2export(struct lustre_handle *conn); #define KKUC_CT_DATA_MAGIC 0x092013cea diff --git a/fs/lustre/include/lustre_osc.h b/fs/lustre/include/lustre_osc.h index f96583a..6c8a9d4 100644 --- a/fs/lustre/include/lustre_osc.h +++ b/fs/lustre/include/lustre_osc.h @@ -380,7 +380,16 @@ struct osc_lock { /* * For async glimpse lock. */ - ols_agl:1; + ols_agl:1, + /* + * for speculative locks - asynchronous glimpse locks and ladvise + * lockahead manual lock requests + * + * Used to tell osc layer to not wait for the ldlm reply from the + * server, so the osc lock will be short lived - It only exists to + * create the ldlm request and is not updated on request completion. + */ + ols_speculative:1; }; /* diff --git a/fs/lustre/ldlm/ldlm_lib.c b/fs/lustre/ldlm/ldlm_lib.c index 04982b8..67f3297 100644 --- a/fs/lustre/ldlm/ldlm_lib.c +++ b/fs/lustre/ldlm/ldlm_lib.c @@ -558,6 +558,7 @@ int client_connect_import(const struct lu_env *env, ocd->ocd_connect_flags, "old %#llx, new %#llx\n", data->ocd_connect_flags, ocd->ocd_connect_flags); data->ocd_connect_flags = ocd->ocd_connect_flags; + data->ocd_connect_flags2 = ocd->ocd_connect_flags2; } ptlrpc_pinger_add_import(imp); diff --git a/fs/lustre/ldlm/ldlm_lock.c b/fs/lustre/ldlm/ldlm_lock.c index 2f2b1ab..ddebcf6 100644 --- a/fs/lustre/ldlm/ldlm_lock.c +++ b/fs/lustre/ldlm/ldlm_lock.c @@ -43,6 +43,8 @@ #include #include "ldlm_internal.h" +struct kmem_cache *ldlm_glimpse_work_kmem; + /* lock types */ char *ldlm_lockname[] = { [0] = "--", @@ -1756,8 +1758,11 @@ static int ldlm_work_gl_ast_lock(struct ptlrpc_request_set *rqset, void *opaq) LDLM_LOCK_RELEASE(lock); - if ((gl_work->gl_flags & LDLM_GL_WORK_NOFREE) == 0) + if (gl_work->gl_flags & LDLM_GL_WORK_SLAB_ALLOCATED) + kmem_cache_free(ldlm_glimpse_work_kmem, gl_work); + else kfree(gl_work); + gl_work = NULL; return rc; } diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 53b4620..c393856 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -1112,9 +1112,12 @@ static bool file_is_noatime(const struct file *file) static void ll_io_init(struct cl_io *io, const struct file *file, int write) { + struct ll_file_data *fd = LUSTRE_FPRIVATE(file); struct inode *inode = file_inode(file); io->u.ci_rw.crw_nonblock = file->f_flags & O_NONBLOCK; + io->ci_lock_no_expand = fd->ll_lock_no_expand; + if (write) { io->u.ci_wr.wr_append = !!(file->f_flags & O_APPEND); io->u.ci_wr.wr_sync = file->f_flags & O_SYNC || @@ -2168,6 +2171,203 @@ static int ll_file_futimes_3(struct file *file, const struct ll_futimes_3 *lfu) return rc; } +static enum cl_lock_mode cl_mode_user_to_kernel(enum lock_mode_user mode) +{ + enum cl_lock_mode cl_mode; + + switch (mode) { + case MODE_READ_USER: + cl_mode = CLM_READ; + break; + case MODE_WRITE_USER: + cl_mode = CLM_WRITE; + break; + default: + cl_mode = -EINVAL; + break; + } + return cl_mode; +} + +static const char *const user_lockname[] = LOCK_MODE_NAMES; + +/* Used to allow the upper layers of the client to request an LDLM lock + * without doing an actual read or write. + * + * Used for ladvise lockahead to manually request specific locks. + * + * @file file this ladvise lock request is on + * @ladvise ladvise struct describing this lock request + * + * Return 0 on success, no detailed result available (sync requests + * and requests sent to the server [not handled locally] + * cannot return detailed results) + * + * LLA_RESULT_{SAME,DIFFERENT} - detailed result of the lock + * request, see definitions for details. + * + * negative errno on error + */ +int ll_file_lock_ahead(struct file *file, struct llapi_lu_ladvise *ladvise) +{ + struct dentry *dentry = file->f_path.dentry; + struct inode *inode = dentry->d_inode; + struct cl_lock_descr *descr = NULL; + struct cl_lock *lock = NULL; + struct cl_io *io = NULL; + struct lu_env *env = NULL; + enum cl_lock_mode cl_mode; + u64 start = ladvise->lla_start; + u64 end = ladvise->lla_end; + u16 refcheck; + int result; + + CDEBUG(D_VFSTRACE, + "Lock request: file=%.*s, inode=%p, mode=%s start=%llu, end=%llu\n", + dentry->d_name.len, dentry->d_name.name, dentry->d_inode, + user_lockname[ladvise->lla_lockahead_mode], (__u64) start, end); + + cl_mode = cl_mode_user_to_kernel(ladvise->lla_lockahead_mode); + if (cl_mode < 0) { + result = cl_mode; + goto out; + } + + /* Get IO environment */ + result = cl_io_get(inode, &env, &io, &refcheck); + if (result <= 0) + goto out; + + result = cl_io_init(env, io, CIT_MISC, io->ci_obj); + if (result > 0) { + /* + * nothing to do for this io. This currently happens when + * stripe sub-object's are not yet created. + */ + result = io->ci_result; + } else if (result == 0) { + lock = vvp_env_lock(env); + descr = &lock->cll_descr; + + descr->cld_obj = io->ci_obj; + /* Convert byte offsets to pages */ + descr->cld_start = cl_index(io->ci_obj, start); + descr->cld_end = cl_index(io->ci_obj, end); + descr->cld_mode = cl_mode; + /* CEF_MUST is used because we do not want to convert a + * lockahead request to a lockless lock + */ + descr->cld_enq_flags = CEF_MUST | CEF_LOCK_NO_EXPAND | + CEF_NONBLOCK; + + if (ladvise->lla_peradvice_flags & LF_ASYNC) + descr->cld_enq_flags |= CEF_SPECULATIVE; + + result = cl_lock_request(env, io, lock); + + /* On success, we need to release the lock */ + if (result >= 0) + cl_lock_release(env, lock); + } + cl_io_fini(env, io); + cl_env_put(env, &refcheck); + + /* -ECANCELED indicates a matching lock with a different extent + * was already present, and -EEXIST indicates a matching lock + * on exactly the same extent was already present. + * We convert them to positive values for userspace to make + * recognizing true errors easier. + * Note we can only return these detailed results on async requests, + * as sync requests look the same as i/o requests for locking. + */ + if (result == -ECANCELED) + result = LLA_RESULT_DIFFERENT; + else if (result == -EEXIST) + result = LLA_RESULT_SAME; + +out: + return result; +} + +static const char *const ladvise_names[] = LU_LADVISE_NAMES; + +static int ll_ladvise_sanity(struct inode *inode, + struct llapi_lu_ladvise *ladvise) +{ + enum lu_ladvise_type advice = ladvise->lla_advice; + /* Note the peradvice flags is a 32 bit field, so per advice flags must + * be in the first 32 bits of enum ladvise_flags + */ + u32 flags = ladvise->lla_peradvice_flags; + /* 3 lines at 80 characters per line, should be plenty */ + int rc = 0; + + if (advice > LU_LADVISE_MAX || advice == LU_LADVISE_INVALID) { + rc = -EINVAL; + CDEBUG(D_VFSTRACE, + "%s: advice with value '%d' not recognized, last supported advice is %s (value '%d'): rc = %d\n", + ll_get_fsname(inode->i_sb, NULL, 0), advice, + ladvise_names[LU_LADVISE_MAX - 1], LU_LADVISE_MAX - 1, + rc); + goto out; + } + + /* Per-advice checks */ + switch (advice) { + case LU_LADVISE_LOCKNOEXPAND: + if (flags & ~LF_LOCKNOEXPAND_MASK) { + rc = -EINVAL; + CDEBUG(D_VFSTRACE, + "%s: Invalid flags (%x) for %s: rc = %d\n", + ll_get_fsname(inode->i_sb, NULL, 0), flags, + ladvise_names[advice], rc); + goto out; + } + break; + case LU_LADVISE_LOCKAHEAD: + /* Currently only READ and WRITE modes can be requested */ + if (ladvise->lla_lockahead_mode >= MODE_MAX_USER || + ladvise->lla_lockahead_mode == 0) { + rc = -EINVAL; + CDEBUG(D_VFSTRACE, + "%s: Invalid mode (%d) for %s: rc = %d\n", + ll_get_fsname(inode->i_sb, NULL, 0), + ladvise->lla_lockahead_mode, + ladvise_names[advice], rc); + goto out; + } + /* fallthrough */ + case LU_LADVISE_WILLREAD: + case LU_LADVISE_DONTNEED: + default: + /* Note fall through above - These checks apply to all advices + * except LOCKNOEXPAND + */ + if (flags & ~LF_DEFAULT_MASK) { + rc = -EINVAL; + CDEBUG(D_VFSTRACE, + "%s: Invalid flags (%x) for %s: rc = %d\n", + ll_get_fsname(inode->i_sb, NULL, 0), flags, + ladvise_names[advice], rc); + goto out; + } + if (ladvise->lla_start >= ladvise->lla_end) { + rc = -EINVAL; + CDEBUG(D_VFSTRACE, + "%s: Invalid range (%llu to %llu) for %s: rc = %d\n", + ll_get_fsname(inode->i_sb, NULL, 0), + ladvise->lla_start, ladvise->lla_end, + ladvise_names[advice], rc); + goto out; + } + break; + } + +out: + return rc; +} +#undef ERRSIZE + /* * Give file access advices * @@ -2216,6 +2416,15 @@ static int ll_ladvise(struct inode *inode, struct file *file, u64 flags, return rc; } +static int ll_lock_noexpand(struct file *file, int flags) +{ + struct ll_file_data *fd = LUSTRE_FPRIVATE(file); + + fd->ll_lock_no_expand = !(flags & LF_UNSET); + + return 0; +} + int ll_ioctl_fsgetxattr(struct inode *inode, unsigned int cmd, unsigned long arg) { @@ -2634,61 +2843,89 @@ int ll_ioctl_fssetxattr(struct inode *inode, unsigned int cmd, return ll_file_futimes_3(file, &lfu); } case LL_IOC_LADVISE: { - struct llapi_ladvise_hdr *ladvise_hdr; - int alloc_size = sizeof(*ladvise_hdr); + struct llapi_ladvise_hdr __user *u_ladvise_hdr; + struct llapi_ladvise_hdr *k_ladvise_hdr; + int alloc_size = sizeof(*k_ladvise_hdr); int num_advise; int i; rc = 0; - ladvise_hdr = kzalloc(alloc_size, GFP_KERNEL); - if (!ladvise_hdr) + u_ladvise_hdr = (void __user *)arg; + k_ladvise_hdr = kzalloc(alloc_size, GFP_KERNEL); + if (!k_ladvise_hdr) return -ENOMEM; - if (copy_from_user(ladvise_hdr, - (const struct llapi_ladvise_hdr __user *)arg, - alloc_size)) { + if (copy_from_user(k_ladvise_hdr, u_ladvise_hdr, alloc_size)) { rc = -EFAULT; goto out_ladvise; } - if (ladvise_hdr->lah_magic != LADVISE_MAGIC || - ladvise_hdr->lah_count < 1) { + if (k_ladvise_hdr->lah_magic != LADVISE_MAGIC || + k_ladvise_hdr->lah_count < 1) { rc = -EINVAL; goto out_ladvise; } - num_advise = ladvise_hdr->lah_count; + num_advise = k_ladvise_hdr->lah_count; if (num_advise >= LAH_COUNT_MAX) { rc = -EFBIG; goto out_ladvise; } - kfree(ladvise_hdr); - alloc_size = offsetof(typeof(*ladvise_hdr), + kfree(k_ladvise_hdr); + alloc_size = offsetof(typeof(*k_ladvise_hdr), lah_advise[num_advise]); - ladvise_hdr = kzalloc(alloc_size, GFP_KERNEL); - if (!ladvise_hdr) + k_ladvise_hdr = kzalloc(alloc_size, GFP_KERNEL); + if (!k_ladvise_hdr) return -ENOMEM; /* * TODO: submit multiple advices to one server in a single RPC */ - if (copy_from_user(ladvise_hdr, - (const struct llapi_advise_hdr __user *)arg, - alloc_size)) { + if (copy_from_user(k_ladvise_hdr, u_ladvise_hdr, alloc_size)) { rc = -EFAULT; goto out_ladvise; } for (i = 0; i < num_advise; i++) { - rc = ll_ladvise(inode, file, ladvise_hdr->lah_flags, - &ladvise_hdr->lah_advise[i]); + struct llapi_lu_ladvise __user *u_ladvise; + struct llapi_lu_ladvise *k_ladvise; + + k_ladvise = &k_ladvise_hdr->lah_advise[i]; + u_ladvise = &u_ladvise_hdr->lah_advise[i]; + + rc = ll_ladvise_sanity(inode, k_ladvise); if (rc) + goto out_ladvise; + + switch (k_ladvise->lla_advice) { + case LU_LADVISE_LOCKNOEXPAND: + rc = ll_lock_noexpand(file, + k_ladvise->lla_peradvice_flags); + goto out_ladvise; + case LU_LADVISE_LOCKAHEAD: + rc = ll_file_lock_ahead(file, k_ladvise); + if (rc < 0) + goto out_ladvise; + + if (put_user(rc, + &u_ladvise->lla_lockahead_result)) { + rc = -EFAULT; + goto out_ladvise; + } break; + default: + rc = ll_ladvise(inode, file, + k_ladvise_hdr->lah_flags, + k_ladvise); + if (rc) + goto out_ladvise; + break; + } } out_ladvise: - kfree(ladvise_hdr); + kfree(k_ladvise_hdr); return rc; } case FS_IOC_FSGETXATTR: diff --git a/fs/lustre/llite/glimpse.c b/fs/lustre/llite/glimpse.c index 05c267f..56a5094 100644 --- a/fs/lustre/llite/glimpse.c +++ b/fs/lustre/llite/glimpse.c @@ -88,7 +88,7 @@ int cl_glimpse_lock(const struct lu_env *env, struct cl_io *io, CDEBUG(D_DLMTRACE, "Glimpsing inode " DFID "\n", PFID(fid)); /* NOTE: this looks like DLM lock request, but it may - * not be one. Due to CEF_ASYNC flag (translated + * not be one. Due to CEF_GLIMPSE flag (translated * to LDLM_FL_HAS_INTENT by osc), this is * glimpse request, that won't revoke any * conflicting DLM locks held. Instead, @@ -104,14 +104,10 @@ int cl_glimpse_lock(const struct lu_env *env, struct cl_io *io, *descr = whole_file; descr->cld_obj = clob; descr->cld_mode = CLM_READ; - descr->cld_enq_flags = CEF_ASYNC | CEF_MUST; + descr->cld_enq_flags = CEF_GLIMPSE | CEF_MUST; if (agl) - descr->cld_enq_flags |= CEF_AGL; + descr->cld_enq_flags |= CEF_SPECULATIVE | CEF_NONBLOCK; /* - * CEF_ASYNC is used because glimpse sub-locks cannot - * deadlock (because they never conflict with other - * locks) and, hence, can be enqueued out-of-order. - * * CEF_MUST protects glimpse lock from conversion into * a lockless mode. */ @@ -137,8 +133,21 @@ int cl_glimpse_lock(const struct lu_env *env, struct cl_io *io, return result; } -static int cl_io_get(struct inode *inode, struct lu_env **envout, - struct cl_io **ioout, u16 *refcheck) +/** + * Get an IO environment for special operations such as glimpse locks and + * manually requested locks (ladvise lockahead) + * + * @inode inode the operation is being performed on + * @envout thread specific execution environment + * @ioout client io description + * @refcheck reference check + * + * Return 1 on success + * 0 not a regular file, cannot get environment + * negative errno on error + */ +int cl_io_get(struct inode *inode, struct lu_env **envout, + struct cl_io **ioout, u16 *refcheck) { struct lu_env *env; struct cl_io *io; diff --git a/fs/lustre/llite/llite_internal.h b/fs/lustre/llite/llite_internal.h index b8f4e0e..392d82c 100644 --- a/fs/lustre/llite/llite_internal.h +++ b/fs/lustre/llite/llite_internal.h @@ -652,6 +652,7 @@ struct ll_file_data { * false: unknown failure, should report. */ bool fd_write_failed; + bool ll_lock_no_expand; rwlock_t fd_lock; /* protect lcc list */ struct list_head fd_lccs; /* list of ll_cl_context */ }; @@ -1163,11 +1164,19 @@ static inline int cl_glimpse_size(struct inode *inode) return __cl_glimpse_size(inode, 0); } +/* AGL is 'asychronous glimpse lock', which is a speculative lock taken as + * part of statahead + */ static inline int cl_agl(struct inode *inode) { return __cl_glimpse_size(inode, 1); } +int ll_file_lock_ahead(struct file *file, struct llapi_lu_ladvise *ladvise); + +int cl_io_get(struct inode *inode, struct lu_env **envout, + struct cl_io **ioout, __u16 *refcheck); + static inline int ll_glimpse_size(struct inode *inode) { struct ll_inode_info *lli = ll_i2info(inode); diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index 83b63d8..3896dc7 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -185,7 +185,7 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt) */ data->ocd_grant_blkbits = PAGE_SHIFT; - /* indicate the features supported by this client */ + /* indicate MDT features supported by this client */ data->ocd_connect_flags = OBD_CONNECT_IBITS | OBD_CONNECT_NODEVOH | OBD_CONNECT_ATTRFID | OBD_CONNECT_VERSION | OBD_CONNECT_BRW_SIZE | @@ -374,6 +374,7 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt) goto out_md_fid; } + /* indicate OST features supported by this client */ data->ocd_connect_flags = OBD_CONNECT_GRANT | OBD_CONNECT_VERSION | OBD_CONNECT_REQPORTAL | OBD_CONNECT_BRW_SIZE | OBD_CONNECT_CANCELSET | OBD_CONNECT_FID | @@ -386,9 +387,25 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt) OBD_CONNECT_JOBSTATS | OBD_CONNECT_LVB_TYPE | OBD_CONNECT_LAYOUTLOCK | OBD_CONNECT_PINGLESS | OBD_CONNECT_LFSCK | - OBD_CONNECT_BULK_MBITS; + OBD_CONNECT_BULK_MBITS | OBD_CONNECT_FLAGS2; + + /* The client currently advertises support for OBD_CONNECT_LOCKAHEAD_OLD + * so it can interoperate with an older version of lockahead which was + * released prior to landing in master. This support will be dropped + * when 2.13 development starts. At the point, we should not just drop + * the connect flag (below), we should also remove the support in the + * code. + * + * Removing it means a few things: + * 1. Remove this section here + * 2. Remove CEF_NONBLOCK in ll_file_lockahead() + * 3. Remove function exp_connect_lockahead_old + * 4. Remove LDLM_FL_LOCKAHEAD_OLD_RESERVED in lustre_dlm_flags.h + */ + if (data->ocd_version < OBD_OCD_VERSION(2, 12, 50, 0)) + data->ocd_connect_flags |= OBD_CONNECT_LOCKAHEAD_OLD; - data->ocd_connect_flags2 = 0; + data->ocd_connect_flags2 = OBD_CONNECT2_LOCKAHEAD; if (!OBD_FAIL_CHECK(OBD_FAIL_OSC_CONNECT_GRANT_PARAM)) data->ocd_connect_flags |= OBD_CONNECT_GRANT_PARAM; diff --git a/fs/lustre/llite/vvp_io.c b/fs/lustre/llite/vvp_io.c index b2160d6..dea820e 100644 --- a/fs/lustre/llite/vvp_io.c +++ b/fs/lustre/llite/vvp_io.c @@ -524,6 +524,9 @@ static int vvp_io_rw_lock(const struct lu_env *env, struct cl_io *io, if (io->u.ci_rw.crw_nonblock) ast_flags |= CEF_NONBLOCK; + if (io->ci_lock_no_expand) + ast_flags |= CEF_LOCK_NO_EXPAND; + result = vvp_mmap_locks(env, vio, io); if (result == 0) result = vvp_io_one_lock(env, io, ast_flags, mode, start, end); diff --git a/fs/lustre/lov/lov_io.c b/fs/lustre/lov/lov_io.c index 8214493..a72069f 100644 --- a/fs/lustre/lov/lov_io.c +++ b/fs/lustre/lov/lov_io.c @@ -120,6 +120,7 @@ static int lov_io_sub_init(const struct lu_env *env, struct lov_io *lio, sub_io->ci_type = io->ci_type; sub_io->ci_no_srvlock = io->ci_no_srvlock; sub_io->ci_noatime = io->ci_noatime; + sub_io->ci_lock_no_expand = io->ci_lock_no_expand; rc = cl_io_sub_init(sub->sub_env, sub_io, io->ci_type, sub_obj); if (rc < 0) diff --git a/fs/lustre/obdclass/cl_lock.c b/fs/lustre/obdclass/cl_lock.c index af88383..fe24379 100644 --- a/fs/lustre/obdclass/cl_lock.c +++ b/fs/lustre/obdclass/cl_lock.c @@ -188,7 +188,7 @@ int cl_lock_request(const struct lu_env *env, struct cl_io *io, if (rc < 0) return rc; - if ((enq_flags & CEF_ASYNC) && !(enq_flags & CEF_AGL)) { + if ((enq_flags & CEF_GLIMPSE) && !(enq_flags & CEF_SPECULATIVE)) { anchor = &cl_env_info(env)->clt_anchor; cl_sync_io_init(anchor, 1); } diff --git a/fs/lustre/obdclass/lprocfs_status.c b/fs/lustre/obdclass/lprocfs_status.c index 01b8f90..1ff12d55 100644 --- a/fs/lustre/obdclass/lprocfs_status.c +++ b/fs/lustre/obdclass/lprocfs_status.c @@ -106,12 +106,13 @@ "multi_mod_rpcs", "dir_stripe", "subtree", - "lock_ahead", + "lockahead", "bulk_mbits", "compact_obdo", "second_flags", /* flags2 names */ "file_secctx", + "lockaheadv2", NULL }; diff --git a/fs/lustre/osc/osc_internal.h b/fs/lustre/osc/osc_internal.h index af2f237..72cf131 100644 --- a/fs/lustre/osc/osc_internal.h +++ b/fs/lustre/osc/osc_internal.h @@ -53,7 +53,8 @@ int osc_enqueue_base(struct obd_export *exp, struct ldlm_res_id *res_id, struct ost_lvb *lvb, int kms_valid, osc_enqueue_upcall_f upcall, void *cookie, struct ldlm_enqueue_info *einfo, - struct ptlrpc_request_set *rqset, int async, int agl); + struct ptlrpc_request_set *rqset, int async, + bool speculative); int osc_match_base(struct obd_export *exp, struct ldlm_res_id *res_id, enum ldlm_type type, union ldlm_policy_data *policy, diff --git a/fs/lustre/osc/osc_lock.c b/fs/lustre/osc/osc_lock.c index 917ae71..f976230 100644 --- a/fs/lustre/osc/osc_lock.c +++ b/fs/lustre/osc/osc_lock.c @@ -158,11 +158,13 @@ static u64 osc_enq2ldlm_flags(u32 enqflags) { u64 result = 0; + CDEBUG(D_DLMTRACE, "flags: %x\n", enqflags); + LASSERT((enqflags & ~CEF_MASK) == 0); if (enqflags & CEF_NONBLOCK) result |= LDLM_FL_BLOCK_NOWAIT; - if (enqflags & CEF_ASYNC) + if (enqflags & CEF_GLIMPSE) result |= LDLM_FL_HAS_INTENT; if (enqflags & CEF_DISCARD_DATA) result |= LDLM_FL_AST_DISCARD_DATA; @@ -170,6 +172,10 @@ static u64 osc_enq2ldlm_flags(u32 enqflags) result |= LDLM_FL_TEST_LOCK; if (enqflags & CEF_LOCK_MATCH) result |= LDLM_FL_MATCH_LOCK; + if (enqflags & CEF_LOCK_NO_EXPAND) + result |= LDLM_FL_NO_EXPANSION; + if (enqflags & CEF_SPECULATIVE) + result |= LDLM_FL_SPECULATIVE; return result; } @@ -345,8 +351,9 @@ static int osc_lock_upcall(void *cookie, struct lustre_handle *lockh, return rc; } -static int osc_lock_upcall_agl(void *cookie, struct lustre_handle *lockh, - int errcode) +static int osc_lock_upcall_speculative(void *cookie, + struct lustre_handle *lockh, + int errcode) { struct osc_object *osc = cookie; struct ldlm_lock *dlmlock; @@ -370,7 +377,7 @@ static int osc_lock_upcall_agl(void *cookie, struct lustre_handle *lockh, lock_res_and_lock(dlmlock); LASSERT(dlmlock->l_granted_mode == dlmlock->l_req_mode); - /* there is no osc_lock associated with AGL lock */ + /* there is no osc_lock associated with speculative lock */ osc_lock_lvb_update(env, osc, dlmlock, NULL); unlock_res_and_lock(dlmlock); @@ -808,7 +815,7 @@ static bool osc_lock_compatible(const struct osc_lock *qing, struct cl_lock_descr *qed_descr = &qed->ols_cl.cls_lock->cll_descr; struct cl_lock_descr *qing_descr = &qing->ols_cl.cls_lock->cll_descr; - if (qed->ols_glimpse) + if (qed->ols_glimpse || qed->ols_speculative) return true; if (qing_descr->cld_mode == CLM_READ && qed_descr->cld_mode == CLM_READ) @@ -925,13 +932,14 @@ static int osc_lock_enqueue(const struct lu_env *env, struct osc_io *oio = osc_env_io(env); struct osc_object *osc = cl2osc(slice->cls_obj); struct osc_lock *oscl = cl2osc_lock(slice); + struct obd_export *exp = osc_export(osc); struct cl_lock *lock = slice->cls_lock; struct ldlm_res_id *resname = &info->oti_resname; union ldlm_policy_data *policy = &info->oti_policy; osc_enqueue_upcall_f upcall = osc_lock_upcall; void *cookie = oscl; bool async = false; - int result; + int result = 0; LASSERTF(ergo(oscl->ols_glimpse, lock->cll_descr.cld_mode <= CLM_READ), "lock = %p, ols = %p\n", lock, oscl); @@ -939,11 +947,23 @@ static int osc_lock_enqueue(const struct lu_env *env, if (oscl->ols_state == OLS_GRANTED) return 0; + if ((oscl->ols_flags & LDLM_FL_NO_EXPANSION) && + !(exp_connect_lockahead_old(exp) || exp_connect_lockahead(exp))) { + result = -EOPNOTSUPP; + CERROR("%s: server does not support lockahead/locknoexpand: rc = %d\n", + exp->exp_obd->obd_name, result); + return result; + } + if (oscl->ols_flags & LDLM_FL_TEST_LOCK) goto enqueue_base; - if (oscl->ols_glimpse) { - LASSERT(equi(oscl->ols_agl, !anchor)); + /* For glimpse and/or speculative locks, do not wait for reply from + * server on LDLM request + */ + if (oscl->ols_glimpse || oscl->ols_speculative) { + /* Speculative and glimpse locks do not have an anchor */ + LASSERT(equi(oscl->ols_speculative, !anchor)); async = true; goto enqueue_base; } @@ -970,25 +990,31 @@ static int osc_lock_enqueue(const struct lu_env *env, /** * DLM lock's ast data must be osc_object; - * if glimpse or AGL lock, async of osc_enqueue_base() must be true, + * if glimpse or speculative lock, async of osc_enqueue_base() + * must be true + * + * For non-speculative locks: * DLM's enqueue callback set to osc_lock_upcall() with cookie as * osc_lock. + * For speculative locks: + * osc_lock_upcall_speculative & cookie is the osc object, since + * there is no osc_lock */ ostid_build_res_name(&osc->oo_oinfo->loi_oi, resname); osc_lock_build_policy(env, lock, policy); - if (oscl->ols_agl) { + if (oscl->ols_speculative) { oscl->ols_einfo.ei_cbdata = NULL; /* hold a reference for callback */ cl_object_get(osc2cl(osc)); - upcall = osc_lock_upcall_agl; + upcall = osc_lock_upcall_speculative; cookie = osc; } - result = osc_enqueue_base(osc_export(osc), resname, &oscl->ols_flags, + result = osc_enqueue_base(exp, resname, &oscl->ols_flags, policy, &oscl->ols_lvb, osc->oo_oinfo->loi_kms_valid, upcall, cookie, &oscl->ols_einfo, PTLRPCD_SET, async, - oscl->ols_agl); + oscl->ols_speculative); if (!result) { if (osc_lock_is_lockless(oscl)) { oio->oi_lockless = 1; @@ -997,9 +1023,12 @@ static int osc_lock_enqueue(const struct lu_env *env, LASSERT(oscl->ols_hold); LASSERT(oscl->ols_dlmlock); } - } else if (oscl->ols_agl) { + } else if (oscl->ols_speculative) { cl_object_put(env, osc2cl(osc)); - result = 0; + if (oscl->ols_glimpse) { + /* hide error for AGL request */ + result = 0; + } } out: @@ -1161,10 +1190,16 @@ int osc_lock_init(const struct lu_env *env, INIT_LIST_HEAD(&oscl->ols_wait_entry); INIT_LIST_HEAD(&oscl->ols_nextlock_oscobj); + /* Speculative lock requests must be either no_expand or glimpse + * request (CEF_GLIMPSE). non-glimpse no_expand speculative extent + * locks will break ofd_intent_cb. (see comment there) + */ + LASSERT(ergo((enqflags & CEF_SPECULATIVE) != 0, + (enqflags & (CEF_LOCK_NO_EXPAND | CEF_GLIMPSE)) != 0)); + oscl->ols_flags = osc_enq2ldlm_flags(enqflags); - oscl->ols_agl = !!(enqflags & CEF_AGL); - if (oscl->ols_agl) - oscl->ols_flags |= LDLM_FL_BLOCK_NOWAIT; + oscl->ols_speculative = !!(enqflags & CEF_SPECULATIVE); + if (oscl->ols_flags & LDLM_FL_HAS_INTENT) { oscl->ols_flags |= LDLM_FL_BLOCK_GRANTED; oscl->ols_glimpse = 1; diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index 64c4fd1..e703cf1 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -106,7 +106,7 @@ struct osc_enqueue_args { void *oa_cookie; struct ost_lvb *oa_lvb; struct lustre_handle oa_lockh; - unsigned int oa_agl:1; + unsigned int oa_speculative; }; static void osc_release_ppga(struct brw_page **ppga, u32 count); @@ -2044,7 +2044,7 @@ static int osc_set_lock_data(struct ldlm_lock *lock, void *data) static int osc_enqueue_fini(struct ptlrpc_request *req, osc_enqueue_upcall_f upcall, void *cookie, struct lustre_handle *lockh, enum ldlm_mode mode, - u64 *flags, int agl, int errcode) + u64 *flags, int speculative, int errcode) { bool intent = *flags & LDLM_FL_HAS_INTENT; int rc; @@ -2059,7 +2059,7 @@ static int osc_enqueue_fini(struct ptlrpc_request *req, ptlrpc_status_ntoh(rep->lock_policy_res1); if (rep->lock_policy_res1) errcode = rep->lock_policy_res1; - if (!agl) + if (!speculative) *flags |= LDLM_FL_LVB_READY; } else if (errcode == ELDLM_OK) { *flags |= LDLM_FL_LVB_READY; @@ -2107,7 +2107,7 @@ static int osc_enqueue_interpret(const struct lu_env *env, /* Let CP AST to grant the lock first. */ OBD_FAIL_TIMEOUT(OBD_FAIL_OSC_CP_ENQ_RACE, 1); - if (aa->oa_agl) { + if (aa->oa_speculative) { LASSERT(!aa->oa_lvb); LASSERT(!aa->oa_flags); aa->oa_flags = &flags; @@ -2119,7 +2119,7 @@ static int osc_enqueue_interpret(const struct lu_env *env, lockh, rc); /* Complete osc stuff. */ rc = osc_enqueue_fini(req, aa->oa_upcall, aa->oa_cookie, lockh, mode, - aa->oa_flags, aa->oa_agl, rc); + aa->oa_flags, aa->oa_speculative, rc); OBD_FAIL_TIMEOUT(OBD_FAIL_OSC_CP_CANCEL_RACE, 10); @@ -2141,7 +2141,8 @@ int osc_enqueue_base(struct obd_export *exp, struct ldlm_res_id *res_id, struct ost_lvb *lvb, int kms_valid, osc_enqueue_upcall_f upcall, void *cookie, struct ldlm_enqueue_info *einfo, - struct ptlrpc_request_set *rqset, int async, int agl) + struct ptlrpc_request_set *rqset, int async, + bool speculative) { struct obd_device *obd = exp->exp_obd; struct lustre_handle lockh = { 0 }; @@ -2182,7 +2183,11 @@ int osc_enqueue_base(struct obd_export *exp, struct ldlm_res_id *res_id, mode = einfo->ei_mode; if (einfo->ei_mode == LCK_PR) mode |= LCK_PW; - if (agl == 0) + /* Normal lock requests must wait for the LVB to be ready before + * matching a lock; speculative lock requests do not need to, + * because they will not actually use the lock. + */ + if (!speculative) match_flags |= LDLM_FL_LVB_READY; if (intent != 0) match_flags |= LDLM_FL_BLOCK_GRANTED; @@ -2195,14 +2200,23 @@ int osc_enqueue_base(struct obd_export *exp, struct ldlm_res_id *res_id, return ELDLM_OK; matched = ldlm_handle2lock(&lockh); - if (agl) { - /* AGL enqueues DLM locks speculatively. Therefore if - * it already exists a DLM lock, it wll just inform the - * caller to cancel the AGL process for this stripe. + if (speculative) { + /* This DLM lock request is speculative, and does not + * have an associated IO request. Therefore if there + * is already a DLM lock, it wll just inform the + * caller to cancel the request for this stripe. */ + lock_res_and_lock(matched); + if (ldlm_extent_equal(&policy->l_extent, + &matched->l_policy_data.l_extent)) + rc = -EEXIST; + else + rc = -ECANCELED; + unlock_res_and_lock(matched); + ldlm_lock_decref(&lockh, mode); LDLM_LOCK_PUT(matched); - return -ECANCELED; + return rc; } if (osc_set_lock_data(matched, einfo->ei_cbdata)) { *flags |= LDLM_FL_LVB_READY; @@ -2254,14 +2268,14 @@ int osc_enqueue_base(struct obd_export *exp, struct ldlm_res_id *res_id, lustre_handle_copy(&aa->oa_lockh, &lockh); aa->oa_upcall = upcall; aa->oa_cookie = cookie; - aa->oa_agl = !!agl; - if (!agl) { + aa->oa_speculative = speculative; + if (!speculative) { aa->oa_flags = flags; aa->oa_lvb = lvb; } else { - /* AGL is essentially to enqueue an DLM lock - * in advance, so we don't care about the - * result of AGL enqueue. + /* speculative locks are essentially to enqueue + * a DLM lock in advance, so we don't care + * about the result of the enqueue. */ aa->oa_lvb = NULL; aa->oa_flags = NULL; @@ -2277,7 +2291,7 @@ int osc_enqueue_base(struct obd_export *exp, struct ldlm_res_id *res_id, } rc = osc_enqueue_fini(req, upcall, cookie, &lockh, einfo->ei_mode, - flags, agl, rc); + flags, speculative, rc); if (intent) ptlrpc_req_finished(req); diff --git a/fs/lustre/ptlrpc/wiretest.c b/fs/lustre/ptlrpc/wiretest.c index 5dde349..eb8bffe 100644 --- a/fs/lustre/ptlrpc/wiretest.c +++ b/fs/lustre/ptlrpc/wiretest.c @@ -40,6 +40,8 @@ #include #include #include +#include + #include "ptlrpc_internal.h" void lustre_assert_wire_constants(void) @@ -1113,14 +1115,16 @@ void lustre_assert_wire_constants(void) OBD_CONNECT_DIR_STRIPE); LASSERTF(OBD_CONNECT_SUBTREE == 0x800000000000000ULL, "found 0x%.16llxULL\n", OBD_CONNECT_SUBTREE); - LASSERTF(OBD_CONNECT_LOCK_AHEAD == 0x1000000000000000ULL, "found 0x%.16llxULL\n", - OBD_CONNECT_LOCK_AHEAD); + LASSERTF(OBD_CONNECT_LOCKAHEAD_OLD == 0x1000000000000000ULL, "found 0x%.16llxULL\n", + OBD_CONNECT_LOCKAHEAD_OLD); LASSERTF(OBD_CONNECT_OBDOPACK == 0x4000000000000000ULL, "found 0x%.16llxULL\n", OBD_CONNECT_OBDOPACK); LASSERTF(OBD_CONNECT_FLAGS2 == 0x8000000000000000ULL, "found 0x%.16llxULL\n", OBD_CONNECT_FLAGS2); LASSERTF(OBD_CONNECT2_FILE_SECCTX == 0x1ULL, "found 0x%.16llxULL\n", OBD_CONNECT2_FILE_SECCTX); + LASSERTF(OBD_CONNECT2_LOCKAHEAD == 0x2ULL, "found 0x%.16llxULL\n", + OBD_CONNECT2_LOCKAHEAD); LASSERTF(OBD_CKSUM_CRC32 == 0x00000001UL, "found 0x%.8xUL\n", (unsigned int)OBD_CKSUM_CRC32); LASSERTF(OBD_CKSUM_ADLER == 0x00000002UL, "found 0x%.8xUL\n", diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index c2f2670..ec19d31 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -774,7 +774,8 @@ struct ptlrpc_body_v2 { */ #define OBD_CONNECT_DIR_STRIPE 0x400000000000000ULL/* striped DNE dir */ #define OBD_CONNECT_SUBTREE 0x800000000000000ULL /* fileset mount */ -#define OBD_CONNECT_LOCK_AHEAD 0x1000000000000000ULL /* lock ahead */ +#define OBD_CONNECT_LOCKAHEAD_OLD 0x1000000000000000ULL /* Old Cray lockahead */ + /** bulk matchbits is sent within ptlrpc_body */ #define OBD_CONNECT_BULK_MBITS 0x2000000000000000ULL #define OBD_CONNECT_OBDOPACK 0x4000000000000000ULL /* compact OUT obdo */ @@ -783,6 +784,9 @@ struct ptlrpc_body_v2 { #define OBD_CONNECT2_FILE_SECCTX 0x1ULL /* set file security * context at create */ +#define OBD_CONNECT2_LOCKAHEAD 0x2ULL /* ladvise lockahead + * v2 + */ /* XXX README XXX: * Please DO NOT add flag values here before first ensuring that this same @@ -2097,6 +2101,12 @@ struct ldlm_extent { __u64 gid; }; +static inline bool ldlm_extent_equal(const struct ldlm_extent *ex1, + const struct ldlm_extent *ex2) +{ + return ex1->start == ex2->start && ex1->end == ex2->end; +} + struct ldlm_inodebits { __u64 bits; }; diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index d432cfd..17bad49 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -1409,11 +1409,16 @@ enum lu_ladvise_type { LU_LADVISE_INVALID = 0, LU_LADVISE_WILLREAD = 1, LU_LADVISE_DONTNEED = 2, + LU_LADVISE_LOCKNOEXPAND = 3, + LU_LADVISE_LOCKAHEAD = 4, + LU_LADVISE_MAX }; -#define LU_LADVISE_NAMES { \ - [LU_LADVISE_WILLREAD] = "willread", \ - [LU_LADVISE_DONTNEED] = "dontneed", \ +#define LU_LADVISE_NAMES { \ + [LU_LADVISE_WILLREAD] = "willread", \ + [LU_LADVISE_DONTNEED] = "dontneed", \ + [LU_LADVISE_LOCKNOEXPAND] = "locknoexpand", \ + [LU_LADVISE_LOCKAHEAD] = "lockahead", \ } /* @@ -1433,10 +1438,20 @@ struct llapi_lu_ladvise { enum ladvise_flag { LF_ASYNC = 0x00000001, + LF_UNSET = 0x00000002, }; #define LADVISE_MAGIC 0x1ADF1CE0 -#define LF_MASK LF_ASYNC +/* Masks of valid flags for each advice */ +#define LF_LOCKNOEXPAND_MASK LF_UNSET +/* Flags valid for all advices not explicitly specified */ +#define LF_DEFAULT_MASK LF_ASYNC +/* All flags */ +#define LF_MASK (LF_ASYNC | LF_UNSET) + +#define lla_lockahead_mode lla_value1 +#define lla_peradvice_flags lla_value2 +#define lla_lockahead_result lla_value3 /* * This is the userspace argument for ladvise, corresponds to ladvise_hdr which @@ -1455,6 +1470,23 @@ struct llapi_ladvise_hdr { #define LAH_COUNT_MAX 1024 +enum lock_mode_user { + MODE_READ_USER = 1, + MODE_WRITE_USER, + MODE_MAX_USER, +}; + +#define LOCK_MODE_NAMES { \ + [MODE_READ_USER] = "READ", \ + [MODE_WRITE_USER] = "WRITE" \ +} + +enum lockahead_results { + LLA_RESULT_SENT = 0, + LLA_RESULT_DIFFERENT, + LLA_RESULT_SAME, +}; + /** @} lustreuser */ #endif /* _LUSTRE_USER_H */ From patchwork Mon Sep 30 18:54:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167135 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A3ABF16C1 for ; Mon, 30 Sep 2019 18:59:55 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8C361224F0 for ; Mon, 30 Sep 2019 18:59:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8C361224F0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 1E4CE5C3FDE; Mon, 30 Sep 2019 11:58:31 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 3814A5C3AE2 for ; Mon, 30 Sep 2019 11:57:09 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 91C5E100543A; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 8F1C7B4; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:54 -0400 Message-Id: <1569869810-23848-36-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 035/151] lustre: jobstats: move jobstats code into separate file. X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Ben Evans We'll be enhancing jobstats in future patches. To make that easier, move the jobstats code into a separate file. Note: the OpenSFS version of this patch added a hash table to cache content read from the environment, but we don't use environment values in this branch. WC-bug-id: https://jira.whamcloud.com/browse/LU-9221 Lustre-commit: 08479b74ec35 ("LU-9221 jobstats: Create a pid-based hash for jobid values") Signed-off-by: Ben Evans Reviewed-on: https://review.whamcloud.com/25208 Reviewed-by: Andreas Dilger Reviewed-by: Andrew Perepechko Reviewed-by: Oleg Drokin Signed-off-by: NeilBrown Signed-off-by: James Simmons --- fs/lustre/obdclass/Makefile | 3 +- fs/lustre/obdclass/class_obd.c | 43 --------------------- fs/lustre/obdclass/jobid.c | 86 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 88 insertions(+), 44 deletions(-) create mode 100644 fs/lustre/obdclass/jobid.c diff --git a/fs/lustre/obdclass/Makefile b/fs/lustre/obdclass/Makefile index a8f30ee..96fce1b 100644 --- a/fs/lustre/obdclass/Makefile +++ b/fs/lustre/obdclass/Makefile @@ -7,4 +7,5 @@ obdclass-y := llog.o llog_cat.o llog_obd.o llog_swab.o class_obd.o \ genops.o obd_sysfs.o lprocfs_status.o lprocfs_counters.o \ lustre_handles.o lustre_peer.o statfs_pack.o linkea.o \ obdo.o obd_config.o obd_mount.o lu_object.o lu_ref.o \ - cl_object.o cl_page.o cl_lock.o cl_io.o kernelcomm.o + cl_object.o cl_page.o cl_lock.o cl_io.o kernelcomm.o \ + jobid.o diff --git a/fs/lustre/obdclass/class_obd.c b/fs/lustre/obdclass/class_obd.c index d8fc0db..8b6032a 100644 --- a/fs/lustre/obdclass/class_obd.c +++ b/fs/lustre/obdclass/class_obd.c @@ -76,49 +76,6 @@ int at_extra = 30; EXPORT_SYMBOL(at_extra); -char obd_jobid_var[JOBSTATS_JOBID_VAR_MAX_LEN + 1] = JOBSTATS_DISABLE; -char obd_jobid_node[LUSTRE_JOBID_SIZE + 1]; - -/* Get jobid of current process from stored variable or calculate - * it from pid and user_id. - * - * Historically this was also done by reading the environment variable - * stored in between the "env_start" & "env_end" of task struct. - * This is now deprecated. - */ -int lustre_get_jobid(char *jobid) -{ - char tmp_jobid[LUSTRE_JOBID_SIZE] = { 0 }; - - /* Jobstats isn't enabled */ - if (strcmp(obd_jobid_var, JOBSTATS_DISABLE) == 0) - goto out_cache_jobid; - - /* Use process name + fsuid as jobid */ - if (strcmp(obd_jobid_var, JOBSTATS_PROCNAME_UID) == 0) { - snprintf(tmp_jobid, LUSTRE_JOBID_SIZE, "%s.%u", - current->comm, - from_kuid(&init_user_ns, current_fsuid())); - goto out_cache_jobid; - } - - /* Whole node dedicated to single job */ - if (strcmp(obd_jobid_var, JOBSTATS_NODELOCAL) == 0) { - strcpy(tmp_jobid, obd_jobid_node); - goto out_cache_jobid; - } - - return -ENOENT; - -out_cache_jobid: - /* Only replace the job ID if it changed. */ - if (strcmp(jobid, tmp_jobid) != 0) - strcpy(jobid, tmp_jobid); - - return 0; -} -EXPORT_SYMBOL(lustre_get_jobid); - static int class_resolve_dev_name(u32 len, const char *name) { int rc; diff --git a/fs/lustre/obdclass/jobid.c b/fs/lustre/obdclass/jobid.c new file mode 100644 index 0000000..3655a2e --- /dev/null +++ b/fs/lustre/obdclass/jobid.c @@ -0,0 +1,86 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * GPL HEADER START + * + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 only, + * as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * General Public License version 2 for more details (a copy is included + * in the LICENSE file that accompanied this code). + * + * You should have received a copy of the GNU General Public License + * version 2 along with this program; If not, see + * http://www.gnu.org/licenses/gpl-2.0.html + * + * GPL HEADER END + */ +/* + * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. + * Use is subject to license terms. + * + * Copyright (c) 2011, 2014, Intel Corporation. + * + * Copyright 2017 Cray Inc, all rights reserved. + * Author: Ben Evans. + * + */ + +#define DEBUG_SUBSYSTEM S_RPC +#include +#ifdef HAVE_UIDGID_HEADER +#include +#endif + +#include +#include +#include + +char obd_jobid_var[JOBSTATS_JOBID_VAR_MAX_LEN + 1] = JOBSTATS_DISABLE; +char obd_jobid_node[LUSTRE_JOBID_SIZE + 1]; + +/* Get jobid of current process from stored variable or calculate + * it from pid and user_id. + * + * Historically this was also done by reading the environment variable + * stored in between the "env_start" & "env_end" of task struct. + * This is now deprecated. + */ + +int lustre_get_jobid(char *jobid) +{ + char tmp_jobid[LUSTRE_JOBID_SIZE] = { 0 }; + + /* Jobstats isn't enabled */ + if (strcmp(obd_jobid_var, JOBSTATS_DISABLE) == 0) + goto out_cache_jobid; + + /* Use process name + fsuid as jobid */ + if (strcmp(obd_jobid_var, JOBSTATS_PROCNAME_UID) == 0) { + snprintf(tmp_jobid, LUSTRE_JOBID_SIZE, "%s.%u", + current->comm, + from_kuid(&init_user_ns, current_fsuid())); + goto out_cache_jobid; + } + + /* Whole node dedicated to single job */ + if (strcmp(obd_jobid_var, JOBSTATS_NODELOCAL) == 0) { + strcpy(tmp_jobid, obd_jobid_node); + goto out_cache_jobid; + } + + return -ENOENT; + +out_cache_jobid: + /* Only replace the job ID if it changed. */ + if (strcmp(jobid, tmp_jobid) != 0) + strcpy(jobid, tmp_jobid); + + return 0; +} +EXPORT_SYMBOL(lustre_get_jobid); From patchwork Mon Sep 30 18:54:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167077 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CB91C13BD for ; Mon, 30 Sep 2019 18:58:04 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B3870224D5 for ; Mon, 30 Sep 2019 18:58:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B3870224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id C37615C38CE; Mon, 30 Sep 2019 11:57:35 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8EBA75C3917 for ; Mon, 30 Sep 2019 11:57:09 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 936EE100543B; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 91F17A9; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:55 -0400 Message-Id: <1569869810-23848-37-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 036/151] lustre: ldlm: don't use jiffies as sysfs parameter X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: James Simmons , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" The ldlm sysfs file handles lru_max_age in jiffies which is wrong since jiffies are not consistent across machine since HZ is configurable at compile time. Talking to most users they thought lru_max_age was in seconds which is incorrect. The best way to fix this is to move lru_max_age to millisecs since most systems lustre deals with sets HZ to 1000. To make it clear it is in milliseconds print out lru_max_age with "ms". Since users tend to think in seconds allow passing in seconds besides milliseconds and internally converting them to nanaseconds. Since we have to support milliseconds move to ktime_t since we can't use time64_t. Unfortunately, this makes a relatively large patch, but I could not find a way to split it up some more without breaking atomicity of the change. WC-bug-id: https://jira.whamcloud.com/browse/LU-8541 Lustre-commit: 800ffd471186 ("LU-8541 ldlm: don't use jiffies as sysfs parameter") Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/28370 Reviewed-by: Andreas Dilger Reviewed-by: Dmitry Eremin Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_dlm.h | 10 +++++----- fs/lustre/ldlm/ldlm_internal.h | 5 +++-- fs/lustre/ldlm/ldlm_lock.c | 7 ++++--- fs/lustre/ldlm/ldlm_lockd.c | 5 +++-- fs/lustre/ldlm/ldlm_request.c | 16 ++++++++-------- fs/lustre/ldlm/ldlm_resource.c | 23 ++++++++++++++++++----- 6 files changed, 41 insertions(+), 25 deletions(-) diff --git a/fs/lustre/include/lustre_dlm.h b/fs/lustre/include/lustre_dlm.h index 1c286eb..b13e0cc 100644 --- a/fs/lustre/include/lustre_dlm.h +++ b/fs/lustre/include/lustre_dlm.h @@ -59,7 +59,7 @@ #define OBD_LDLM_DEVICENAME "ldlm" #define LDLM_DEFAULT_LRU_SIZE (100 * num_online_cpus()) -#define LDLM_DEFAULT_MAX_ALIVE (65 * 60 * HZ) /* 65 min */ +#define LDLM_DEFAULT_MAX_ALIVE (64 * 60) /* 65 min */ #define LDLM_DEFAULT_PARALLEL_AST_LIMIT 1024 /** @@ -410,8 +410,9 @@ struct ldlm_namespace { * controlled by available memory on this client and on server. */ unsigned int ns_max_unused; + /** Maximum allowed age (last used time) for locks in the LRU */ - unsigned int ns_max_age; + ktime_t ns_max_age; /** * Used to rate-limit ldlm_namespace_dump calls. @@ -702,10 +703,9 @@ struct ldlm_lock { time64_t l_last_activity; /** - * Time last used by e.g. being matched by lock match. - * Jiffies. Should be converted to time if needed. + * Time, in nanoseconds, last used by e.g. being matched by lock match. */ - unsigned long l_last_used; + ktime_t l_last_used; /** Originally requested extent for the extent lock. */ struct ldlm_extent l_req_extent; diff --git a/fs/lustre/ldlm/ldlm_internal.h b/fs/lustre/ldlm/ldlm_internal.h index a6dee9f..c3788c2 100644 --- a/fs/lustre/ldlm/ldlm_internal.h +++ b/fs/lustre/ldlm/ldlm_internal.h @@ -147,8 +147,9 @@ void ldlm_lock_decref_internal_nolock(struct ldlm_lock *lock, struct ldlm_lock *ldlm_lock_get(struct ldlm_lock *lock); int ldlm_run_ast_work(struct ldlm_namespace *ns, struct list_head *rpc_list, enum ldlm_desc_ast_t ast_type); -int ldlm_lock_remove_from_lru_check(struct ldlm_lock *lock, time_t last_use); -#define ldlm_lock_remove_from_lru(lock) ldlm_lock_remove_from_lru_check(lock, 0) +int ldlm_lock_remove_from_lru_check(struct ldlm_lock *lock, ktime_t last_use); +#define ldlm_lock_remove_from_lru(lock) \ + ldlm_lock_remove_from_lru_check(lock, ktime_set(0, 0)) int ldlm_lock_remove_from_lru_nolock(struct ldlm_lock *lock); void ldlm_lock_destroy_nolock(struct ldlm_lock *lock); diff --git a/fs/lustre/ldlm/ldlm_lock.c b/fs/lustre/ldlm/ldlm_lock.c index ddebcf6..235a137 100644 --- a/fs/lustre/ldlm/ldlm_lock.c +++ b/fs/lustre/ldlm/ldlm_lock.c @@ -224,13 +224,14 @@ int ldlm_lock_remove_from_lru_nolock(struct ldlm_lock *lock) * * 1 the lock was in LRU list and removed. */ -int ldlm_lock_remove_from_lru_check(struct ldlm_lock *lock, time_t last_use) +int ldlm_lock_remove_from_lru_check(struct ldlm_lock *lock, ktime_t last_use) { struct ldlm_namespace *ns = ldlm_lock_to_ns(lock); int rc = 0; spin_lock(&ns->ns_lock); - if (last_use == 0 || last_use == lock->l_last_used) + if (!ktime_compare(last_use, ktime_set(0, 0)) || + !ktime_compare(last_use, lock->l_last_used)) rc = ldlm_lock_remove_from_lru_nolock(lock); spin_unlock(&ns->ns_lock); @@ -244,7 +245,7 @@ static void ldlm_lock_add_to_lru_nolock(struct ldlm_lock *lock) { struct ldlm_namespace *ns = ldlm_lock_to_ns(lock); - lock->l_last_used = jiffies; + lock->l_last_used = ktime_get(); LASSERT(list_empty(&lock->l_lru)); LASSERT(lock->l_resource->lr_type != LDLM_FLOCK); list_add_tail(&lock->l_lru, &ns->ns_unused_list); diff --git a/fs/lustre/ldlm/ldlm_lockd.c b/fs/lustre/ldlm/ldlm_lockd.c index 56f042c..83f5a22 100644 --- a/fs/lustre/ldlm/ldlm_lockd.c +++ b/fs/lustre/ldlm/ldlm_lockd.c @@ -303,8 +303,9 @@ static void ldlm_handle_gl_callback(struct ptlrpc_request *req, lock_res_and_lock(lock); if (lock->l_granted_mode == LCK_PW && !lock->l_readers && !lock->l_writers && - time_after(jiffies, - lock->l_last_used + 10 * HZ)) { + ktime_after(ktime_get(), + ktime_add(lock->l_last_used, + ktime_set(10, 0)))) { unlock_res_and_lock(lock); if (ldlm_bl_to_thread_lock(ns, NULL, lock)) ldlm_handle_bl_callback(ns, NULL, lock); diff --git a/fs/lustre/ldlm/ldlm_request.c b/fs/lustre/ldlm/ldlm_request.c index b7dcfda..4185d42 100644 --- a/fs/lustre/ldlm/ldlm_request.c +++ b/fs/lustre/ldlm/ldlm_request.c @@ -1166,10 +1166,10 @@ static enum ldlm_policy_res ldlm_cancel_lrur_policy(struct ldlm_namespace *ns, int unused, int added, int count) { - unsigned long cur = jiffies; + ktime_t cur = ktime_get(); struct ldlm_pool *pl = &ns->ns_pool; u64 slv, lvf, lv; - unsigned long la; + s64 la; /* Stop LRU processing when we reach past @count or have checked all * locks in LRU. @@ -1181,12 +1181,13 @@ static enum ldlm_policy_res ldlm_cancel_lrur_policy(struct ldlm_namespace *ns, * Despite of the LV, It doesn't make sense to keep the lock which * is unused for ns_max_age time. */ - if (time_after(jiffies, lock->l_last_used + ns->ns_max_age)) + if (ktime_after(ktime_get(), + ktime_add(lock->l_last_used, ns->ns_max_age))) return LDLM_POLICY_CANCEL_LOCK; slv = ldlm_pool_get_slv(pl); lvf = ldlm_pool_get_lvf(pl); - la = (cur - lock->l_last_used) / HZ; + la = ktime_to_ns(ktime_sub(cur, lock->l_last_used)) / NSEC_PER_SEC; lv = lvf * la * unused; /* Inform pool about current CLV to see it via debugfs. */ @@ -1235,7 +1236,8 @@ static enum ldlm_policy_res ldlm_cancel_aged_policy(struct ldlm_namespace *ns, int count) { if ((added >= count) && - time_before(jiffies, lock->l_last_used + ns->ns_max_age)) + ktime_before(ktime_get(), + ktime_add(lock->l_last_used, ns->ns_max_age))) return LDLM_POLICY_KEEP_LOCK; return LDLM_POLICY_CANCEL_LOCK; @@ -1382,7 +1384,7 @@ static int ldlm_prepare_lru_list(struct ldlm_namespace *ns, while (!list_empty(&ns->ns_unused_list)) { enum ldlm_policy_res result; - time_t last_use = 0; + ktime_t last_use = ktime_set(0, 0); /* all unused locks */ if (remained-- <= 0) @@ -1402,8 +1404,6 @@ static int ldlm_prepare_lru_list(struct ldlm_namespace *ns, continue; last_use = lock->l_last_used; - if (last_use == jiffies) - continue; /* Somebody is already doing CANCEL. No need for this * lock in LRU, do not traverse it again. diff --git a/fs/lustre/ldlm/ldlm_resource.c b/fs/lustre/ldlm/ldlm_resource.c index 372a290..bf8abd4 100644 --- a/fs/lustre/ldlm/ldlm_resource.c +++ b/fs/lustre/ldlm/ldlm_resource.c @@ -263,7 +263,7 @@ static ssize_t lru_max_age_show(struct kobject *kobj, struct attribute *attr, struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace, ns_kobj); - return sprintf(buf, "%u\n", ns->ns_max_age); + return sprintf(buf, "%lld\n", ktime_to_ms(ns->ns_max_age)); } static ssize_t lru_max_age_store(struct kobject *kobj, struct attribute *attr, @@ -271,14 +271,27 @@ static ssize_t lru_max_age_store(struct kobject *kobj, struct attribute *attr, { struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace, ns_kobj); - unsigned long tmp; + int scale = NSEC_PER_MSEC; + unsigned long long tmp; + char *buf; int err; - err = kstrtoul(buffer, 10, &tmp); + /* Did the user ask in seconds or milliseconds. Default is in ms */ + buf = strstr(buffer, "ms"); + if (!buf) { + buf = strchr(buffer, 's'); + if (buf) + scale = NSEC_PER_SEC; + } + + if (buf) + *buf = '\0'; + + err = kstrtoull(buffer, 10, &tmp); if (err != 0) return -EINVAL; - ns->ns_max_age = tmp; + ns->ns_max_age = ktime_set(0, tmp * scale); return count; } @@ -639,7 +652,7 @@ struct ldlm_namespace *ldlm_namespace_new(struct obd_device *obd, char *name, ns->ns_max_parallel_ast = LDLM_DEFAULT_PARALLEL_AST_LIMIT; ns->ns_nr_unused = 0; ns->ns_max_unused = LDLM_DEFAULT_LRU_SIZE; - ns->ns_max_age = LDLM_DEFAULT_MAX_ALIVE; + ns->ns_max_age = ktime_set(LDLM_DEFAULT_MAX_ALIVE, 0); ns->ns_orig_connect_flags = 0; ns->ns_connect_flags = 0; ns->ns_stopping = 0; From patchwork Mon Sep 30 18:54:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167067 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2803F16B1 for ; Mon, 30 Sep 2019 18:57:43 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0C5DB224D5 for ; Mon, 30 Sep 2019 18:57:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0C5DB224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id BA9EA5C3AA6; Mon, 30 Sep 2019 11:57:23 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id E61AB5C3B0A for ; Mon, 30 Sep 2019 11:57:09 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 96209100543C; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 94D05B5; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:56 -0400 Message-Id: <1569869810-23848-38-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 037/151] lnet: Handle ping buffer with only loopback NID X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Olaf Weber During startup lnet_peer_data_resent() can see a ping buffer for the local node which contains only the loopback NID. This shows up as pi_nnis == 1, and there is nothing to be done (or that needs to be done) in that case. WC-bug-id: https://jira.whamcloud.com/browse/LU-9933 Lustre-commit: 002e25b92779 ("LU-9933 lnet: Handle ping buffer with only loopback NID") Signed-off-by: Olaf Weber Reviewed-on: https://review.whamcloud.com/28811 Reviewed-by: James Simmons Reviewed-by: Amir Shehata Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/lnet/peer.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/net/lnet/lnet/peer.c b/net/lnet/lnet/peer.c index d7a2e3b..9fdb6a9 100644 --- a/net/lnet/lnet/peer.c +++ b/net/lnet/lnet/peer.c @@ -2589,10 +2589,12 @@ static int lnet_peer_data_present(struct lnet_peer *lp) * * The peer for the loopback interface is a special case: this * is the peer for the local node, and we want to set its - * primary NID to the correct value here. + * primary NID to the correct value here. Moreover, this peer + * can show up with only the loopback NID in the ping buffer. */ - if (pbuf->pb_info.pi_nnis > 1) - nid = pbuf->pb_info.pi_ni[1].ns_nid; + if (pbuf->pb_info.pi_nnis <= 1) + goto out; + nid = pbuf->pb_info.pi_ni[1].ns_nid; if (LNET_NETTYP(LNET_NIDNET(lp->lp_primary_nid)) == LOLND) { rc = lnet_peer_set_primary_nid(lp, nid, flags); if (!rc) From patchwork Mon Sep 30 18:54:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167139 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A524E912 for ; Mon, 30 Sep 2019 19:00:07 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8C30D224D5 for ; Mon, 30 Sep 2019 19:00:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8C30D224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id C56435C4016; Mon, 30 Sep 2019 11:58:35 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 35A985C3B10 for ; Mon, 30 Sep 2019 11:57:10 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 99BBD100543D; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 97A4BBD; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:57 -0400 Message-Id: <1569869810-23848-39-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 038/151] lustre: llite: enable readahead for small read_ahead_per_file X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Erich Focht , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Erich Focht Fixes for a regression introduced by http://review.whamcloud.com/19368 for the case that max_read_ahead_per_file_mb is smaller than max_pages_per_rpc. With 16MB RPCs this happens pretty easily. In that case the readahead window stayed zero and the backend saw only requests of the size of the user IOs. This patch restores the previous behavior for this corner case while keeping the fix for large RPCs introduced by the alignment. When max_read_ahead_per_file_mb is smaller than max_pages_per_rpc the RPC size will not be optimal, but will be at least 1MB and the readahead window will be as large as expected instead of zero. WC-bug-id: https://jira.whamcloud.com/browse/LU-9214 Lustre-commit: 32e64eeefa7d ("LU-9214 llite: enable readahead for small read_ahead_per_file") Signed-off-by: Erich Focht Reviewed-on: https://review.whamcloud.com/25996 Reviewed-by: Andreas Dilger Reviewed-by: Jinshan Xiong Reviewed-by: Patrick Farrell Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/rw.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/fs/lustre/llite/rw.c b/fs/lustre/llite/rw.c index 32f028db..ca0b357 100644 --- a/fs/lustre/llite/rw.c +++ b/fs/lustre/llite/rw.c @@ -719,7 +719,10 @@ static void ras_increase_window(struct inode *inode, wlen = min(ras->ras_window_len + ras->ras_rpc_size, ra->ra_max_pages_per_file); - ras->ras_window_len = ras_align(ras, wlen, NULL); + if (wlen < ras->ras_rpc_size) + ras->ras_window_len = wlen; + else + ras->ras_window_len = ras_align(ras, wlen, NULL); } } From patchwork Mon Sep 30 18:54:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167143 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8415416B1 for ; Mon, 30 Sep 2019 19:00:24 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6BE58224D5 for ; Mon, 30 Sep 2019 19:00:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6BE58224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 7BE335C4045; Mon, 30 Sep 2019 11:58:39 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 7636C5C3B15 for ; Mon, 30 Sep 2019 11:57:10 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 9BEAD100543E; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 9AA58BB; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:58 -0400 Message-Id: <1569869810-23848-40-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 039/151] lnet: don't discover loopback interface X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Amir Shehata , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Amir Shehata Whenever we send messages destined to the loopback interface it should always go over the loopback interface. To achieve that there is no real need to initiate discovery on the loopback. This will result in a non-mr peer created for the loopback, Which makes sense because if we are to send messages to ourselves we do not want to use the different interfaces rather just keep sending over the lolnd. In effect this is a special case where we want to behave as a non-mr node. When sending a message destined for the loopback interface there is no need to go through the selection process, it is sufficient to shortcut all the MR logic and send directly over the lolnd. WC-bug-id: https://jira.whamcloud.com/browse/LU-9992 Lustre-commit: de823879c43d ("LU-9992 lnet: don't discover loopback interface") Signed-off-by: Amir Shehata Reviewed-on: https://review.whamcloud.com/29007 Reviewed-by: Sonia Sharma Reviewed-by: Olaf Weber Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/lnet/lib-move.c | 11 +++++++++++ net/lnet/lnet/peer.c | 9 +++++++++ 2 files changed, 20 insertions(+) diff --git a/net/lnet/lnet/lib-move.c b/net/lnet/lnet/lib-move.c index 875d289..6f3e941 100644 --- a/net/lnet/lnet/lib-move.c +++ b/net/lnet/lnet/lib-move.c @@ -1391,6 +1391,17 @@ void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats, lnet_net_unlock(cpt); return PTR_ERR(lpni); } + + /* If we're being asked to send to the loopback interface, there + * is no need to go through any selection. We can just shortcut + * the entire process and send over lolnd + */ + if (LNET_NETTYP(LNET_NIDNET(dst_nid)) == LOLND) { + lnet_peer_ni_decref_locked(lpni); + best_ni = the_lnet.ln_loni; + goto send; + } + /* * Now that we have a peer_ni, check if we want to discover * the peer. Traffic to the LNET_RESERVED_PORTAL should not diff --git a/net/lnet/lnet/peer.c b/net/lnet/lnet/peer.c index 9fdb6a9..1266e32 100644 --- a/net/lnet/lnet/peer.c +++ b/net/lnet/lnet/peer.c @@ -211,6 +211,15 @@ init_waitqueue_head(&lp->lp_dc_waitq); spin_lock_init(&lp->lp_lock); lp->lp_primary_nid = nid; + + /* Turn off discovery for loopback peer. If you're creating a peer + * for the loopback interface then that was initiated when we + * attempted to send a message over the loopback. There is no need + * to ever use a different interface when sending messages to + * myself. + */ + if (LNET_NETTYP(LNET_NIDNET(nid)) == LOLND) + lp->lp_state = LNET_PEER_NO_DISCOVERY; lp->lp_cpt = lnet_nid_cpt_hash(nid, LNET_CPT_NUMBER); CDEBUG(D_NET, "%p nid %s\n", lp, libcfs_nid2str(lp->lp_primary_nid)); From patchwork Mon Sep 30 18:54:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167147 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E695F16B1 for ; Mon, 30 Sep 2019 19:00:33 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CEF4F224D5 for ; Mon, 30 Sep 2019 19:00:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CEF4F224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id D3E7A5C406F; Mon, 30 Sep 2019 11:58:42 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id B84FC5C395B for ; Mon, 30 Sep 2019 11:57:10 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 9F6B5100543F; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 9E749A9; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:54:59 -0400 Message-Id: <1569869810-23848-41-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 040/151] lnet: reduce logging severity X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Amir Shehata , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Amir Shehata On shutdown a push event can be triggered for a non-existent peer. Reducing the severity of the log. WC-bug-id: https://jira.whamcloud.com/browse/LU-9904 Lustre-commit: 7fc8037d61b2 ("LU-9904 lnet: reduce logging severity") Signed-off-by: Amir Shehata Reviewed-on: https://review.whamcloud.com/29026 Reviewed-by: Sonia Sharma Reviewed-by: Olaf Weber Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/lnet/peer.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/net/lnet/lnet/peer.c b/net/lnet/lnet/peer.c index 1266e32..68c6eb7 100644 --- a/net/lnet/lnet/peer.c +++ b/net/lnet/lnet/peer.c @@ -1808,7 +1808,8 @@ void lnet_peer_push_event(struct lnet_event *ev) /* lnet_find_peer() adds a refcount */ lp = lnet_find_peer(ev->source.nid); if (!lp) { - CERROR("Push Put from unknown %s (source %s)\n", + CDEBUG(D_NET, + "Push Put from unknown %s (source %s). Ignoring...\n", libcfs_nid2str(ev->initiator.nid), libcfs_nid2str(ev->source.nid)); return; From patchwork Mon Sep 30 18:55:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167151 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AB691912 for ; Mon, 30 Sep 2019 19:00:42 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 93530224D5 for ; Mon, 30 Sep 2019 19:00:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 93530224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 399955C40AB; Mon, 30 Sep 2019 11:58:46 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 0759C5C395B for ; Mon, 30 Sep 2019 11:57:11 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id A3C8F10055E0; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id A163BB4; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:00 -0400 Message-Id: <1569869810-23848-42-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 041/151] lustre: ptlrpc: migrate pinger to 64 bit time X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: James Simmons , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" Change imp_next_ping, cl_*grant_* timer fields, and ti_timeout to time64_t. With these changes the pinger will be 64 bit time compliant. This prevents any possible confusion with jiffies. WC-bug-id: https://jira.whamcloud.com/browse/LU-9019 Lustre-commit: 44bdc137c901 ("LU-9019 ptlrpc: migrate pinger to 64 bit time") Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/28035 Reviewed-by: Dmitry Eremin Reviewed-by: Fan Yong Reviewed-by: Andreas Dilger Signed-off-by: James Simmons --- fs/lustre/include/lustre_import.h | 2 +- fs/lustre/include/lustre_net.h | 2 +- fs/lustre/include/obd.h | 6 +++--- fs/lustre/osc/lproc_osc.c | 2 +- fs/lustre/osc/osc_request.c | 11 +++++----- fs/lustre/ptlrpc/client.c | 7 +++--- fs/lustre/ptlrpc/import.c | 36 +++++++++++++++++++------------ fs/lustre/ptlrpc/pinger.c | 44 ++++++++++++++++++-------------------- fs/lustre/ptlrpc/ptlrpc_internal.h | 2 +- fs/lustre/ptlrpc/ptlrpcd.c | 2 +- 10 files changed, 60 insertions(+), 54 deletions(-) diff --git a/fs/lustre/include/lustre_import.h b/fs/lustre/include/lustre_import.h index fc1f87c..6e22274 100644 --- a/fs/lustre/include/lustre_import.h +++ b/fs/lustre/include/lustre_import.h @@ -250,7 +250,7 @@ struct obd_import { */ struct lustre_handle imp_remote_handle; /** When to perform next ping. time in jiffies. */ - unsigned long imp_next_ping; + time64_t imp_next_ping; /** When we last successfully connected. time in 64bit jiffies */ u64 imp_last_success_conn; diff --git a/fs/lustre/include/lustre_net.h b/fs/lustre/include/lustre_net.h index 20ce196..a0a974c 100644 --- a/fs/lustre/include/lustre_net.h +++ b/fs/lustre/include/lustre_net.h @@ -2287,7 +2287,7 @@ enum timeout_event { typedef int (*timeout_cb_t)(struct timeout_item *, void *); int ptlrpc_pinger_add_import(struct obd_import *imp); int ptlrpc_pinger_del_import(struct obd_import *imp); -int ptlrpc_add_timeout_client(int time, enum timeout_event event, +int ptlrpc_add_timeout_client(time64_t time, enum timeout_event event, timeout_cb_t cb, void *data, struct list_head *obd_list); int ptlrpc_del_timeout_client(struct list_head *obd_list, diff --git a/fs/lustre/include/obd.h b/fs/lustre/include/obd.h index 3bdde31..9514260 100644 --- a/fs/lustre/include/obd.h +++ b/fs/lustre/include/obd.h @@ -117,7 +117,7 @@ struct brw_page { struct timeout_item { enum timeout_event ti_event; - unsigned long ti_timeout; + time64_t ti_timeout; timeout_cb_t ti_cb; void *ti_cb_data; struct list_head ti_obd_list; @@ -203,9 +203,9 @@ struct client_obd { */ long cl_reserved_grant; wait_queue_head_t cl_cache_waiters; /* waiting for cache/grant */ - unsigned long cl_next_shrink_grant; /* jiffies */ + time64_t cl_next_shrink_grant; /* seconds */ struct list_head cl_grant_shrink_list; /* Timeout event list */ - int cl_grant_shrink_interval; /* seconds */ + time64_t cl_grant_shrink_interval; /* seconds */ /* A chunk is an optimal size used by osc_extent to determine * the extent size. A chunk is max(PAGE_SIZE, OST block size) diff --git a/fs/lustre/osc/lproc_osc.c b/fs/lustre/osc/lproc_osc.c index 8e85863..2a57982 100644 --- a/fs/lustre/osc/lproc_osc.c +++ b/fs/lustre/osc/lproc_osc.c @@ -326,7 +326,7 @@ static ssize_t grant_shrink_interval_show(struct kobject *kobj, struct obd_device *obd = container_of(kobj, struct obd_device, obd_kset.kobj); - return sprintf(buf, "%d\n", obd->u.cli.cl_grant_shrink_interval); + return sprintf(buf, "%lld\n", obd->u.cli.cl_grant_shrink_interval); } static ssize_t grant_shrink_interval_store(struct kobject *kobj, diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index e703cf1..b84856d 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -716,9 +716,9 @@ static void osc_announce_cached(struct client_obd *cli, struct obdo *oa, void osc_update_next_shrink(struct client_obd *cli) { - cli->cl_next_shrink_grant = - jiffies + cli->cl_grant_shrink_interval * HZ; - CDEBUG(D_CACHE, "next time %ld to shrink grant\n", + cli->cl_next_shrink_grant = ktime_get_seconds() + + cli->cl_grant_shrink_interval; + CDEBUG(D_CACHE, "next time %lld to shrink grant\n", cli->cl_next_shrink_grant); } @@ -841,14 +841,13 @@ int osc_shrink_grant_to_target(struct client_obd *cli, u64 target_bytes) static int osc_should_shrink_grant(struct client_obd *client) { - unsigned long time = jiffies; - unsigned long next_shrink = client->cl_next_shrink_grant; + time64_t next_shrink = client->cl_next_shrink_grant; if ((client->cl_import->imp_connect_data.ocd_connect_flags & OBD_CONNECT_GRANT_SHRINK) == 0) return 0; - if (time_after_eq(time, next_shrink - 5)) { + if (ktime_get_seconds() >= next_shrink - 5) { /* Get the current RPC size directly, instead of going via: * cli_brw_size(obd->u.cli.cl_import->imp_obd->obd_self_export) * Keep comment here so that it can be found by searching. diff --git a/fs/lustre/ptlrpc/client.c b/fs/lustre/ptlrpc/client.c index 4888578..fc909a8 100644 --- a/fs/lustre/ptlrpc/client.c +++ b/fs/lustre/ptlrpc/client.c @@ -2206,7 +2206,7 @@ static void ptlrpc_interrupted_set(struct ptlrpc_request_set *set) /** * Get the smallest timeout in the set; this does NOT set a timeout. */ -int ptlrpc_set_next_timeout(struct ptlrpc_request_set *set) +time64_t ptlrpc_set_next_timeout(struct ptlrpc_request_set *set) { time64_t now = ktime_get_real_seconds(); int timeout = 0; @@ -2252,7 +2252,8 @@ int ptlrpc_set_next_timeout(struct ptlrpc_request_set *set) int ptlrpc_set_wait(struct ptlrpc_request_set *set) { struct ptlrpc_request *req; - int rc, timeout; + time64_t timeout; + int rc; if (set->set_producer) (void)ptlrpc_set_producer(set); @@ -2272,7 +2273,7 @@ int ptlrpc_set_wait(struct ptlrpc_request_set *set) * wait until all complete, interrupted, or an in-flight * req times out */ - CDEBUG(D_RPCTRACE, "set %p going to sleep for %d seconds\n", + CDEBUG(D_RPCTRACE, "set %p going to sleep for %lld seconds\n", set, timeout); if (timeout == 0 && !signal_pending(current)) { diff --git a/fs/lustre/ptlrpc/import.c b/fs/lustre/ptlrpc/import.c index 67a66a5..3cb9b84 100644 --- a/fs/lustre/ptlrpc/import.c +++ b/fs/lustre/ptlrpc/import.c @@ -218,10 +218,10 @@ void ptlrpc_deactivate_import(struct obd_import *imp) } EXPORT_SYMBOL(ptlrpc_deactivate_import); -static unsigned int +static time64_t ptlrpc_inflight_deadline(struct ptlrpc_request *req, time64_t now) { - long dl; + time64_t dl; if (!(((req->rq_phase == RQ_PHASE_RPC) && !req->rq_waiting) || (req->rq_phase == RQ_PHASE_BULK) || @@ -246,7 +246,7 @@ static unsigned int ptlrpc_inflight_timeout(struct obd_import *imp) { time64_t now = ktime_get_real_seconds(); struct ptlrpc_request *req, *n; - unsigned int timeout = 0; + time64_t timeout = 0; spin_lock(&imp->imp_lock); list_for_each_entry_safe(req, n, &imp->imp_sending_list, rq_list) @@ -265,7 +265,7 @@ static unsigned int ptlrpc_inflight_timeout(struct obd_import *imp) void ptlrpc_invalidate_import(struct obd_import *imp) { struct ptlrpc_request *req, *n; - unsigned int timeout; + time64_t timeout; int rc; atomic_inc(&imp->imp_inval_count); @@ -282,6 +282,7 @@ void ptlrpc_invalidate_import(struct obd_import *imp) * no guarantee that some rdma transfer is not in progress right now. */ do { + long timeout_jiffies; /* Calculate max timeout for waiting on rpcs to error * out. Use obd_timeout if calculated value is smaller * than it. @@ -298,16 +299,17 @@ void ptlrpc_invalidate_import(struct obd_import *imp) } CDEBUG(D_RPCTRACE, - "Sleeping %d sec for inflight to error out\n", + "Sleeping %llds for inflight to error out\n", timeout); /* Wait for all requests to error out and call completion * callbacks. Cap it at obd_timeout -- these should all * have been locally cancelled by ptlrpc_abort_inflight. */ + timeout_jiffies = max_t(long, timeout * HZ, 1); rc = wait_event_idle_timeout(imp->imp_recovery_waitq, atomic_read(&imp->imp_inflight) == 0, - obd_timeout * HZ); + timeout_jiffies); if (rc == 0) { const char *cli_tgt = obd2cli_tgt(imp->imp_obd); @@ -421,6 +423,7 @@ void ptlrpc_fail_import(struct obd_import *imp, u32 conn_cnt) int ptlrpc_reconnect_import(struct obd_import *imp) { + long timeout_jiffies = obd_timeout * HZ; int rc; ptlrpc_pinger_force(imp); @@ -430,7 +433,7 @@ int ptlrpc_reconnect_import(struct obd_import *imp) rc = wait_event_idle_timeout(imp->imp_recovery_waitq, !ptlrpc_import_in_recovery(imp), - obd_timeout * HZ); + timeout_jiffies); CDEBUG(D_HA, "%s: recovery finished s:%s\n", obd2cli_tgt(imp->imp_obd), ptlrpc_import_state_name(imp->imp_state)); return rc == 0 ? -ETIMEDOUT : 0; @@ -1506,22 +1509,27 @@ int ptlrpc_disconnect_import(struct obd_import *imp, int noclose) } if (ptlrpc_import_in_recovery(imp)) { - unsigned long timeout; + long timeout_jiffies; + time64_t timeout; if (AT_OFF) { if (imp->imp_server_timeout) - timeout = obd_timeout * HZ / 2; + timeout = obd_timeout >> 1; else - timeout = obd_timeout * HZ; + timeout = obd_timeout; } else { - int idx = import_at_get_index(imp, - imp->imp_client->cli_request_portal); - timeout = at_get(&imp->imp_at.iat_service_estimate[idx]) * HZ; + u32 req_portal; + int idx; + + req_portal = imp->imp_client->cli_request_portal; + idx = import_at_get_index(imp, req_portal); + timeout = at_get(&imp->imp_at.iat_service_estimate[idx]); } + timeout_jiffies = timeout * HZ; if (wait_event_idle_timeout(imp->imp_recovery_waitq, !ptlrpc_import_in_recovery(imp), - max(timeout, 1UL)) == 0) + max_t(long, timeout_jiffies, 1)) == 0) l_wait_event_abortable( imp->imp_recovery_waitq, !ptlrpc_import_in_recovery(imp)); diff --git a/fs/lustre/ptlrpc/pinger.c b/fs/lustre/ptlrpc/pinger.c index 6a437f4..762fd0e 100644 --- a/fs/lustre/ptlrpc/pinger.c +++ b/fs/lustre/ptlrpc/pinger.c @@ -100,15 +100,15 @@ static int ptlrpc_ping(struct obd_import *imp) static void ptlrpc_update_next_ping(struct obd_import *imp, int soon) { - int time = soon ? PING_INTERVAL_SHORT : PING_INTERVAL; + time64_t time = soon ? PING_INTERVAL_SHORT : PING_INTERVAL; if (imp->imp_state == LUSTRE_IMP_DISCON) { - int dtime = max_t(int, CONNECTION_SWITCH_MIN, + time64_t dtime = max_t(time64_t, CONNECTION_SWITCH_MIN, AT_OFF ? 0 : at_get(&imp->imp_at.iat_net_latency)); time = min(time, dtime); } - imp->imp_next_ping = jiffies + time * HZ; + imp->imp_next_ping = ktime_get_seconds() + time; } static inline int imp_is_deactive(struct obd_import *imp) @@ -117,23 +117,23 @@ static inline int imp_is_deactive(struct obd_import *imp) OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_IMP_DEACTIVE)); } -static inline int ptlrpc_next_reconnect(struct obd_import *imp) +static inline time64_t ptlrpc_next_reconnect(struct obd_import *imp) { if (imp->imp_server_timeout) - return jiffies + obd_timeout / 2 * HZ; + return ktime_get_seconds() + (obd_timeout >> 1); else - return jiffies + obd_timeout * HZ; + return ktime_get_seconds() + obd_timeout; } -static long pinger_check_timeout(unsigned long time) +static time64_t pinger_check_timeout(time64_t time) { struct timeout_item *item; - unsigned long timeout = PING_INTERVAL; + time64_t timeout = PING_INTERVAL; - /* The timeout list is a increase order sorted list */ + /* This list is sorted in increasing timeout order */ mutex_lock(&pinger_mutex); list_for_each_entry(item, &timeout_list, ti_chain) { - int ti_timeout = item->ti_timeout; + time64_t ti_timeout = item->ti_timeout; if (timeout > ti_timeout) timeout = ti_timeout; @@ -141,7 +141,7 @@ static long pinger_check_timeout(unsigned long time) } mutex_unlock(&pinger_mutex); - return time + timeout * HZ - jiffies; + return time + timeout - ktime_get_seconds(); } static bool ir_up; @@ -161,7 +161,7 @@ void ptlrpc_pinger_ir_down(void) EXPORT_SYMBOL(ptlrpc_pinger_ir_down); static void ptlrpc_pinger_process_import(struct obd_import *imp, - unsigned long this_ping) + time64_t this_ping) { int level; int force; @@ -180,8 +180,7 @@ static void ptlrpc_pinger_process_import(struct obd_import *imp, imp->imp_force_verify = 0; - if (time_after_eq(imp->imp_next_ping - 5, this_ping) && - !force) { + if (imp->imp_next_ping - 5 >= this_ping && !force) { spin_unlock(&imp->imp_lock); return; } @@ -224,8 +223,8 @@ static void ptlrpc_pinger_process_import(struct obd_import *imp, static void ptlrpc_pinger_main(struct work_struct *ws) { - unsigned long this_ping = jiffies; - long time_to_next_wake; + time64_t this_ping = ktime_get_seconds(); + time64_t time_to_next_wake; struct timeout_item *item; struct obd_import *imp; @@ -238,8 +237,7 @@ static void ptlrpc_pinger_main(struct work_struct *ws) ptlrpc_pinger_process_import(imp, this_ping); /* obd_timeout might have changed */ if (imp->imp_pingable && imp->imp_next_ping && - time_after(imp->imp_next_ping, - this_ping + PING_INTERVAL * HZ)) + imp->imp_next_ping > this_ping + PING_INTERVAL) ptlrpc_update_next_ping(imp, 0); } mutex_unlock(&pinger_mutex); @@ -253,9 +251,9 @@ static void ptlrpc_pinger_main(struct work_struct *ws) * we will SKIP the next ping at next_ping, and the * ping will get sent 2 timeouts from now! Beware. */ - CDEBUG(D_INFO, "next wakeup in %ld (%ld)\n", + CDEBUG(D_INFO, "next wakeup in %lld (%lld)\n", time_to_next_wake, - this_ping + PING_INTERVAL * HZ); + this_ping + PING_INTERVAL); } while (time_to_next_wake <= 0); queue_delayed_work(pinger_wq, &ping_work, @@ -357,7 +355,7 @@ int ptlrpc_pinger_del_import(struct obd_import *imp) * Register a timeout callback to the pinger list, and the callback will * be called when timeout happens. */ -static struct timeout_item *ptlrpc_new_timeout(int time, +static struct timeout_item *ptlrpc_new_timeout(time64_t time, enum timeout_event event, timeout_cb_t cb, void *data) { @@ -382,7 +380,7 @@ static struct timeout_item *ptlrpc_new_timeout(int time, * Note: the timeout list is an sorted list with increased timeout value. */ static struct timeout_item* -ptlrpc_pinger_register_timeout(int time, enum timeout_event event, +ptlrpc_pinger_register_timeout(time64_t time, enum timeout_event event, timeout_cb_t cb, void *data) { struct timeout_item *item, *tmp; @@ -410,7 +408,7 @@ static struct timeout_item *ptlrpc_new_timeout(int time, /* Add a client_obd to the timeout event list, when timeout(@time) * happens, the callback(@cb) will be called. */ -int ptlrpc_add_timeout_client(int time, enum timeout_event event, +int ptlrpc_add_timeout_client(time64_t time, enum timeout_event event, timeout_cb_t cb, void *data, struct list_head *obd_list) { diff --git a/fs/lustre/ptlrpc/ptlrpc_internal.h b/fs/lustre/ptlrpc/ptlrpc_internal.h index 201ccdd..40cf7b7 100644 --- a/fs/lustre/ptlrpc/ptlrpc_internal.h +++ b/fs/lustre/ptlrpc/ptlrpc_internal.h @@ -69,7 +69,7 @@ struct ptlrpc_bulk_desc *ptlrpc_new_bulk(unsigned int nfrags, void ptlrpc_set_add_new_req(struct ptlrpcd_ctl *pc, struct ptlrpc_request *req); void ptlrpc_expired_set(struct ptlrpc_request_set *set); -int ptlrpc_set_next_timeout(struct ptlrpc_request_set *set); +time64_t ptlrpc_set_next_timeout(struct ptlrpc_request_set *set); void ptlrpc_resend_req(struct ptlrpc_request *request); void ptlrpc_set_bulk_mbits(struct ptlrpc_request *req); void ptlrpc_assign_next_xid_nolock(struct ptlrpc_request *req); diff --git a/fs/lustre/ptlrpc/ptlrpcd.c b/fs/lustre/ptlrpc/ptlrpcd.c index 92b477d..0b73d58 100644 --- a/fs/lustre/ptlrpc/ptlrpcd.c +++ b/fs/lustre/ptlrpc/ptlrpcd.c @@ -434,7 +434,7 @@ static int ptlrpcd(void *arg) * new_req_list and ptlrpcd_check() moves them into the set. */ do { - int timeout; + time64_t timeout; timeout = ptlrpc_set_next_timeout(set); From patchwork Mon Sep 30 18:55:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167153 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 92ACF16B1 for ; Mon, 30 Sep 2019 19:00:51 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7B2A9224D5 for ; Mon, 30 Sep 2019 19:00:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7B2A9224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 7FB765C40CC; Mon, 30 Sep 2019 11:58:49 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 5D41A5C3B2E for ; Mon, 30 Sep 2019 11:57:11 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id A6FB310055F6; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id A4DECB5; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:01 -0400 Message-Id: <1569869810-23848-43-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 042/151] lustre: mdc: add cl_device to the MDC X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mikhal Pershin , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mikhal Pershin Introduce cl_device for MDC so it is able to prepare and send IO requests to the MDT. This patch adds basic interfaces for device and object methods. The common code is exported from OSC for MDC. WC-bug-id: https://jira.whamcloud.com/browse/LU-3285 Lustre-commit: 68ff85b8858e ("LU-3285 mdc: add cl_device to the MDC") Signed-off-by: Mikhal Pershin Reviewed-on: https://review.whamcloud.com/28009 Reviewed-by: Bobi Jam Reviewed-by: Jinshan Xiong Signed-off-by: James Simmons --- fs/lustre/include/lustre_osc.h | 23 +++++ fs/lustre/mdc/Makefile | 1 + fs/lustre/mdc/mdc_dev.c | 192 +++++++++++++++++++++++++++++++++++++++++ fs/lustre/mdc/mdc_internal.h | 6 ++ fs/lustre/mdc/mdc_request.c | 6 +- fs/lustre/osc/osc_dev.c | 22 +++-- fs/lustre/osc/osc_object.c | 29 ++++--- 7 files changed, 257 insertions(+), 22 deletions(-) create mode 100644 fs/lustre/mdc/mdc_dev.c diff --git a/fs/lustre/include/lustre_osc.h b/fs/lustre/include/lustre_osc.h index 6c8a9d4..e91fae2 100644 --- a/fs/lustre/include/lustre_osc.h +++ b/fs/lustre/include/lustre_osc.h @@ -489,6 +489,29 @@ void osc_io_unplug(const struct lu_env *env, struct client_obd *cli, int osc_lock_is_lockless(const struct osc_lock *olck); +/* osc_dev.c */ +int osc_device_init(const struct lu_env *env, struct lu_device *d, + const char *name, struct lu_device *next); +struct lu_device *osc_device_fini(const struct lu_env *env, + struct lu_device *d); +struct lu_device *osc_device_free(const struct lu_env *env, + struct lu_device *d); + +/* osc_object.c */ +int osc_object_init(const struct lu_env *env, struct lu_object *obj, + const struct lu_object_conf *conf); +void osc_object_free(const struct lu_env *env, struct lu_object *obj); +int osc_lvb_print(const struct lu_env *env, void *cookie, + lu_printer_t p, const struct ost_lvb *lvb); +int osc_object_print(const struct lu_env *env, void *cookie, + lu_printer_t p, const struct lu_object *obj); +int osc_attr_get(const struct lu_env *env, struct cl_object *obj, + struct cl_attr *attr); +int osc_attr_update(const struct lu_env *env, struct cl_object *obj, + const struct cl_attr *attr, unsigned int valid); +int osc_object_glimpse(const struct lu_env *env, const struct cl_object *obj, + struct ost_lvb *lvb); + /**************************************************************************** * * Accessors and type conversions. diff --git a/fs/lustre/mdc/Makefile b/fs/lustre/mdc/Makefile index 1aeaa68..2cbc641 100644 --- a/fs/lustre/mdc/Makefile +++ b/fs/lustre/mdc/Makefile @@ -2,3 +2,4 @@ ccflags-y += -I$(srctree)/$(src)/../include obj-$(CONFIG_LUSTRE_FS) += mdc.o mdc-y := mdc_changelog.o mdc_request.o mdc_reint.o mdc_lib.o mdc_locks.o lproc_mdc.o +mdc-y += mdc_dev.o diff --git a/fs/lustre/mdc/mdc_dev.c b/fs/lustre/mdc/mdc_dev.c new file mode 100644 index 0000000..aa86d93 --- /dev/null +++ b/fs/lustre/mdc/mdc_dev.c @@ -0,0 +1,192 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * GPL HEADER START + * + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 only, + * as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * General Public License version 2 for more details (a copy is included + * in the LICENSE file that accompanied this code). + * + * You should have received a copy of the GNU General Public License + * version 2 along with this program; If not, see + * http://www.gnu.org/licenses/gpl-2.0.html + * + * GPL HEADER END + */ +/* + * Copyright (c) 2017 Intel Corporation. + */ +/* + * This file is part of Lustre, http://www.lustre.org/ + * + * Implementation of cl_device, cl_req for MDC layer. + * + * Author: Mikhail Pershin + */ + +#define DEBUG_SUBSYSTEM S_MDC + +#include +#include + +#include "mdc_internal.h" + +int mdc_page_init(const struct lu_env *env, struct cl_object *obj, + struct cl_page *page, pgoff_t index) +{ + return -ENOTSUPP; +} + +int mdc_lock_init(const struct lu_env *env, + struct cl_object *obj, struct cl_lock *lock, + const struct cl_io *unused) +{ + return -ENOTSUPP; +} + +int mdc_io_init(const struct lu_env *env, + struct cl_object *obj, struct cl_io *io) +{ + return -ENOTSUPP; +} + +/** + * Implementation of struct cl_req_operations::cro_attr_set() for MDC + * layer. MDC is responsible for struct obdo::o_id and struct obdo::o_seq + * fields. + */ +static void mdc_req_attr_set(const struct lu_env *env, struct cl_object *obj, + struct cl_req_attr *attr) +{ + u64 flags = attr->cra_flags; + + /* Copy object FID to cl_attr */ + attr->cra_oa->o_oi.oi_fid = *lu_object_fid(&obj->co_lu); + + if (flags & OBD_MD_FLGROUP) + attr->cra_oa->o_valid |= OBD_MD_FLGROUP; + + if (flags & OBD_MD_FLID) + attr->cra_oa->o_valid |= OBD_MD_FLID; +} + +static const struct cl_object_operations mdc_ops = { + .coo_page_init = mdc_page_init, + .coo_lock_init = mdc_lock_init, + .coo_io_init = mdc_io_init, + .coo_attr_get = osc_attr_get, + .coo_attr_update = osc_attr_update, + .coo_glimpse = osc_object_glimpse, + .coo_req_attr_set = mdc_req_attr_set, +}; + +static int mdc_object_init(const struct lu_env *env, struct lu_object *obj, + const struct lu_object_conf *conf) +{ + struct osc_object *osc = lu2osc(obj); + + if (osc->oo_initialized) + return 0; + + osc->oo_initialized = true; + + return osc_object_init(env, obj, conf); +} + +static void mdc_object_free(const struct lu_env *env, struct lu_object *obj) +{ + osc_object_free(env, obj); +} + +static const struct lu_object_operations mdc_lu_obj_ops = { + .loo_object_init = mdc_object_init, + .loo_object_delete = NULL, + .loo_object_release = NULL, + .loo_object_free = mdc_object_free, + .loo_object_print = osc_object_print, + .loo_object_invariant = NULL +}; + +struct lu_object *mdc_object_alloc(const struct lu_env *env, + const struct lu_object_header *unused, + struct lu_device *dev) +{ + struct osc_object *osc; + struct lu_object *obj; + + osc = kmem_cache_zalloc(osc_object_kmem, GFP_NOFS); + if (osc) { + obj = osc2lu(osc); + lu_object_init(obj, NULL, dev); + osc->oo_cl.co_ops = &mdc_ops; + obj->lo_ops = &mdc_lu_obj_ops; + osc->oo_initialized = false; + } else { + obj = NULL; + } + return obj; +} + +static int mdc_cl_process_config(const struct lu_env *env, + struct lu_device *d, struct lustre_cfg *cfg) +{ + return mdc_process_config(d->ld_obd, 0, cfg); +} + +const struct lu_device_operations mdc_lu_ops = { + .ldo_object_alloc = mdc_object_alloc, + .ldo_process_config = mdc_cl_process_config, + .ldo_recovery_complete = NULL, +}; + +static struct lu_device *mdc_device_alloc(const struct lu_env *env, + struct lu_device_type *t, + struct lustre_cfg *cfg) +{ + struct lu_device *d; + struct osc_device *od; + struct obd_device *obd; + int rc; + + od = kzalloc(sizeof(*od), GFP_NOFS); + if (!od) + return ERR_PTR(-ENOMEM); + + cl_device_init(&od->od_cl, t); + d = osc2lu_dev(od); + d->ld_ops = &mdc_lu_ops; + + /* Setup MDC OBD */ + obd = class_name2obd(lustre_cfg_string(cfg, 0)); + if (!obd) + return ERR_PTR(-ENODEV); + + rc = mdc_setup(obd, cfg); + if (rc < 0) { + osc_device_free(env, d); + return ERR_PTR(rc); + } + od->od_exp = obd->obd_self_export; + return d; +} + +static const struct lu_device_type_operations mdc_device_type_ops = { + .ldto_device_alloc = mdc_device_alloc, + .ldto_device_free = osc_device_free, + .ldto_device_init = osc_device_init, + .ldto_device_fini = osc_device_fini +}; + +struct lu_device_type mdc_device_type = { + .ldt_tags = LU_DEVICE_CL, + .ldt_name = LUSTRE_MDC_NAME, + .ldt_ops = &mdc_device_type_ops, + .ldt_ctx_tags = LCT_CL_THREAD +}; diff --git a/fs/lustre/mdc/mdc_internal.h b/fs/lustre/mdc/mdc_internal.h index 2b849e8..943b383 100644 --- a/fs/lustre/mdc/mdc_internal.h +++ b/fs/lustre/mdc/mdc_internal.h @@ -90,6 +90,9 @@ int mdc_resource_get_unused(struct obd_export *exp, const struct lu_fid *fid, /* mdc/mdc_request.c */ int mdc_fid_alloc(const struct lu_env *env, struct obd_export *exp, struct lu_fid *fid, struct md_op_data *op_data); +int mdc_setup(struct obd_device *obd, struct lustre_cfg *cfg); +int mdc_process_config(struct obd_device *obd, u32 len, void *buf); + struct obd_client_handle; int mdc_set_open_replay_data(struct obd_export *exp, @@ -149,4 +152,7 @@ static inline unsigned long hash_x_index(u64 hash, int hash64) return ~0UL - (hash + !hash); } +/* mdc_dev.c */ +extern struct lu_device_type mdc_device_type; + #endif diff --git a/fs/lustre/mdc/mdc_request.c b/fs/lustre/mdc/mdc_request.c index dd90e931..1103c15 100644 --- a/fs/lustre/mdc/mdc_request.c +++ b/fs/lustre/mdc/mdc_request.c @@ -2473,7 +2473,7 @@ static void mdc_llog_finish(struct obd_device *obd) llog_cleanup(NULL, ctxt); } -static int mdc_setup(struct obd_device *obd, struct lustre_cfg *cfg) +int mdc_setup(struct obd_device *obd, struct lustre_cfg *cfg) { int rc; @@ -2566,7 +2566,7 @@ static int mdc_cleanup(struct obd_device *obd) return client_obd_cleanup(obd); } -static int mdc_process_config(struct obd_device *obd, u32 len, void *buf) +int mdc_process_config(struct obd_device *obd, u32 len, void *buf) { struct lustre_cfg *lcfg = buf; size_t count = class_modify_config(lcfg, PARAM_MDC, @@ -2635,7 +2635,7 @@ static int __init mdc_init(void) return rc; return class_register_type(&mdc_obd_ops, &mdc_md_ops, - LUSTRE_MDC_NAME, NULL); + LUSTRE_MDC_NAME, &mdc_device_type); } static void /*__exit*/ mdc_exit(void) diff --git a/fs/lustre/osc/osc_dev.c b/fs/lustre/osc/osc_dev.c index 080c32c..7178e6b 100644 --- a/fs/lustre/osc/osc_dev.c +++ b/fs/lustre/osc/osc_dev.c @@ -48,11 +48,16 @@ */ struct kmem_cache *osc_lock_kmem; +EXPORT_SYMBOL(osc_lock_kmem); struct kmem_cache *osc_object_kmem; +EXPORT_SYMBOL(osc_object_kmem); + struct kmem_cache *osc_thread_kmem; struct kmem_cache *osc_session_kmem; struct kmem_cache *osc_extent_kmem; +EXPORT_SYMBOL(osc_extent_kmem); struct kmem_cache *osc_quota_kmem; +EXPORT_SYMBOL(osc_quota_kmem); struct lu_kmem_descr osc_caches[] = { { @@ -162,20 +167,20 @@ static int osc_cl_process_config(const struct lu_env *env, .ldo_recovery_complete = NULL }; -static int osc_device_init(const struct lu_env *env, struct lu_device *d, - const char *name, struct lu_device *next) +int osc_device_init(const struct lu_env *env, struct lu_device *d, + const char *name, struct lu_device *next) { return 0; } - -static struct lu_device *osc_device_fini(const struct lu_env *env, - struct lu_device *d) +EXPORT_SYMBOL(osc_device_init); +struct lu_device *osc_device_fini(const struct lu_env *env, + struct lu_device *d) { return NULL; } - -static struct lu_device *osc_device_free(const struct lu_env *env, - struct lu_device *d) +EXPORT_SYMBOL(osc_device_fini); +struct lu_device *osc_device_free(const struct lu_env *env, + struct lu_device *d) { struct osc_device *od = lu2osc_dev(d); @@ -183,6 +188,7 @@ static struct lu_device *osc_device_free(const struct lu_env *env, kfree(od); return NULL; } +EXPORT_SYMBOL(osc_device_free); static struct lu_device *osc_device_alloc(const struct lu_env *env, struct lu_device_type *t, diff --git a/fs/lustre/osc/osc_object.c b/fs/lustre/osc/osc_object.c index ffa31d5..9ba3785 100644 --- a/fs/lustre/osc/osc_object.c +++ b/fs/lustre/osc/osc_object.c @@ -51,8 +51,8 @@ * */ -static int osc_object_init(const struct lu_env *env, struct lu_object *obj, - const struct lu_object_conf *conf) +int osc_object_init(const struct lu_env *env, struct lu_object *obj, + const struct lu_object_conf *conf) { struct osc_object *osc = lu2osc(obj); const struct cl_object_conf *cconf = lu2cl_conf(conf); @@ -82,8 +82,9 @@ static int osc_object_init(const struct lu_env *env, struct lu_object *obj, return 0; } +EXPORT_SYMBOL(osc_object_init); -static void osc_object_free(const struct lu_env *env, struct lu_object *obj) +void osc_object_free(const struct lu_env *env, struct lu_object *obj) { struct osc_object *osc = lu2osc(obj); @@ -105,6 +106,7 @@ static void osc_object_free(const struct lu_env *env, struct lu_object *obj) lu_object_fini(obj); kmem_cache_free(osc_object_kmem, osc); } +EXPORT_SYMBOL(osc_object_free); int osc_lvb_print(const struct lu_env *env, void *cookie, lu_printer_t p, const struct ost_lvb *lvb) @@ -114,9 +116,10 @@ int osc_lvb_print(const struct lu_env *env, void *cookie, lvb->lvb_size, lvb->lvb_mtime, lvb->lvb_atime, lvb->lvb_ctime, lvb->lvb_blocks); } +EXPORT_SYMBOL(osc_lvb_print); -static int osc_object_print(const struct lu_env *env, void *cookie, - lu_printer_t p, const struct lu_object *obj) +int osc_object_print(const struct lu_env *env, void *cookie, + lu_printer_t p, const struct lu_object *obj) { struct osc_object *osc = lu2osc(obj); struct lov_oinfo *oinfo = osc->oo_oinfo; @@ -130,9 +133,10 @@ static int osc_object_print(const struct lu_env *env, void *cookie, osc_lvb_print(env, cookie, p, &oinfo->loi_lvb); return 0; } +EXPORT_SYMBOL(osc_object_print); -static int osc_attr_get(const struct lu_env *env, struct cl_object *obj, - struct cl_attr *attr) +int osc_attr_get(const struct lu_env *env, struct cl_object *obj, + struct cl_attr *attr) { struct lov_oinfo *oinfo = cl2osc(obj)->oo_oinfo; @@ -140,9 +144,10 @@ static int osc_attr_get(const struct lu_env *env, struct cl_object *obj, attr->cat_kms = oinfo->loi_kms_valid ? oinfo->loi_kms : 0; return 0; } +EXPORT_SYMBOL(osc_attr_get); -static int osc_attr_update(const struct lu_env *env, struct cl_object *obj, - const struct cl_attr *attr, unsigned int valid) +int osc_attr_update(const struct lu_env *env, struct cl_object *obj, + const struct cl_attr *attr, unsigned int valid) { struct lov_oinfo *oinfo = cl2osc(obj)->oo_oinfo; struct ost_lvb *lvb = &oinfo->loi_lvb; @@ -164,9 +169,10 @@ static int osc_attr_update(const struct lu_env *env, struct cl_object *obj, } return 0; } +EXPORT_SYMBOL(osc_attr_update); -static int osc_object_glimpse(const struct lu_env *env, - const struct cl_object *obj, struct ost_lvb *lvb) +int osc_object_glimpse(const struct lu_env *env, + const struct cl_object *obj, struct ost_lvb *lvb) { struct lov_oinfo *oinfo = cl2osc(obj)->oo_oinfo; @@ -174,6 +180,7 @@ static int osc_object_glimpse(const struct lu_env *env, lvb->lvb_blocks = oinfo->loi_lvb.lvb_blocks; return 0; } +EXPORT_SYMBOL(osc_object_glimpse); static int osc_object_ast_clear(struct ldlm_lock *lock, void *data) { From patchwork Mon Sep 30 18:55:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167141 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 53D3316B1 for ; Mon, 30 Sep 2019 19:00:09 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3C2D8224D5 for ; Mon, 30 Sep 2019 19:00:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3C2D8224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 1D7405C401C; Mon, 30 Sep 2019 11:58:36 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id B47795C33AB for ; Mon, 30 Sep 2019 11:57:11 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id AB3B610055F7; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id A841BBB; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:02 -0400 Message-Id: <1569869810-23848-44-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 043/151] lustre: lov: add MDT target to the LOV device X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mikhal Pershin , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mikhal Pershin MDC becomes LOV target like OSC for Data-on-MDT needs. Patch does the following: - new composite layout entry type is added - LLT_DOM to describe Data-on-MDT striping. - LOV process config log and checks for MDC targets organizing them separately from OSCs - LOV operations are changed where needed to understand new layout entry type WC-bug-id: https://jira.whamcloud.com/browse/LU-3285 Lustre-commit: 8b352709a66f ("LU-3285 lov: add MDT target to the LOV device") Signed-off-by: Mikhal Pershin Reviewed-on: https://review.whamcloud.com/28010 Reviewed-by: Jinshan Xiong Reviewed-by: Andreas Dilger Signed-off-by: James Simmons --- fs/lustre/include/obd.h | 8 + fs/lustre/lmv/lmv_obd.c | 2 +- fs/lustre/lov/lov_cl_internal.h | 76 +++- fs/lustre/lov/lov_dev.c | 276 +++++++++++-- fs/lustre/lov/lov_ea.c | 20 +- fs/lustre/lov/lov_internal.h | 7 + fs/lustre/lov/lov_io.c | 6 +- fs/lustre/lov/lov_obd.c | 39 +- fs/lustre/lov/lov_object.c | 696 +++++++++++++++++++++----------- fs/lustre/lov/lov_offset.c | 3 + fs/lustre/mdc/mdc_request.c | 7 +- fs/lustre/obdclass/obd_config.c | 36 +- fs/lustre/ptlrpc/wiretest.c | 4 +- include/uapi/linux/lustre/lustre_user.h | 2 +- 14 files changed, 883 insertions(+), 299 deletions(-) diff --git a/fs/lustre/include/obd.h b/fs/lustre/include/obd.h index 9514260..baa97a9 100644 --- a/fs/lustre/include/obd.h +++ b/fs/lustre/include/obd.h @@ -381,6 +381,11 @@ struct lov_tgt_desc { ltd_reap:1; /* should this target be deleted */ }; +struct lov_md_tgt_desc { + struct obd_device *lmtd_mdc; + u32 lmtd_index; +}; + struct lov_obd { struct lov_desc desc; struct lov_tgt_desc **lov_tgts; /* sparse array */ @@ -403,10 +408,13 @@ struct lov_obd { struct rw_semaphore lov_notify_lock; struct kobject *lov_tgts_kobj; + /* Data-on-MDT: MDC array */ + struct lov_md_tgt_desc *lov_mdc_tgts; }; struct lmv_tgt_desc { struct obd_uuid ltd_uuid; + struct obd_device *ltd_obd; struct obd_export *ltd_exp; u32 ltd_idx; struct mutex ltd_fid_mutex; diff --git a/fs/lustre/lmv/lmv_obd.c b/fs/lustre/lmv/lmv_obd.c index bcbda30..aabd043 100644 --- a/fs/lustre/lmv/lmv_obd.c +++ b/fs/lustre/lmv/lmv_obd.c @@ -389,7 +389,7 @@ static int lmv_add_target(struct obd_device *obd, struct obd_uuid *uuidp, if ((index < lmv->tgts_size) && lmv->tgts[index]) { tgt = lmv->tgts[index]; - CERROR("%s: UUID %s already assigned at LOV target index %d: rc = %d\n", + CERROR("%s: UUID %s already assigned at LMV target index %d: rc = %d\n", obd->obd_name, obd_uuid2str(&tgt->ltd_uuid), index, -EEXIST); mutex_unlock(&lmv->lmv_init_mutex); diff --git a/fs/lustre/lov/lov_cl_internal.h b/fs/lustre/lov/lov_cl_internal.h index 22ef7b2..069b30e 100644 --- a/fs/lustre/lov/lov_cl_internal.h +++ b/fs/lustre/lov/lov_cl_internal.h @@ -91,6 +91,12 @@ enum lov_device_flags { * Upper half. */ +/* Data-on-MDT array item in lov_device::ld_md_tgts[] */ +struct lovdom_device { + struct cl_device *ldm_mdc; + int ldm_idx; +}; + struct lov_device { /* * XXX Locking of lov-private data is missing. @@ -101,6 +107,13 @@ struct lov_device { u32 ld_target_nr; struct lovsub_device **ld_target; u32 ld_flags; + + /* Data-on-MDT devices */ + u32 ld_md_tgts_nr; + struct lovdom_device *ld_md_tgts; + struct obd_device *ld_lmv; + /* LU site for subdevices */ + struct lu_site ld_site; }; /** @@ -129,6 +142,34 @@ static inline char *llt2str(enum lov_layout_type llt) return ""; } +/** + * Return lov_layout_entry_type associated with a given composite layout + * entry. + */ +static inline u32 lov_entry_type(struct lov_stripe_md_entry *lsme) +{ + if ((lov_pattern(lsme->lsme_pattern) == LOV_PATTERN_RAID0) || + (lov_pattern(lsme->lsme_pattern) == LOV_PATTERN_MDT)) + return lov_pattern(lsme->lsme_pattern); + return 0; +} + +struct lov_layout_entry; +struct lov_object; +struct lov_lock_sub; + +struct lov_comp_layout_entry_ops { + int (*lco_init)(const struct lu_env *env, struct lov_device *dev, + struct lov_object *lov, unsigned int index, + const struct cl_object_conf *conf, + struct lov_layout_entry *lle); + void (*lco_fini)(const struct lu_env *env, + struct lov_layout_entry *lle); + int (*lco_getattr)(const struct lu_env *env, struct lov_object *obj, + unsigned int index, struct lov_layout_entry *lle, + struct cl_attr **attr); +}; + struct lov_layout_raid0 { unsigned int lo_nr; /** @@ -165,6 +206,25 @@ struct lov_layout_raid0 { struct cl_attr lo_attr; }; +struct lov_layout_dom { + /* keep this always at first place so DOM layout entry + * can be addressed also as RAID0 after initialization. + */ + struct lov_layout_raid0 lo_dom_r0; + struct lovsub_object *lo_dom; + struct lov_oinfo *lo_loi; +}; + +struct lov_layout_entry { + u32 lle_type; + struct lu_extent lle_extent; + struct lov_comp_layout_entry_ops *lle_comp_ops; + union { + struct lov_layout_raid0 lle_raid0; + struct lov_layout_dom lle_dom; + }; +}; + /** * lov-specific file state. * @@ -220,13 +280,10 @@ struct lov_object { } released; struct lov_layout_composite { /** - * Current valid entry count of lo_entries. + * Current valid entry count of entries. */ unsigned int lo_entry_count; - struct lov_layout_entry { - struct lu_extent lle_extent; - struct lov_layout_raid0 lle_raid0; - } *lo_entries; + struct lov_layout_entry *lo_entries; } composite; } u; /** @@ -633,6 +690,15 @@ static inline struct lov_thread_info *lov_env_info(const struct lu_env *env) return info; } +static inline struct lov_layout_entry *lov_entry(struct lov_object *lov, int i) +{ + LASSERT(lov->lo_type == LLT_COMP); + LASSERTF(i < lov->u.composite.lo_entry_count, + "entry %d entry_count %d", i, lov->u.composite.lo_entry_count); + + return &lov->u.composite.lo_entries[i]; +} + static inline struct lov_layout_raid0 *lov_r0(struct lov_object *lov, int i) { LASSERT(lov->lo_type == LLT_COMP); diff --git a/fs/lustre/lov/lov_dev.c b/fs/lustre/lov/lov_dev.c index a55b3f9..5ddf49a 100644 --- a/fs/lustre/lov/lov_dev.c +++ b/fs/lustre/lov/lov_dev.c @@ -146,23 +146,55 @@ struct lu_context_key lov_session_key = { /* type constructor/destructor: lov_type_{init,fini,start,stop}() */ LU_TYPE_INIT_FINI(lov, &lov_key, &lov_session_key); + +static int lov_mdc_dev_init(const struct lu_env *env, struct lov_device *ld, + struct lu_device *mdc_dev, u32 idx, u32 nr) +{ + struct cl_device *cl; + + cl = cl_type_setup(env, &ld->ld_site, &lovsub_device_type, + mdc_dev); + if (IS_ERR(cl)) + return PTR_ERR(cl); + + ld->ld_md_tgts[nr].ldm_mdc = cl; + ld->ld_md_tgts[nr].ldm_idx = idx; + return 0; +} + static struct lu_device *lov_device_fini(const struct lu_env *env, struct lu_device *d) { - int i; struct lov_device *ld = lu2lov_dev(d); + int i; LASSERT(ld->ld_lov); - if (!ld->ld_target) - return NULL; - lov_foreach_target(ld, i) { - struct lovsub_device *lsd; + if (ld->ld_lmv) { + class_decref(ld->ld_lmv, "lov", d); + ld->ld_lmv = NULL; + } + + if (ld->ld_md_tgts) { + for (i = 0; i < ld->ld_md_tgts_nr; i++) { + if (!ld->ld_md_tgts[i].ldm_mdc) + continue; - lsd = ld->ld_target[i]; - if (lsd) { - cl_stack_fini(env, lovsub2cl_dev(lsd)); - ld->ld_target[i] = NULL; + cl_stack_fini(env, ld->ld_md_tgts[i].ldm_mdc); + ld->ld_md_tgts[i].ldm_mdc = NULL; + ld->ld_lov->lov_mdc_tgts[i].lmtd_mdc = NULL; + } + } + + if (ld->ld_target) { + lov_foreach_target(ld, i) { + struct lovsub_device *lsd; + + lsd = ld->ld_target[i]; + if (lsd) { + cl_stack_fini(env, lovsub2cl_dev(lsd)); + ld->ld_target[i] = NULL; + } } } return NULL; @@ -175,9 +207,28 @@ static int lov_device_init(const struct lu_env *env, struct lu_device *d, int i; int rc = 0; - LASSERT(d->ld_site); + /* check all added already MDC subdevices and initialize them */ + for (i = 0; i < ld->ld_md_tgts_nr; i++) { + struct obd_device *mdc; + u32 idx; + + mdc = ld->ld_lov->lov_mdc_tgts[i].lmtd_mdc; + idx = ld->ld_lov->lov_mdc_tgts[i].lmtd_index; + + if (!mdc) + continue; + + rc = lov_mdc_dev_init(env, ld, mdc->obd_lu_dev, idx, i); + if (rc) { + CERROR("%s: failed to add MDC %s as target: rc = %d\n", + d->ld_obd->obd_name, + obd_uuid2str(&mdc->obd_uuid), rc); + goto out_err; + } + } + if (!ld->ld_target) - return rc; + return 0; lov_foreach_target(ld, i) { struct lovsub_device *lsd; @@ -188,21 +239,21 @@ static int lov_device_init(const struct lu_env *env, struct lu_device *d, if (!desc) continue; - cl = cl_type_setup(env, d->ld_site, &lovsub_device_type, + cl = cl_type_setup(env, &ld->ld_site, &lovsub_device_type, desc->ltd_obd->obd_lu_dev); if (IS_ERR(cl)) { rc = PTR_ERR(cl); - break; + goto out_err; } + lsd = cl2lovsub_dev(cl); ld->ld_target[i] = lsd; } + ld->ld_flags |= LOV_DEV_INITIALIZED; + return 0; - if (rc) - lov_device_fini(env, d); - else - ld->ld_flags |= LOV_DEV_INITIALIZED; - +out_err: + lu_device_fini(d); return rc; } @@ -211,8 +262,17 @@ static struct lu_device *lov_device_free(const struct lu_env *env, { struct lov_device *ld = lu2lov_dev(d); + lu_site_fini(&ld->ld_site); + cl_device_fini(lu2cl_dev(d)); kfree(ld->ld_target); + ld->ld_target = NULL; + kfree(ld->ld_md_tgts); + ld->ld_md_tgts = NULL; + /* free array of MDCs */ + kfree(ld->ld_lov->lov_mdc_tgts); + ld->ld_lov->lov_mdc_tgts = NULL; + kfree(ld); return NULL; } @@ -277,9 +337,7 @@ static int lov_cl_add_target(const struct lu_env *env, struct lu_device *dev, rc = lov_expand_targets(env, ld); if (rc == 0 && ld->ld_flags & LOV_DEV_INITIALIZED) { - LASSERT(dev->ld_site); - - cl = cl_type_setup(env, dev->ld_site, &lovsub_device_type, + cl = cl_type_setup(env, &ld->ld_site, &lovsub_device_type, tgt->ltd_obd->obd_lu_dev); if (!IS_ERR(cl)) { lsd = cl2lovsub_dev(cl); @@ -297,6 +355,84 @@ static int lov_cl_add_target(const struct lu_env *env, struct lu_device *dev, return rc; } +/** + * Add new MDC target device in LOV. + * + * This function is part of the configuration log processing. It adds new MDC + * device to the MDC device array indexed by their indexes. + * + * @env execution environment + * @d LU device of LOV device + * @mdc MDC device to add + * @idx MDC device index + * + * Return: 0 if successful + * negative value on error + */ +static int lov_add_mdc_target(const struct lu_env *env, struct lu_device *d, + struct obd_device *mdc, u32 idx) +{ + struct lov_device *ld = lu2lov_dev(d); + struct obd_device *lov_obd = d->ld_obd; + struct obd_device *lmv_obd; + int next; + int rc = 0; + + LASSERT(mdc); + if (ld->ld_md_tgts_nr == LOV_MDC_TGT_MAX) { + /* If the maximum value of LOV_MDC_TGT_MAX will become too + * small then all MD target handling must be rewritten in LOD + * manner, check lod_add_device() and related functionality. + */ + CERROR("%s: cannot serve more than %d MDC devices\n", + lov_obd->obd_name, LOV_MDC_TGT_MAX); + return -ERANGE; + } + + /* grab FLD from lmv, do that here, when first MDC is added + * to be sure LMV is set up and can be found + */ + if (!ld->ld_lmv) { + next = 0; + while ((lmv_obd = class_devices_in_group(&lov_obd->obd_uuid, + &next)) != NULL) { + if ((strncmp(lmv_obd->obd_type->typ_name, + LUSTRE_LMV_NAME, + strlen(LUSTRE_LMV_NAME)) == 0)) + break; + } + if (!lmv_obd) { + CERROR("%s: cannot find LMV OBD by UUID (%s)\n", + lov_obd->obd_name, + obd_uuid2str(&lmv_obd->obd_uuid)); + return -ENODEV; + } + spin_lock(&lmv_obd->obd_dev_lock); + class_incref(lmv_obd, "lov", ld); + spin_unlock(&lmv_obd->obd_dev_lock); + ld->ld_lmv = lmv_obd; + } + + LASSERT(!lov_obd->u.lov.lov_mdc_tgts[ld->ld_md_tgts_nr].lmtd_mdc); + + if (ld->ld_flags & LOV_DEV_INITIALIZED) { + rc = lov_mdc_dev_init(env, ld, mdc->obd_lu_dev, idx, + ld->ld_md_tgts_nr); + if (rc) { + CERROR("%s: failed to add MDC %s as target: rc = %d\n", + lov_obd->obd_name, obd_uuid2str(&mdc->obd_uuid), + rc); + return rc; + } + } + + lov_obd->u.lov.lov_mdc_tgts[ld->ld_md_tgts_nr].lmtd_mdc = mdc; + lov_obd->u.lov.lov_mdc_tgts[ld->ld_md_tgts_nr].lmtd_index = idx; + ld->ld_md_tgts_nr++; + + return rc; +} + static int lov_process_config(const struct lu_env *env, struct lu_device *d, struct lustre_cfg *cfg) { @@ -309,23 +445,52 @@ static int lov_process_config(const struct lu_env *env, lov_tgts_getref(obd); cmd = cfg->lcfg_command; + rc = lov_process_config_base(d->ld_obd, cfg, &index, &gen); - if (rc == 0) { - switch (cmd) { - case LCFG_LOV_ADD_OBD: - case LCFG_LOV_ADD_INA: - rc = lov_cl_add_target(env, d, index); - if (rc != 0) - lov_del_target(d->ld_obd, index, NULL, 0); - break; - case LCFG_LOV_DEL_OBD: - lov_cl_del_target(env, d, index); - break; + if (rc < 0) + goto out; + + switch (cmd) { + case LCFG_LOV_ADD_OBD: + case LCFG_LOV_ADD_INA: + rc = lov_cl_add_target(env, d, index); + if (rc != 0) + lov_del_target(d->ld_obd, index, NULL, 0); + break; + case LCFG_LOV_DEL_OBD: + lov_cl_del_target(env, d, index); + break; + case LCFG_ADD_MDC: + { + struct obd_device *mdc; + struct obd_uuid tgt_uuid; + + /* modify_mdc_tgts add 0:lustre-clilmv 1:lustre-MDT0000_UUID + * 2:0 3:1 4:lustre-MDT0000-mdc_UUID + */ + if (LUSTRE_CFG_BUFLEN(cfg, 1) > sizeof(tgt_uuid.uuid)) { + rc = -EINVAL; + goto out; } - } - lov_tgts_putref(obd); + obd_str2uuid(&tgt_uuid, lustre_cfg_buf(cfg, 1)); + if (sscanf(lustre_cfg_buf(cfg, 2), "%d", &index) != 1) { + rc = -EINVAL; + goto out; + } + mdc = class_find_client_obd(&tgt_uuid, LUSTRE_MDC_NAME, + &obd->obd_uuid); + if (!mdc) { + rc = -ENODEV; + goto out; + } + rc = lov_add_mdc_target(env, d, mdc, index); + break; + } + } +out: + lov_tgts_putref(obd); return rc; } @@ -355,13 +520,50 @@ static struct lu_device *lov_device_alloc(const struct lu_env *env, obd = class_name2obd(lustre_cfg_string(cfg, 0)); LASSERT(obd); rc = lov_setup(obd, cfg); - if (rc) { - lov_device_free(env, d); - return ERR_PTR(rc); + if (rc) + goto out; + + /* Alloc MDC devices array */ + /* XXX: need dynamic allocation at some moment */ + ld->ld_md_tgts = kcalloc(LOV_MDC_TGT_MAX, sizeof(*ld->ld_md_tgts), + GFP_NOFS); + if (!ld->ld_md_tgts) { + rc = -ENOMEM; + goto out; } + ld->ld_md_tgts_nr = 0; ld->ld_lov = &obd->u.lov; + ld->ld_lov->lov_mdc_tgts = + kcalloc(LOV_MDC_TGT_MAX, + sizeof(*ld->ld_lov->lov_mdc_tgts), + GFP_NOFS); + if (!ld->ld_lov->lov_mdc_tgts) { + rc = -ENOMEM; + goto out_md_tgts; + } + + rc = lu_site_init(&ld->ld_site, d); + if (rc != 0) + goto out_mdc_tgts; + + rc = lu_site_init_finish(&ld->ld_site); + if (rc != 0) + goto out_site; + return d; +out_site: + lu_site_fini(&ld->ld_site); +out_mdc_tgts: + kfree(ld->ld_lov->lov_mdc_tgts); + ld->ld_lov->lov_mdc_tgts = NULL; +out_md_tgts: + kfree(ld->ld_md_tgts); + ld->ld_md_tgts = NULL; +out: + kfree(ld); + + return ERR_PTR(rc); } static const struct lu_device_type_operations lov_device_type_ops = { diff --git a/fs/lustre/lov/lov_ea.c b/fs/lustre/lov/lov_ea.c index 395ef77..e1630f6 100644 --- a/fs/lustre/lov/lov_ea.c +++ b/fs/lustre/lov/lov_ea.c @@ -95,7 +95,8 @@ static int lsm_lmm_verify_v1v3(struct lov_mds_md *lmm, size_t lmm_size, return -EINVAL; } - if (lov_pattern(le32_to_cpu(lmm->lmm_pattern)) != LOV_PATTERN_RAID0) { + if (lov_pattern(le32_to_cpu(lmm->lmm_pattern)) != LOV_PATTERN_MDT && + lov_pattern(le32_to_cpu(lmm->lmm_pattern)) != LOV_PATTERN_RAID0) { CERROR("bad striping pattern\n"); lov_dump_lmm_common(D_WARNING, lmm); return -EINVAL; @@ -206,6 +207,12 @@ void lsm_free(struct lov_stripe_md *lsm) } } + /* with Data-on-MDT set maxbytes to stripe size */ + if (lsme_is_dom(lsme)) { + lov_bytes = lsme->lsme_stripe_size; + goto out_dom; + } + for (i = 0; i < stripe_count; i++) { struct lov_tgt_desc *ltd; struct lov_oinfo *loi; @@ -253,6 +260,7 @@ void lsm_free(struct lov_stripe_md *lsm) lov_bytes = min_stripe_maxbytes * stripe_count; +out_dom: if (maxbytes) { if (lov_bytes < min_stripe_maxbytes) /* handle overflow */ *maxbytes = MAX_LFS_FILESIZE; @@ -385,7 +393,8 @@ static int lsm_verify_comp_md_v1(struct lov_comp_md_v1 *lcm, unsigned int magic; stripe_count = le16_to_cpu(lmm->lmm_stripe_count); - if (stripe_count == 0) + if (stripe_count == 0 && + lov_pattern(le32_to_cpu(lmm->lmm_pattern)) != LOV_PATTERN_MDT) return ERR_PTR(-EINVAL); /* un-instantiated lmm contains no ost id info, i.e. lov_ost_data_v1 */ @@ -474,9 +483,10 @@ static int lsm_verify_comp_md_v1(struct lov_comp_md_v1 *lcm, /* the last component hasn't been defined, or * lsm_maxbytes overflowed. */ - if (lsme->lsme_extent.e_end != LUSTRE_EOF || - lsm->lsm_maxbytes < - (loff_t)lsme->lsme_extent.e_start) + if (!lsme_is_dom(lsme) && + (lsme->lsme_extent.e_end != LUSTRE_EOF || + lsm->lsm_maxbytes < + (loff_t)lsme->lsme_extent.e_start)) lsm->lsm_maxbytes = MAX_LFS_FILESIZE; } } diff --git a/fs/lustre/lov/lov_internal.h b/fs/lustre/lov/lov_internal.h index f69f2d6..e18ea8e 100644 --- a/fs/lustre/lov/lov_internal.h +++ b/fs/lustre/lov/lov_internal.h @@ -57,6 +57,11 @@ struct lov_stripe_md_entry { struct lov_oinfo *lsme_oinfo[]; }; +static inline bool lsme_is_dom(struct lov_stripe_md_entry *lsme) +{ + return (lov_pattern(lsme->lsme_pattern) == LOV_PATTERN_MDT); +} + static inline void copy_lsm_entry(struct lov_stripe_md_entry *dst, struct lov_stripe_md_entry *src) { @@ -300,6 +305,8 @@ struct lov_stripe_md *lov_unpackmd(struct lov_obd *lov, void *buf, /* lov_cl.c */ extern struct lu_device_type lov_device_type; +#define LOV_MDC_TGT_MAX 256 + /* ost_pool methods */ int lov_ost_pool_init(struct ost_pool *op, unsigned int count); int lov_ost_pool_extend(struct ost_pool *op, unsigned int min_count); diff --git a/fs/lustre/lov/lov_io.c b/fs/lustre/lov/lov_io.c index a72069f..c7fe4a2 100644 --- a/fs/lustre/lov/lov_io.c +++ b/fs/lustre/lov/lov_io.c @@ -533,7 +533,11 @@ static int lov_io_setattr_iter_init(const struct lu_env *env, if (cl_io_is_trunc(io) && lio->lis_pos > 0) { index = lov_lsm_entry(lsm, lio->lis_pos - 1); - if (index > 0 && !lsm_entry_inited(lsm, index)) { + /* no entry found for such offset */ + if (index < 0) { + io->ci_result = -ENODATA; + return io->ci_result; + } else if (!lsm_entry_inited(lsm, index)) { io->ci_need_write_intent = 1; io->ci_result = -ENODATA; return io->ci_result; diff --git a/fs/lustre/lov/lov_obd.c b/fs/lustre/lov/lov_obd.c index 5dbc00e..4ced5f7 100644 --- a/fs/lustre/lov/lov_obd.c +++ b/fs/lustre/lov/lov_obd.c @@ -852,6 +852,9 @@ int lov_process_config_base(struct obd_device *obd, struct lustre_cfg *lcfg, int rc = 0; switch (cmd = lcfg->lcfg_command) { + case LCFG_ADD_MDC: + case LCFG_DEL_MDC: + break; case LCFG_LOV_ADD_OBD: case LCFG_LOV_ADD_INA: case LCFG_LOV_DEL_OBD: { @@ -1179,31 +1182,32 @@ static int lov_set_info_async(const struct lu_env *env, struct obd_export *exp, { struct obd_device *obddev = class_exp2obd(exp); struct lov_obd *lov = &obddev->u.lov; - u32 count; - int i, rc = 0, err; struct lov_tgt_desc *tgt; - int do_inactive = 0, no_set = 0; + bool do_inactive = false; + bool no_set = false; + int rc = 0; + int err; + u32 i; if (!set) { - no_set = 1; + no_set = true; set = ptlrpc_prep_set(); if (!set) return -ENOMEM; } lov_tgts_getref(obddev); - count = lov->desc.ld_tgt_count; if (KEY_IS(KEY_CHECKSUM)) { - do_inactive = 1; + do_inactive = true; } else if (KEY_IS(KEY_CACHE_SET)) { LASSERT(!lov->lov_cache); lov->lov_cache = val; - do_inactive = 1; + do_inactive = true; cl_cache_incref(lov->lov_cache); } - for (i = 0; i < count; i++) { + for (i = 0; i < lov->desc.ld_tgt_count; i++) { tgt = lov->lov_tgts[i]; /* OST was disconnected */ @@ -1216,14 +1220,29 @@ static int lov_set_info_async(const struct lu_env *env, struct obd_export *exp, err = obd_set_info_async(env, tgt->ltd_exp, keylen, key, vallen, val, set); - if (!rc) + + if (rc == 0) + rc = err; + } + + /* cycle through MDC target for Data-on-MDT */ + for (i = 0; i < LOV_MDC_TGT_MAX; i++) { + struct obd_device *mdc; + + mdc = lov->lov_mdc_tgts[i].lmtd_mdc; + if (!mdc) + continue; + + err = obd_set_info_async(env, mdc->obd_self_export, + keylen, key, vallen, val, set); + if (rc == 0) rc = err; } lov_tgts_putref(obddev); if (no_set) { err = ptlrpc_set_wait(set); - if (!rc) + if (rc == 0) rc = err; ptlrpc_set_destroy(set); } diff --git a/fs/lustre/lov/lov_object.c b/fs/lustre/lov/lov_object.c index caeff89..186b875 100644 --- a/fs/lustre/lov/lov_object.c +++ b/fs/lustre/lov/lov_object.c @@ -90,13 +90,6 @@ static void lov_lsm_put(struct lov_stripe_md *lsm) * Lov object layout operations. * */ -static int lov_init_empty(const struct lu_env *env, struct lov_device *dev, - struct lov_object *lov, struct lov_stripe_md *lsm, - const struct cl_object_conf *conf, - union lov_layout_state *state) -{ - return 0; -} static struct cl_object *lov_sub_find(const struct lu_env *env, struct cl_device *dev, @@ -110,9 +103,25 @@ static struct cl_object *lov_sub_find(const struct lu_env *env, return lu2cl(o); } +static int lov_page_slice_fixup(struct lov_object *lov, + struct cl_object *stripe) +{ + struct cl_object_header *hdr = cl_object_header(&lov->lo_cl); + struct cl_object *o; + + if (!stripe) + return hdr->coh_page_bufsize - lov->lo_cl.co_slice_off - + cfs_size_round(sizeof(struct lov_page)); + + cl_object_for_each(o, stripe) + o->co_slice_off += hdr->coh_page_bufsize; + + return cl_object_header(stripe)->coh_page_bufsize; +} + static int lov_init_sub(const struct lu_env *env, struct lov_object *lov, - struct cl_object *subobj, struct lov_layout_raid0 *r0, - struct lov_oinfo *oinfo, int idx) + struct cl_object *subobj, struct lov_oinfo *oinfo, + int idx) { int stripe = lov_comp_stripe(idx); int entry = lov_comp_entry(idx); @@ -146,13 +155,14 @@ static int lov_init_sub(const struct lu_env *env, struct lov_object *lov, spin_lock(&subhdr->coh_attr_guard); parent = subhdr->coh_parent; if (!parent) { + struct lovsub_object *lso = cl2lovsub(subobj); + subhdr->coh_parent = hdr; spin_unlock(&subhdr->coh_attr_guard); subhdr->coh_nesting = hdr->coh_nesting + 1; lu_object_ref_add(&subobj->co_lu, "lov-parent", lov); - r0->lo_sub[stripe] = cl2lovsub(subobj); - r0->lo_sub[stripe]->lso_super = lov; - r0->lo_sub[stripe]->lso_index = idx; + lso->lso_super = lov; + lso->lso_index = idx; result = 0; } else { struct lu_object *old_obj; @@ -183,33 +193,19 @@ static int lov_init_sub(const struct lu_env *env, struct lov_object *lov, return result; } -static int lov_page_slice_fixup(struct lov_object *lov, - struct cl_object *stripe) -{ - struct cl_object_header *hdr = cl_object_header(&lov->lo_cl); - struct cl_object *o; - - if (!stripe) - return hdr->coh_page_bufsize - lov->lo_cl.co_slice_off - - cfs_size_round(sizeof(struct lov_page)); - - cl_object_for_each(o, stripe) - o->co_slice_off += hdr->coh_page_bufsize; - - return cl_object_header(stripe)->coh_page_bufsize; -} - static int lov_init_raid0(const struct lu_env *env, struct lov_device *dev, - struct lov_object *lov, int index, - struct lov_layout_raid0 *r0) + struct lov_object *lov, unsigned int index, + const struct cl_object_conf *conf, + struct lov_layout_entry *lle) { struct lov_stripe_md_entry *lse = lov_lse(lov, index); + struct lov_layout_raid0 *r0 = &lle->lle_raid0; struct lov_thread_info *lti = lov_env_info(env); struct cl_object_conf *subconf = <i->lti_stripe_conf; struct lu_fid *ofid = <i->lti_fid; struct cl_object *stripe; int result; - int psz; + int psz, sz; int i; spin_lock_init(&r0->lo_sub_lock); @@ -261,7 +257,7 @@ static int lov_init_raid0(const struct lu_env *env, struct lov_device *dev, goto out; } - result = lov_init_sub(env, lov, stripe, r0, oinfo, + result = lov_init_sub(env, lov, stripe, oinfo, lov_comp_index(index, i)); if (result == -EAGAIN) { /* try again */ --i; @@ -270,8 +266,9 @@ static int lov_init_raid0(const struct lu_env *env, struct lov_device *dev, } if (result == 0) { - int sz = lov_page_slice_fixup(lov, stripe); + r0->lo_sub[i] = cl2lovsub(stripe); + sz = lov_page_slice_fixup(lov, stripe); LASSERT(ergo(psz > 0, psz == sz)); psz = sz; } @@ -282,12 +279,333 @@ static int lov_init_raid0(const struct lu_env *env, struct lov_device *dev, return result; } +static void lov_subobject_kill(const struct lu_env *env, struct lov_object *lov, + struct lov_layout_raid0 *r0, + struct lovsub_object *los, int idx) +{ + struct cl_object *sub; + struct lu_site *site; + wait_queue_head_t *wq; + + LASSERT(r0->lo_sub[idx] == los); + + sub = lovsub2cl(los); + site = sub->co_lu.lo_dev->ld_site; + wq = lu_site_wq_from_fid(site, &sub->co_lu.lo_header->loh_fid); + + cl_object_kill(env, sub); + /* release a reference to the sub-object and ... */ + lu_object_ref_del(&sub->co_lu, "lov-parent", lov); + cl_object_put(env, sub); + + /* ... wait until it is actually destroyed---sub-object clears its + * ->lo_sub[] slot in lovsub_object_free() + */ + wait_event(*wq, r0->lo_sub[idx] != los); + LASSERT(!r0->lo_sub[idx]); +} + +static void lov_delete_raid0(const struct lu_env *env, struct lov_object *lov, + struct lov_layout_entry *lle) +{ + struct lov_layout_raid0 *r0 = &lle->lle_raid0; + + if (r0->lo_sub) { + int i; + + for (i = 0; i < r0->lo_nr; ++i) { + struct lovsub_object *los = r0->lo_sub[i]; + + if (los) { + cl_object_prune(env, &los->lso_cl); + /* + * If top-level object is to be evicted from + * the cache, so are its sub-objects. + */ + lov_subobject_kill(env, lov, r0, los, i); + } + } + } +} + +static void lov_fini_raid0(const struct lu_env *env, + struct lov_layout_entry *lle) +{ + struct lov_layout_raid0 *r0 = &lle->lle_raid0; + + if (r0->lo_sub) { + kvfree(r0->lo_sub); + r0->lo_sub = NULL; + } +} + +static int lov_print_raid0(const struct lu_env *env, void *cookie, + lu_printer_t p, const struct lov_layout_entry *lle) +{ + const struct lov_layout_raid0 *r0 = &lle->lle_raid0; + int i; + + for (i = 0; i < r0->lo_nr; ++i) { + struct lu_object *sub; + + if (r0->lo_sub[i]) { + sub = lovsub2lu(r0->lo_sub[i]); + lu_object_print(env, cookie, p, sub); + } else { + (*p)(env, cookie, "sub %d absent\n", i); + } + } + return 0; +} + +static int lov_attr_get_raid0(const struct lu_env *env, struct lov_object *lov, + unsigned int index, struct lov_layout_entry *lle, + struct cl_attr **lov_attr) +{ + struct lov_layout_raid0 *r0 = &lle->lle_raid0; + struct lov_stripe_md *lsm = lov->lo_lsm; + struct ost_lvb *lvb = &lov_env_info(env)->lti_lvb; + struct cl_attr *attr = &r0->lo_attr; + u64 kms = 0; + int result = 0; + + if (r0->lo_attr_valid) { + *lov_attr = attr; + return 0; + } + + memset(lvb, 0, sizeof(*lvb)); + + /* XXX: timestamps can be negative by sanity:test_39m, + * how can it be? + */ + lvb->lvb_atime = LLONG_MIN; + lvb->lvb_ctime = LLONG_MIN; + lvb->lvb_mtime = LLONG_MIN; + + /* + * XXX that should be replaced with a loop over sub-objects, + * doing cl_object_attr_get() on them. But for now, let's + * reuse old lov code. + */ + + /* + * XXX take lsm spin-lock to keep lov_merge_lvb_kms() + * happy. It's not needed, because new code uses + * ->coh_attr_guard spin-lock to protect consistency of + * sub-object attributes. + */ + lov_stripe_lock(lsm); + result = lov_merge_lvb_kms(lsm, index, lvb, &kms); + lov_stripe_unlock(lsm); + if (result == 0) { + cl_lvb2attr(attr, lvb); + attr->cat_kms = kms; + r0->lo_attr_valid = 1; + *lov_attr = attr; + } + + return result; +} + +static struct lov_comp_layout_entry_ops raid0_ops = { + .lco_init = lov_init_raid0, + .lco_fini = lov_fini_raid0, + .lco_getattr = lov_attr_get_raid0, +}; + +static int lov_attr_get_dom(const struct lu_env *env, struct lov_object *lov, + unsigned int index, struct lov_layout_entry *lle, + struct cl_attr **lov_attr) +{ + struct lov_layout_dom *dom = &lle->lle_dom; + struct lov_oinfo *loi = dom->lo_loi; + struct cl_attr *attr = &dom->lo_dom_r0.lo_attr; + + if (dom->lo_dom_r0.lo_attr_valid) { + *lov_attr = attr; + return 0; + } + + if (OST_LVB_IS_ERR(loi->loi_lvb.lvb_blocks)) + return OST_LVB_GET_ERR(loi->loi_lvb.lvb_blocks); + + cl_lvb2attr(attr, &loi->loi_lvb); + attr->cat_kms = attr->cat_size > loi->loi_kms ? attr->cat_size : + loi->loi_kms; + dom->lo_dom_r0.lo_attr_valid = 1; + *lov_attr = attr; + + return 0; +} + +/** + * Lookup FLD to get MDS index of the given DOM object FID. + * + * @ld LOV device + * @fid FID to lookup + * @nr index in MDC array to return back + * + * Return: 0 and @mds filled with MDS index if successful + * negative value on error + */ +static int lov_fld_lookup(struct lov_device *ld, const struct lu_fid *fid, + u32 *nr) +{ + u32 mds_idx; + int i, rc; + + rc = fld_client_lookup(&ld->ld_lmv->u.lmv.lmv_fld, fid_seq(fid), + &mds_idx, LU_SEQ_RANGE_MDT, NULL); + if (rc) { + CERROR("%s: error while looking for mds number. Seq %#llx, err = %d\n", + lu_dev_name(cl2lu_dev(&ld->ld_cl)), fid_seq(fid), rc); + return rc; + } + + CDEBUG(D_INODE, "FLD lookup got mds #%x for fid=" DFID "\n", + mds_idx, PFID(fid)); + + /* find proper MDC device in the array */ + for (i = 0; i < ld->ld_md_tgts_nr; i++) { + if (ld->ld_md_tgts[i].ldm_mdc && + ld->ld_md_tgts[i].ldm_idx == mds_idx) + break; + } + + if (i == ld->ld_md_tgts_nr) { + CERROR("%s: cannot find corresponding MDC device for mds #%x for fid=" DFID "\n", + lu_dev_name(cl2lu_dev(&ld->ld_cl)), mds_idx, PFID(fid)); + rc = -EINVAL; + } else { + *nr = i; + } + return rc; +} + +/** + * Implementation of lov_comp_layout_entry_ops::lco_init for DOM object. + * + * Init the DOM object for the first time. It prepares also RAID0 entry + * for it to use in common methods with ordinary RAID0 layout entries. + * + * @env execution environment + * @dev LOV device + * @lov LOV object + * @index Composite layout entry index in LSM + * @lle Composite LOV layout entry + */ +static int lov_init_dom(const struct lu_env *env, struct lov_device *dev, + struct lov_object *lov, unsigned int index, + const struct cl_object_conf *conf, + struct lov_layout_entry *lle) +{ + struct lov_thread_info *lti = lov_env_info(env); + struct lov_stripe_md_entry *lsme = lov_lse(lov, index); + struct cl_object *clo; + struct lu_object *o = lov2lu(lov); + const struct lu_fid *fid = lu_object_fid(o); + struct cl_device *mdcdev; + struct lov_oinfo *loi = NULL; + struct cl_object_conf *sconf = <i->lti_stripe_conf; + struct inode *inode = conf->coc_inode; + u32 idx = 0; + int rc; + + LASSERT(index == 0); + + /* find proper MDS device */ + rc = lov_fld_lookup(dev, fid, &idx); + if (rc) + return rc; + + LASSERTF(dev->ld_md_tgts[idx].ldm_mdc, + "LOV md target[%u] is NULL\n", idx); + + /* check lsm is DOM, more checks are needed */ + LASSERT(lsme->lsme_stripe_count == 0); + + /* + * Create lower cl_objects. + */ + mdcdev = dev->ld_md_tgts[idx].ldm_mdc; + + LASSERTF(mdcdev, "non-initialized mdc subdev\n"); + + /* DoM object has no oinfo in LSM entry, create it exclusively */ + loi = kmem_cache_zalloc(lov_oinfo_slab, GFP_NOFS); + if (!loi) + return -ENOMEM; + + fid_to_ostid(lu_object_fid(lov2lu(lov)), &loi->loi_oi); + /* Initialize lvb structure */ + loi->loi_lvb.lvb_mtime = inode->i_mtime.tv_sec; + loi->loi_lvb.lvb_atime = inode->i_atime.tv_sec; + loi->loi_lvb.lvb_ctime = inode->i_ctime.tv_sec; + loi->loi_lvb.lvb_blocks = inode->i_blocks; + loi->loi_lvb.lvb_size = i_size_read(inode); + if (loi->loi_lvb.lvb_size > lsme->lsme_stripe_size) + loi->loi_lvb.lvb_size = lsme->lsme_stripe_size; + loi_kms_set(loi, loi->loi_lvb.lvb_size); + + sconf->u.coc_oinfo = loi; +again: + clo = lov_sub_find(env, mdcdev, fid, sconf); + if (IS_ERR(clo)) { + rc = PTR_ERR(clo); + goto out; + } + + rc = lov_init_sub(env, lov, clo, loi, lov_comp_index(index, 0)); + if (rc == -EAGAIN) /* try again */ + goto again; + else if (rc != 0) + goto out; + + lle->lle_dom.lo_dom = cl2lovsub(clo); + spin_lock_init(&lle->lle_dom.lo_dom_r0.lo_sub_lock); + lle->lle_dom.lo_dom_r0.lo_nr = 1; + lle->lle_dom.lo_dom_r0.lo_sub = &lle->lle_dom.lo_dom; + lle->lle_dom.lo_loi = loi; + + rc = lov_page_slice_fixup(lov, clo); + return rc; + +out: + kmem_cache_free(lov_oinfo_slab, loi); + return rc; +} + +/** + * Implementation of lov_layout_operations::llo_fini for DOM object. + * + * Finish the DOM object and free related memory. + * + * @env execution environment + * @lov LOV object + * @state LOV layout state + */ +static void lov_fini_dom(const struct lu_env *env, + struct lov_layout_entry *lle) +{ + if (lle->lle_dom.lo_dom) + lle->lle_dom.lo_dom = NULL; + kmem_cache_free(lov_oinfo_slab, lle->lle_dom.lo_loi); +} + +static struct lov_comp_layout_entry_ops dom_ops = { + .lco_init = lov_init_dom, + .lco_fini = lov_fini_dom, + .lco_getattr = lov_attr_get_dom, +}; + static int lov_init_composite(const struct lu_env *env, struct lov_device *dev, struct lov_object *lov, struct lov_stripe_md *lsm, const struct cl_object_conf *conf, union lov_layout_state *state) { struct lov_layout_composite *comp = &state->composite; + struct lov_layout_entry *lle; unsigned int entry_count; unsigned int psz = 0; int result = 0; @@ -306,24 +624,45 @@ static int lov_init_composite(const struct lu_env *env, struct lov_device *dev, if (!comp->lo_entries) return -ENOMEM; + /* Initiate all entry types and extents data at first */ for (i = 0; i < entry_count; i++) { - struct lov_layout_entry *le = &comp->lo_entries[i]; + lle = &comp->lo_entries[i]; - le->lle_extent = lsm->lsm_entries[i]->lsme_extent; + lle->lle_type = lov_entry_type(lsm->lsm_entries[i]); + switch (lle->lle_type) { + case LOV_PATTERN_RAID0: + lle->lle_comp_ops = &raid0_ops; + break; + case LOV_PATTERN_MDT: + lle->lle_comp_ops = &dom_ops; + break; + default: + CERROR("%s: unknown composite layout entry type %i\n", + lov2obd(dev->ld_lov)->obd_name, + lsm->lsm_entries[i]->lsme_pattern); + dump_lsm(D_ERROR, lsm); + return -EIO; + } + lle->lle_extent = lsm->lsm_entries[i]->lsme_extent; + } + + i = 0; + lov_foreach_layout_entry(lov, lle) { /** * If the component has not been init-ed on MDS side, for * PFL layout, we'd know that the components beyond this one * will be dynamically init-ed later on file write/trunc ops. */ - if (!lsm_entry_inited(lsm, i)) - continue; - - result = lov_init_raid0(env, dev, lov, i, &le->lle_raid0); - if (result < 0) - break; + if (lsm_entry_inited(lsm, i)) { + result = lle->lle_comp_ops->lco_init(env, dev, lov, i, + conf, lle); + if (result < 0) + break; - LASSERT(ergo(psz > 0, psz == result)); - psz = result; + LASSERT(ergo(psz > 0, psz == result)); + psz = result; + } + i++; } if (psz > 0) cl_object_header(&lov->lo_cl)->coh_page_bufsize += psz; @@ -331,10 +670,19 @@ static int lov_init_composite(const struct lu_env *env, struct lov_device *dev, return result > 0 ? 0 : result; } -static int lov_init_released(const struct lu_env *env, struct lov_device *dev, - struct lov_object *lov, struct lov_stripe_md *lsm, +static int lov_init_empty(const struct lu_env *env, struct lov_device *dev, + struct lov_object *lov, struct lov_stripe_md *lsm, + const struct cl_object_conf *conf, + union lov_layout_state *state) +{ + return 0; +} + +static int lov_init_released(const struct lu_env *env, + struct lov_device *dev, struct lov_object *lov, + struct lov_stripe_md *lsm, const struct cl_object_conf *conf, - union lov_layout_state *state) + union lov_layout_state *state) { LASSERT(lsm); LASSERT(lsm->lsm_is_released); @@ -344,41 +692,6 @@ static int lov_init_released(const struct lu_env *env, struct lov_device *dev, return 0; } -static struct cl_object *lov_find_subobj(const struct lu_env *env, - struct lov_object *lov, - struct lov_stripe_md *lsm, - int index) -{ - struct lov_device *dev = lu2lov_dev(lov2lu(lov)->lo_dev); - struct lov_thread_info *lti = lov_env_info(env); - struct lu_fid *ofid = <i->lti_fid; - int stripe = lov_comp_stripe(index); - int entry = lov_comp_entry(index); - struct cl_object *result = NULL; - struct cl_device *subdev; - struct lov_oinfo *oinfo; - int ost_idx; - int rc; - - if (lov->lo_type != LLT_COMP) - goto out; - - if (entry >= lsm->lsm_entry_count || - stripe >= lsm->lsm_entries[entry]->lsme_stripe_count) - goto out; - - oinfo = lsm->lsm_entries[entry]->lsme_oinfo[stripe]; - ost_idx = oinfo->loi_ost_idx; - rc = ostid_to_fid(ofid, &oinfo->loi_oi, ost_idx); - if (rc) - goto out; - - subdev = lovsub2cl_dev(dev->ld_target[ost_idx]); - result = lov_sub_find(env, subdev, ofid, NULL); -out: - return result ? result : ERR_PTR(-EINVAL); -} - static int lov_delete_empty(const struct lu_env *env, struct lov_object *lov, union lov_layout_state *state) { @@ -388,75 +701,6 @@ static int lov_delete_empty(const struct lu_env *env, struct lov_object *lov, return 0; } -static void lov_subobject_kill(const struct lu_env *env, struct lov_object *lov, - struct lov_layout_raid0 *r0, - struct lovsub_object *los, int idx) -{ - struct cl_object *sub; - struct lu_site *site; - wait_queue_head_t *wq; - wait_queue_entry_t *waiter; - - LASSERT(r0->lo_sub[idx] == los); - - sub = lovsub2cl(los); - site = sub->co_lu.lo_dev->ld_site; - wq = lu_site_wq_from_fid(site, &sub->co_lu.lo_header->loh_fid); - - cl_object_kill(env, sub); - /* release a reference to the sub-object and ... */ - lu_object_ref_del(&sub->co_lu, "lov-parent", lov); - cl_object_put(env, sub); - - /* ... wait until it is actually destroyed---sub-object clears its - * ->lo_sub[] slot in lovsub_object_fini() - */ - if (r0->lo_sub[idx] == los) { - waiter = &lov_env_info(env)->lti_waiter; - init_waitqueue_entry(waiter, current); - add_wait_queue(wq, waiter); - set_current_state(TASK_UNINTERRUPTIBLE); - while (1) { - /* this wait-queue is signaled at the end of - * lu_object_free(). - */ - set_current_state(TASK_UNINTERRUPTIBLE); - spin_lock(&r0->lo_sub_lock); - if (r0->lo_sub[idx] == los) { - spin_unlock(&r0->lo_sub_lock); - schedule(); - } else { - spin_unlock(&r0->lo_sub_lock); - set_current_state(TASK_RUNNING); - break; - } - } - remove_wait_queue(wq, waiter); - } - LASSERT(!r0->lo_sub[idx]); -} - -static void lov_delete_raid0(const struct lu_env *env, struct lov_object *lov, - struct lov_layout_raid0 *r0) -{ - if (r0->lo_sub) { - int i; - - for (i = 0; i < r0->lo_nr; ++i) { - struct lovsub_object *los = r0->lo_sub[i]; - - if (los) { - cl_object_prune(env, &los->lso_cl); - /* - * If top-level object is to be evicted from - * the cache, so are its sub-objects. - */ - lov_subobject_kill(env, lov, r0, los, i); - } - } - } -} - static int lov_delete_composite(const struct lu_env *env, struct lov_object *lov, union lov_layout_state *state) @@ -469,7 +713,7 @@ static int lov_delete_composite(const struct lu_env *env, lov_layout_wait(env, lov); if (comp->lo_entries) lov_foreach_layout_entry(lov, entry) - lov_delete_raid0(env, lov, &entry->lle_raid0); + lov_delete_raid0(env, lov, entry); return 0; } @@ -480,15 +724,6 @@ static void lov_fini_empty(const struct lu_env *env, struct lov_object *lov, LASSERT(lov->lo_type == LLT_EMPTY || lov->lo_type == LLT_RELEASED); } -static void lov_fini_raid0(const struct lu_env *env, - struct lov_layout_raid0 *r0) -{ - if (r0->lo_sub) { - kvfree(r0->lo_sub); - r0->lo_sub = NULL; - } -} - static void lov_fini_composite(const struct lu_env *env, struct lov_object *lov, union lov_layout_state *state) @@ -499,7 +734,7 @@ static void lov_fini_composite(const struct lu_env *env, struct lov_layout_entry *entry; lov_foreach_layout_entry(lov, entry) - lov_fini_raid0(env, &entry->lle_raid0); + entry->lle_comp_ops->lco_fini(env, entry); kvfree(comp->lo_entries); comp->lo_entries = NULL; @@ -523,24 +758,6 @@ static int lov_print_empty(const struct lu_env *env, void *cookie, return 0; } -static int lov_print_raid0(const struct lu_env *env, void *cookie, - lu_printer_t p, struct lov_layout_raid0 *r0) -{ - int i; - - for (i = 0; i < r0->lo_nr; ++i) { - struct lu_object *sub; - - if (r0->lo_sub[i]) { - sub = lovsub2lu(r0->lo_sub[i]); - lu_object_print(env, cookie, p, sub); - } else { - (*p)(env, cookie, "sub %d absent\n", i); - } - } - return 0; -} - static int lov_print_composite(const struct lu_env *env, void *cookie, lu_printer_t p, const struct lu_object *o) { @@ -556,12 +773,15 @@ static int lov_print_composite(const struct lu_env *env, void *cookie, for (i = 0; i < lsm->lsm_entry_count; i++) { struct lov_stripe_md_entry *lse = lsm->lsm_entries[i]; + struct lov_layout_entry *lle = lov_entry(lov, i); - (*p)(env, cookie, DEXT ": { 0x%08X, %u, %u, %#x, %u, %u }\n", + (*p)(env, cookie, + DEXT ": { 0x%08X, %u, %#x, %u, %#x, %u, %u }\n", PEXT(&lse->lsme_extent), lse->lsme_magic, - lse->lsme_id, lse->lsme_layout_gen, lse->lsme_flags, - lse->lsme_stripe_count, lse->lsme_stripe_size); - lov_print_raid0(env, cookie, p, lov_r0(lov, i)); + lse->lsme_id, lse->lsme_pattern, lse->lsme_layout_gen, + lse->lsme_flags, lse->lsme_stripe_count, + lse->lsme_stripe_size); + lov_print_raid0(env, cookie, p, lle); } return 0; @@ -595,52 +815,6 @@ static int lov_attr_get_empty(const struct lu_env *env, struct cl_object *obj, return 0; } -static int lov_attr_get_raid0(const struct lu_env *env, struct lov_object *lov, - unsigned int index, struct lov_layout_raid0 *r0) -{ - struct lov_stripe_md *lsm = lov->lo_lsm; - struct ost_lvb *lvb = &lov_env_info(env)->lti_lvb; - struct cl_attr *attr = &r0->lo_attr; - int result = 0; - u64 kms = 0; - - if (r0->lo_attr_valid) - return 0; - - memset(lvb, 0, sizeof(*lvb)); - - /* XXX: timestamps can be negative by sanity:test_39m, - * how can it be? - */ - lvb->lvb_atime = LLONG_MIN; - lvb->lvb_ctime = LLONG_MIN; - lvb->lvb_mtime = LLONG_MIN; - - /* - * XXX that should be replaced with a loop over sub-objects, - * doing cl_object_attr_get() on them. But for now, let's - * reuse old lov code. - */ - - /* - * XXX take lsm spin-lock to keep lov_merge_lvb_kms() - * happy. It's not needed, because new code uses - * ->coh_attr_guard spin-lock to protect consistency of - * sub-object attributes. - */ - lov_stripe_lock(lsm); - result = lov_merge_lvb_kms(lsm, index, lvb, &kms); - lov_stripe_unlock(lsm); - if (result) - return result; - - cl_lvb2attr(attr, lvb); - attr->cat_kms = kms; - r0->lo_attr_valid = 1; - - return result; -} - static int lov_attr_get_composite(const struct lu_env *env, struct cl_object *obj, struct cl_attr *attr) @@ -653,19 +827,22 @@ static int lov_attr_get_composite(const struct lu_env *env, attr->cat_size = 0; attr->cat_blocks = 0; lov_foreach_layout_entry(lov, entry) { - struct lov_layout_raid0 *r0 = &entry->lle_raid0; - struct cl_attr *lov_attr = &r0->lo_attr; + struct cl_attr *lov_attr = NULL; /* PFL: This component has not been init-ed. */ if (!lsm_entry_inited(lov->lo_lsm, index)) break; - result = lov_attr_get_raid0(env, lov, index, r0); - if (result != 0) - break; + result = entry->lle_comp_ops->lco_getattr(env, lov, index, + entry, &lov_attr); + if (result < 0) + return result; index++; + if (!lov_attr) + continue; + /* merge results */ attr->cat_blocks += lov_attr->cat_blocks; if (attr->cat_size < lov_attr->cat_size) @@ -679,7 +856,7 @@ static int lov_attr_get_composite(const struct lu_env *env, if (attr->cat_mtime < lov_attr->cat_mtime) attr->cat_mtime = lov_attr->cat_mtime; } - return result; + return 0; } static const struct lov_layout_operations lov_dispatch[] = { @@ -1235,6 +1412,49 @@ struct fiemap_state { bool fs_enough; }; +static struct cl_object *lov_find_subobj(const struct lu_env *env, + struct lov_object *lov, + struct lov_stripe_md *lsm, + int index) +{ + struct lov_device *dev = lu2lov_dev(lov2lu(lov)->lo_dev); + struct lov_thread_info *lti = lov_env_info(env); + struct lu_fid *ofid = <i->lti_fid; + struct lov_oinfo *oinfo; + struct cl_device *subdev; + int entry = lov_comp_entry(index); + int stripe = lov_comp_stripe(index); + int ost_idx; + int rc; + struct cl_object *result; + + if (lov->lo_type != LLT_COMP) { + result = NULL; + goto out; + } + + if (entry >= lsm->lsm_entry_count || + stripe >= lsm->lsm_entries[entry]->lsme_stripe_count) { + result = NULL; + goto out; + } + + oinfo = lsm->lsm_entries[entry]->lsme_oinfo[stripe]; + ost_idx = oinfo->loi_ost_idx; + rc = ostid_to_fid(ofid, &oinfo->loi_oi, ost_idx); + if (rc != 0) { + result = NULL; + goto out; + } + + subdev = lovsub2cl_dev(dev->ld_target[ost_idx]); + result = lov_sub_find(env, subdev, ofid, NULL); +out: + if (!result) + result = ERR_PTR(-EINVAL); + return result; +} + static int fiemap_for_stripe(const struct lu_env *env, struct cl_object *obj, struct lov_stripe_md *lsm, struct fiemap *fiemap, size_t *buflen, struct ll_fiemap_info_key *fmkey, @@ -1457,6 +1677,12 @@ static int lov_object_fiemap(const struct lu_env *env, struct cl_object *obj, } } + /* No support for DOM layout yet. */ + if (lsme_is_dom(lsm->lsm_entries[0])) { + rc = -ENOTSUPP; + goto out_lsm; + } + if (lsm->lsm_is_released) { if (fiemap->fm_start < fmkey->lfik_oa.o_size) { /** diff --git a/fs/lustre/lov/lov_offset.c b/fs/lustre/lov/lov_offset.c index 26f5066..56a2d7b 100644 --- a/fs/lustre/lov/lov_offset.c +++ b/fs/lustre/lov/lov_offset.c @@ -43,6 +43,9 @@ static u64 stripe_width(struct lov_stripe_md *lsm, unsigned int index) LASSERT(index < lsm->lsm_entry_count); + if (lsme_is_dom(entry)) + return (loff_t)entry->lsme_stripe_size; + return entry->lsme_stripe_size * entry->lsme_stripe_count; } diff --git a/fs/lustre/mdc/mdc_request.c b/fs/lustre/mdc/mdc_request.c index 1103c15..eefaf44 100644 --- a/fs/lustre/mdc/mdc_request.c +++ b/fs/lustre/mdc/mdc_request.c @@ -2265,7 +2265,12 @@ static int mdc_set_info_async(const struct lu_env *env, return 0; } - CERROR("Unknown key %s\n", (char *)key); + /* TODO: these OSC-related keys are ignored for now */ + if (KEY_IS(KEY_CHECKSUM) || KEY_IS(KEY_CACHE_SET) || + KEY_IS(KEY_CACHE_LRU_SHRINK) || KEY_IS(KEY_GRANT_SHRINK)) + return 0; + + CERROR("%s: Unknown key %s\n", exp->exp_obd->obd_name, (char *)key); return -EINVAL; } diff --git a/fs/lustre/obdclass/obd_config.c b/fs/lustre/obdclass/obd_config.c index 73264fd..26b3e01 100644 --- a/fs/lustre/obdclass/obd_config.c +++ b/fs/lustre/obdclass/obd_config.c @@ -972,7 +972,6 @@ int class_process_config(struct lustre_cfg *lcfg) err = -EINVAL; goto out; } - switch (lcfg->lcfg_command) { case LCFG_SETUP: { err = class_setup(obd, lcfg); @@ -1020,6 +1019,41 @@ int class_process_config(struct lustre_cfg *lcfg) err = 0; goto out; } + /* Process config log ADD_MDC record twice to add MDC also to LOV + * for Data-on-MDT: + * + * add 0:lustre-clilmv 1:lustre-MDT0000_UUID 2:0 3:1 + * 4:lustre-MDT0000-mdc_UUID + */ + case LCFG_ADD_MDC: { + struct obd_device *lov_obd; + char *clilmv; + + err = obd_process_config(obd, sizeof(*lcfg), lcfg); + if (err) + goto out; + + /* make sure this is client LMV log entry */ + clilmv = strstr(lustre_cfg_string(lcfg, 0), "clilmv"); + if (!clilmv) + goto out; + + /* replace 'lmv' with 'lov' name to address LOV device and + * process llog record to add MDC there. + */ + clilmv[4] = 'o'; + lov_obd = class_name2obd(lustre_cfg_string(lcfg, 0)); + if (!lov_obd) { + err = -ENOENT; + CERROR("%s: Cannot find LOV by %s name, rc = %d\n", + obd->obd_name, lustre_cfg_string(lcfg, 0), err); + } else { + err = obd_process_config(lov_obd, sizeof(*lcfg), lcfg); + } + /* restore 'lmv' name */ + clilmv[4] = 'm'; + goto out; + } default: { err = obd_process_config(obd, sizeof(*lcfg), lcfg); goto out; diff --git a/fs/lustre/ptlrpc/wiretest.c b/fs/lustre/ptlrpc/wiretest.c index eb8bffe..2a38d1e 100644 --- a/fs/lustre/ptlrpc/wiretest.c +++ b/fs/lustre/ptlrpc/wiretest.c @@ -1479,8 +1479,8 @@ void lustre_assert_wire_constants(void) (unsigned int)LOV_PATTERN_RAID0); LASSERTF(LOV_PATTERN_RAID1 == 0x00000002UL, "found 0x%.8xUL\n", (unsigned int)LOV_PATTERN_RAID1); - LASSERTF(LOV_PATTERN_FIRST == 0x00000100UL, "found 0x%.8xUL\n", - (unsigned int)LOV_PATTERN_FIRST); + LASSERTF(LOV_PATTERN_MDT == 0x00000100UL, "found 0x%.8xUL\n", + (unsigned int)LOV_PATTERN_MDT); LASSERTF(LOV_PATTERN_CMOBD == 0x00000200UL, "found 0x%.8xUL\n", (unsigned int)LOV_PATTERN_CMOBD); diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index 17bad49..4a6ed5e 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -337,7 +337,7 @@ enum ll_lease_type { #define LOV_PATTERN_RAID0 0x001 #define LOV_PATTERN_RAID1 0x002 -#define LOV_PATTERN_FIRST 0x100 +#define LOV_PATTERN_MDT 0x100 #define LOV_PATTERN_CMOBD 0x200 #define LOV_PATTERN_F_MASK 0xffff0000 From patchwork Mon Sep 30 18:55:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167083 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1C37C16C1 for ; Mon, 30 Sep 2019 18:58:12 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 04AB0224D5 for ; Mon, 30 Sep 2019 18:58:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 04AB0224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id BD0995C3CA3; Mon, 30 Sep 2019 11:57:39 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 1BD335C3546 for ; Mon, 30 Sep 2019 11:57:12 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id ACA1A10055F8; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id AB9E9A9; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:03 -0400 Message-Id: <1569869810-23848-45-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 044/151] lustre: mdt: IO request handling in MDT X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mikhal Pershin , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mikhal Pershin Add methods to handle IO requests in MDT similar to OFD. Introduce MDS_INODEBITS_DOM bit for data on MDT, destinguish IO requests to MDT and OST and take appropriate lock in target code. WC-bug-id: https://jira.whamcloud.com/browse/LU-3285 Lustre-commit: 2bcc5ad0ed6a ("LU-3285 mdt: IO request handling in MDT") Signed-off-by: Mikhal Pershin Reviewed-on: https://review.whamcloud.com/28013 Reviewed-by: Jinshan Xiong Reviewed-by: Bobi Jam Signed-off-by: James Simmons --- fs/lustre/include/lustre_osc.h | 3 +++ fs/lustre/llite/llite_lib.c | 12 +++++++----- fs/lustre/mdc/mdc_request.c | 19 +++++++++++++++++-- fs/lustre/osc/osc_request.c | 3 ++- include/uapi/linux/lustre/lustre_idl.h | 5 +++-- 5 files changed, 32 insertions(+), 10 deletions(-) diff --git a/fs/lustre/include/lustre_osc.h b/fs/lustre/include/lustre_osc.h index e91fae2..f14ae85 100644 --- a/fs/lustre/include/lustre_osc.h +++ b/fs/lustre/include/lustre_osc.h @@ -512,6 +512,9 @@ int osc_attr_update(const struct lu_env *env, struct cl_object *obj, int osc_object_glimpse(const struct lu_env *env, const struct cl_object *obj, struct ost_lvb *lvb); +/* osc_request.c */ +void osc_init_grant(struct client_obd *cli, struct obd_connect_data *ocd); + /**************************************************************************** * * Accessors and type conversions. diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index 3896dc7..418f464 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -187,7 +187,7 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt) /* indicate MDT features supported by this client */ data->ocd_connect_flags = OBD_CONNECT_IBITS | OBD_CONNECT_NODEVOH | - OBD_CONNECT_ATTRFID | + OBD_CONNECT_ATTRFID | OBD_CONNECT_GRANT | OBD_CONNECT_VERSION | OBD_CONNECT_BRW_SIZE | OBD_CONNECT_CANCELSET | OBD_CONNECT_FID | OBD_CONNECT_AT | OBD_CONNECT_LOV_V3 | @@ -201,10 +201,10 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt) OBD_CONNECT_FLOCK_DEAD | OBD_CONNECT_DISP_STRIPE | OBD_CONNECT_LFSCK | OBD_CONNECT_OPEN_BY_FID | - OBD_CONNECT_DIR_STRIPE | - OBD_CONNECT_BULK_MBITS | - OBD_CONNECT_SUBTREE | - OBD_CONNECT_FLAGS2 | OBD_CONNECT_MULTIMODRPCS; + OBD_CONNECT_DIR_STRIPE | + OBD_CONNECT_BULK_MBITS | OBD_CONNECT_CKSUM | + OBD_CONNECT_SUBTREE | + OBD_CONNECT_FLAGS2 | OBD_CONNECT_MULTIMODRPCS; data->ocd_connect_flags2 = 0; @@ -215,6 +215,8 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt) OBD_CONNECT_LARGE_ACL; #endif + data->ocd_cksum_types = cksum_types_supported_client(); + if (OBD_FAIL_CHECK(OBD_FAIL_MDC_LIGHTWEIGHT)) /* flag mdc connection as lightweight, only used for test * purpose, use with care diff --git a/fs/lustre/mdc/mdc_request.c b/fs/lustre/mdc/mdc_request.c index eefaf44..96e0d0a 100644 --- a/fs/lustre/mdc/mdc_request.c +++ b/fs/lustre/mdc/mdc_request.c @@ -54,6 +54,7 @@ #include #include #include +#include #include "mdc_internal.h" @@ -2354,6 +2355,15 @@ static int mdc_import_event(struct obd_device *obd, struct obd_import *imp, LASSERT(imp->imp_obd == obd); switch (event) { + case IMP_EVENT_DISCON: { + struct client_obd *cli = &obd->u.cli; + + spin_lock(&cli->cl_loi_list_lock); + cli->cl_avail_grant = 0; + cli->cl_lost_grant = 0; + spin_unlock(&cli->cl_loi_list_lock); + break; + } case IMP_EVENT_INACTIVE: { struct client_obd *cli = &obd->u.cli; /* @@ -2384,10 +2394,15 @@ static int mdc_import_event(struct obd_device *obd, struct obd_import *imp, mdc_hsm_ct_reregister, (void *)imp); break; - case IMP_EVENT_OCD: + case IMP_EVENT_OCD: { + struct obd_connect_data *ocd = &imp->imp_connect_data; + + if (OCD_HAS_FLAG(ocd, GRANT)) + osc_init_grant(&obd->u.cli, ocd); + rc = obd_notify_observer(obd, obd, OBD_NOTIFY_OCD); break; - case IMP_EVENT_DISCON: + } case IMP_EVENT_DEACTIVATE: case IMP_EVENT_ACTIVATE: break; diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index b84856d..96d2c01 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -897,7 +897,7 @@ static int osc_del_shrink_grant(struct client_obd *client) TIMEOUT_GRANT); } -static void osc_init_grant(struct client_obd *cli, struct obd_connect_data *ocd) +void osc_init_grant(struct client_obd *cli, struct obd_connect_data *ocd) { /* * ocd_grant is the total grant amount we're expect to hold: if we've @@ -953,6 +953,7 @@ static void osc_init_grant(struct client_obd *cli, struct obd_connect_data *ocd) list_empty(&cli->cl_grant_shrink_list)) osc_add_shrink_grant(cli); } +EXPORT_SYMBOL(osc_init_grant); /* We assume that the reason this OSC got a short read is because it read * beyond the end of a stripe file; i.e. lustre is reading a sparse file diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index ec19d31..bcbe557 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -100,7 +100,7 @@ #define MDC_REPLY_PORTAL 10 /*#define MDC_BULK_PORTAL 11 */ #define MDS_REQUEST_PORTAL 12 -/*#define MDS_REPLY_PORTAL 13 */ +#define MDS_IO_PORTAL 13 #define MDS_BULK_PORTAL 14 #define LDLM_CB_REQUEST_PORTAL 15 #define LDLM_CB_REPLY_PORTAL 16 @@ -1429,8 +1429,9 @@ enum mdt_reint_cmd { */ #define MDS_INODELOCK_PERM 0x000010 #define MDS_INODELOCK_XATTR 0x000020 /* extended attributes */ +#define MDS_INODELOCK_DOM 0x000040 /* Data for data-on-mdt files */ -#define MDS_INODELOCK_MAXSHIFT 5 +#define MDS_INODELOCK_MAXSHIFT 6 /* This FULL lock is useful to take on unlink sort of operations */ #define MDS_INODELOCK_FULL ((1 << (MDS_INODELOCK_MAXSHIFT + 1)) - 1) From patchwork Mon Sep 30 18:55:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167145 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 33F7D912 for ; Mon, 30 Sep 2019 19:00:26 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1C5A7224D5 for ; Mon, 30 Sep 2019 19:00:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1C5A7224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id EC7EA5C3D39; Mon, 30 Sep 2019 11:58:39 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 719635C3546 for ; Mon, 30 Sep 2019 11:57:12 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id B068A10055F9; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id AEC85B4; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:04 -0400 Message-Id: <1569869810-23848-46-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 045/151] lustre: osc: common client setup/cleanup X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mikhal Pershin , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mikhal Pershin Make OSC code related to client setup/cleanup common for both OSB and MDC. Add new events in mdc_import_event(). WC-bug-id: https://jira.whamcloud.com/browse/LU-3285 Lustre-commit: a1bd0f62ee08 ("LU-3285 osc: common client setup/cleanup") Signed-off-by: Mikhal Pershin Reviewed-on: https://review.whamcloud.com/28014 Reviewed-by: Lai Siyao Reviewed-by: Jinshan Xiong Signed-off-by: James Simmons --- fs/lustre/include/lustre_osc.h | 30 +++++++++++++-- fs/lustre/include/obd.h | 4 ++ fs/lustre/ldlm/ldlm_lib.c | 3 ++ fs/lustre/mdc/mdc_request.c | 59 +++++++++++++---------------- fs/lustre/obdclass/genops.c | 22 +++++------ fs/lustre/osc/osc_cache.c | 17 ++------- fs/lustre/osc/osc_internal.h | 1 + fs/lustre/osc/osc_object.c | 2 +- fs/lustre/osc/osc_request.c | 85 +++++++++++++++++++++++++++--------------- 9 files changed, 130 insertions(+), 93 deletions(-) diff --git a/fs/lustre/include/lustre_osc.h b/fs/lustre/include/lustre_osc.h index f14ae85..5723261 100644 --- a/fs/lustre/include/lustre_osc.h +++ b/fs/lustre/include/lustre_osc.h @@ -51,6 +51,7 @@ #include #include +#include struct osc_quota_info { /* linkage for quota hash table */ @@ -479,9 +480,23 @@ int osc_cache_writeback_range(const struct lu_env *env, struct osc_object *obj, pgoff_t start, pgoff_t end, int hp, int discard); int osc_cache_wait_range(const struct lu_env *env, struct osc_object *obj, pgoff_t start, pgoff_t end); -void osc_io_unplug(const struct lu_env *env, struct client_obd *cli, - struct osc_object *osc); -int lru_queue_work(const struct lu_env *env, void *data); + +int __osc_io_unplug(const struct lu_env *env, struct client_obd *cli, + struct osc_object *osc, int async); + +static inline int osc_io_unplug_async(const struct lu_env *env, + struct client_obd *cli, + struct osc_object *osc) +{ + return __osc_io_unplug(env, cli, osc, 1); +} + +static inline void osc_io_unplug(const struct lu_env *env, + struct client_obd *cli, + struct osc_object *osc) +{ + (void)__osc_io_unplug(env, cli, osc, 0); +} void osc_object_set_contended(struct osc_object *obj); void osc_object_clear_contended(struct osc_object *obj); @@ -511,9 +526,18 @@ int osc_attr_update(const struct lu_env *env, struct cl_object *obj, const struct cl_attr *attr, unsigned int valid); int osc_object_glimpse(const struct lu_env *env, const struct cl_object *obj, struct ost_lvb *lvb); +int osc_object_invalidate(const struct lu_env *env, struct osc_object *osc); /* osc_request.c */ void osc_init_grant(struct client_obd *cli, struct obd_connect_data *ocd); +int osc_setup_common(struct obd_device *obd, struct lustre_cfg *lcfg); +int osc_precleanup_common(struct obd_device *obd); +int osc_cleanup_common(struct obd_device *obd); +int osc_set_info_async(const struct lu_env *env, struct obd_export *exp, + u32 keylen, void *key, u32 vallen, void *val, + struct ptlrpc_request_set *set); +int osc_ldlm_resource_invalidate(struct cfs_hash *hs, struct cfs_hash_bd *bd, + struct hlist_node *hnode, void *arg); /**************************************************************************** * diff --git a/fs/lustre/include/obd.h b/fs/lustre/include/obd.h index baa97a9..62a8ea9 100644 --- a/fs/lustre/include/obd.h +++ b/fs/lustre/include/obd.h @@ -316,6 +316,10 @@ struct client_obd { atomic_t cl_mgc_refcount; struct obd_export *cl_mgc_mgsexp; + /* in-flight control list and total RPCs counter */ + struct list_head cl_flight_waiters; + u32 cl_rpcs_in_flight; + /* checksumming for data sent over the network */ unsigned int cl_checksum:1, /* 0 = disabled, 1 = enabled */ cl_checksum_dump:1; /* same */ diff --git a/fs/lustre/ldlm/ldlm_lib.c b/fs/lustre/ldlm/ldlm_lib.c index 67f3297..3a7b3fb 100644 --- a/fs/lustre/ldlm/ldlm_lib.c +++ b/fs/lustre/ldlm/ldlm_lib.c @@ -351,6 +351,9 @@ int client_obd_setup(struct obd_device *obddev, struct lustre_cfg *lcfg) atomic_long_set(&cli->cl_unstable_count, 0); INIT_LIST_HEAD(&cli->cl_shrink_list); + INIT_LIST_HEAD(&cli->cl_flight_waiters); + cli->cl_rpcs_in_flight = 0; + init_waitqueue_head(&cli->cl_destroy_waitq); atomic_set(&cli->cl_destroy_in_flight, 0); /* Turn on checksumming by default. */ diff --git a/fs/lustre/mdc/mdc_request.c b/fs/lustre/mdc/mdc_request.c index 96e0d0a..85eb0ec 100644 --- a/fs/lustre/mdc/mdc_request.c +++ b/fs/lustre/mdc/mdc_request.c @@ -2242,14 +2242,6 @@ static int mdc_set_info_async(const struct lu_env *env, return do_set_info_async(imp, MDS_SET_INFO, LUSTRE_MDS_VERSION, keylen, key, vallen, val, set); } - if (KEY_IS(KEY_SPTLRPC_CONF)) { - sptlrpc_conf_client_adapt(exp->exp_obd); - return 0; - } - if (KEY_IS(KEY_FLUSH_CTX)) { - sptlrpc_import_flush_my_ctx(imp); - return 0; - } if (KEY_IS(KEY_CHANGELOG_CLEAR)) { rc = do_set_info_async(imp, MDS_SET_INFO, LUSTRE_MDS_VERSION, keylen, key, vallen, val, set); @@ -2266,13 +2258,7 @@ static int mdc_set_info_async(const struct lu_env *env, return 0; } - /* TODO: these OSC-related keys are ignored for now */ - if (KEY_IS(KEY_CHECKSUM) || KEY_IS(KEY_CACHE_SET) || - KEY_IS(KEY_CACHE_LRU_SHRINK) || KEY_IS(KEY_GRANT_SHRINK)) - return 0; - - CERROR("%s: Unknown key %s\n", exp->exp_obd->obd_name, (char *)key); - return -EINVAL; + return osc_set_info_async(env, exp, keylen, key, vallen, val, set); } static int mdc_get_info(const struct lu_env *env, struct obd_export *exp, @@ -2350,22 +2336,19 @@ static int mdc_fsync(struct obd_export *exp, const struct lu_fid *fid, static int mdc_import_event(struct obd_device *obd, struct obd_import *imp, enum obd_import_event event) { + struct client_obd *cli = &obd->u.cli; int rc = 0; LASSERT(imp->imp_obd == obd); switch (event) { - case IMP_EVENT_DISCON: { - struct client_obd *cli = &obd->u.cli; - + case IMP_EVENT_DISCON: spin_lock(&cli->cl_loi_list_lock); cli->cl_avail_grant = 0; cli->cl_lost_grant = 0; spin_unlock(&cli->cl_loi_list_lock); break; - } - case IMP_EVENT_INACTIVE: { - struct client_obd *cli = &obd->u.cli; + case IMP_EVENT_INACTIVE: /* * Flush current sequence to make client obtain new one * from server in case of disconnect/reconnect. @@ -2377,12 +2360,28 @@ static int mdc_import_event(struct obd_device *obd, struct obd_import *imp, rc = obd_notify_observer(obd, obd, OBD_NOTIFY_INACTIVE); break; - } case IMP_EVENT_INVALIDATE: { struct ldlm_namespace *ns = obd->obd_namespace; + struct lu_env *env; + u16 refcheck; ldlm_namespace_cleanup(ns, LDLM_FL_LOCAL_ONLY); + env = cl_env_get(&refcheck); + if (!IS_ERR(env)) { + /* Reset grants. All pages go to failing rpcs due to + * the invalid import. + */ + osc_io_unplug(env, cli, NULL); + + cfs_hash_for_each_nolock(ns->ns_rs_hash, + osc_ldlm_resource_invalidate, + env, 0); + cl_env_put(env, &refcheck); + ldlm_namespace_cleanup(ns, LDLM_FL_LOCAL_ONLY); + } else { + rc = PTR_ERR(env); + } break; } case IMP_EVENT_ACTIVE: @@ -2398,7 +2397,7 @@ static int mdc_import_event(struct obd_device *obd, struct obd_import *imp, struct obd_connect_data *ocd = &imp->imp_connect_data; if (OCD_HAS_FLAG(ocd, GRANT)) - osc_init_grant(&obd->u.cli, ocd); + osc_init_grant(cli, ocd); rc = obd_notify_observer(obd, obd, OBD_NOTIFY_OCD); break; @@ -2497,14 +2496,10 @@ int mdc_setup(struct obd_device *obd, struct lustre_cfg *cfg) { int rc; - rc = ptlrpcd_addref(); + rc = osc_setup_common(obd, cfg); if (rc < 0) return rc; - rc = client_obd_setup(obd, cfg); - if (rc) - goto err_ptlrpcd_decref; - rc = mdc_tunables_init(obd); if (rc) goto err_osc_cleanup; @@ -2536,8 +2531,6 @@ int mdc_setup(struct obd_device *obd, struct lustre_cfg *cfg) ptlrpc_lprocfs_unregister_obd(obd); err_osc_cleanup: client_obd_cleanup(obd); -err_ptlrpcd_decref: - ptlrpcd_decref(); return rc; } @@ -2565,6 +2558,8 @@ static int mdc_init_ea_size(struct obd_export *exp, u32 easize, u32 def_easize) static int mdc_precleanup(struct obd_device *obd) { + osc_precleanup_common(obd); + /* Failsafe, ok if racy */ if (atomic_read(&obd->obd_type->typ_refcnt) <= 1) libcfs_kkuc_group_rem(0, KUC_GRP_HSM); @@ -2581,9 +2576,7 @@ static int mdc_precleanup(struct obd_device *obd) static int mdc_cleanup(struct obd_device *obd) { - ptlrpcd_decref(); - - return client_obd_cleanup(obd); + return osc_cleanup_common(obd); } int mdc_process_config(struct obd_device *obd, u32 len, void *buf) diff --git a/fs/lustre/obdclass/genops.c b/fs/lustre/obdclass/genops.c index 4b528a7..68d267f 100644 --- a/fs/lustre/obdclass/genops.c +++ b/fs/lustre/obdclass/genops.c @@ -1291,14 +1291,14 @@ int obd_get_request_slot(struct client_obd *cli) int rc; spin_lock(&cli->cl_loi_list_lock); - if (cli->cl_r_in_flight < cli->cl_max_rpcs_in_flight) { - cli->cl_r_in_flight++; + if (cli->cl_rpcs_in_flight < cli->cl_max_rpcs_in_flight) { + cli->cl_rpcs_in_flight++; spin_unlock(&cli->cl_loi_list_lock); return 0; } init_waitqueue_head(&orsw.orsw_waitq); - list_add_tail(&orsw.orsw_entry, &cli->cl_loi_read_list); + list_add_tail(&orsw.orsw_entry, &cli->cl_flight_waiters); orsw.orsw_signaled = false; spin_unlock(&cli->cl_loi_list_lock); @@ -1314,7 +1314,7 @@ int obd_get_request_slot(struct client_obd *cli) if (rc) { if (!orsw.orsw_signaled) { if (list_empty(&orsw.orsw_entry)) - cli->cl_r_in_flight--; + cli->cl_rpcs_in_flight--; else list_del(&orsw.orsw_entry); } @@ -1336,16 +1336,16 @@ void obd_put_request_slot(struct client_obd *cli) struct obd_request_slot_waiter *orsw; spin_lock(&cli->cl_loi_list_lock); - cli->cl_r_in_flight--; + cli->cl_rpcs_in_flight--; /* If there is free slot, wakeup the first waiter. */ - if (!list_empty(&cli->cl_loi_read_list) && - likely(cli->cl_r_in_flight < cli->cl_max_rpcs_in_flight)) { - orsw = list_first_entry(&cli->cl_loi_read_list, + if (!list_empty(&cli->cl_flight_waiters) && + likely(cli->cl_rpcs_in_flight < cli->cl_max_rpcs_in_flight)) { + orsw = list_first_entry(&cli->cl_flight_waiters, struct obd_request_slot_waiter, orsw_entry); list_del_init(&orsw->orsw_entry); - cli->cl_r_in_flight++; + cli->cl_rpcs_in_flight++; wake_up(&orsw->orsw_waitq); } spin_unlock(&cli->cl_loi_list_lock); @@ -1395,14 +1395,14 @@ int obd_set_max_rpcs_in_flight(struct client_obd *cli, u32 max) /* We increase the max_rpcs_in_flight, then wakeup some waiters. */ for (i = 0; i < diff; i++) { - orsw = list_first_entry_or_null(&cli->cl_loi_read_list, + orsw = list_first_entry_or_null(&cli->cl_flight_waiters, struct obd_request_slot_waiter, orsw_entry); if (!orsw) break; list_del_init(&orsw->orsw_entry); - cli->cl_r_in_flight++; + cli->cl_rpcs_in_flight++; wake_up(&orsw->orsw_waitq); } spin_unlock(&cli->cl_loi_list_lock); diff --git a/fs/lustre/osc/osc_cache.c b/fs/lustre/osc/osc_cache.c index 1d06654..65fdbc1 100644 --- a/fs/lustre/osc/osc_cache.c +++ b/fs/lustre/osc/osc_cache.c @@ -2260,8 +2260,8 @@ static void osc_check_rpcs(const struct lu_env *env, struct client_obd *cli) spin_unlock(&cli->cl_loi_list_lock); } -static int __osc_io_unplug(const struct lu_env *env, struct client_obd *cli, - struct osc_object *osc, int async) +int __osc_io_unplug(const struct lu_env *env, struct client_obd *cli, + struct osc_object *osc, int async) { int rc = 0; @@ -2278,18 +2278,7 @@ static int __osc_io_unplug(const struct lu_env *env, struct client_obd *cli, return rc; } - -static int osc_io_unplug_async(const struct lu_env *env, - struct client_obd *cli, struct osc_object *osc) -{ - return __osc_io_unplug(env, cli, osc, 1); -} - -void osc_io_unplug(const struct lu_env *env, struct client_obd *cli, - struct osc_object *osc) -{ - (void)__osc_io_unplug(env, cli, osc, 0); -} +EXPORT_SYMBOL(__osc_io_unplug); int osc_prep_async_page(struct osc_object *osc, struct osc_page *ops, struct page *page, loff_t offset) diff --git a/fs/lustre/osc/osc_internal.h b/fs/lustre/osc/osc_internal.h index 72cf131..873556a 100644 --- a/fs/lustre/osc/osc_internal.h +++ b/fs/lustre/osc/osc_internal.h @@ -44,6 +44,7 @@ int osc_shrink_grant_to_target(struct client_obd *cli, u64 target_bytes); void osc_update_next_shrink(struct client_obd *cli); +int lru_queue_work(const struct lu_env *env, void *data); typedef int (*osc_enqueue_upcall_f)(void *cookie, struct lustre_handle *lockh, int rc); diff --git a/fs/lustre/osc/osc_object.c b/fs/lustre/osc/osc_object.c index 9ba3785..d34e6f2 100644 --- a/fs/lustre/osc/osc_object.c +++ b/fs/lustre/osc/osc_object.c @@ -462,5 +462,5 @@ int osc_object_invalidate(const struct lu_env *env, struct osc_object *osc) return 0; } - +EXPORT_SYMBOL(osc_object_invalidate); /** @} osc */ diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index 96d2c01..e804554 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -737,10 +737,6 @@ static void osc_update_grant(struct client_obd *cli, struct ost_body *body) } } -static int osc_set_info_async(const struct lu_env *env, struct obd_export *exp, - u32 keylen, void *key, u32 vallen, - void *val, struct ptlrpc_request_set *set); - static int osc_shrink_grant_interpret(const struct lu_env *env, struct ptlrpc_request *req, void *aa, int rc) @@ -2524,9 +2520,10 @@ static int osc_iocontrol(unsigned int cmd, struct obd_export *exp, int len, return err; } -static int osc_set_info_async(const struct lu_env *env, struct obd_export *exp, - u32 keylen, void *key, u32 vallen, - void *val, struct ptlrpc_request_set *set) +int osc_set_info_async(const struct lu_env *env, struct obd_export *exp, + u32 keylen, void *key, + u32 vallen, void *val, + struct ptlrpc_request_set *set) { struct ptlrpc_request *req; struct obd_device *obd = exp->exp_obd; @@ -2642,6 +2639,7 @@ static int osc_set_info_async(const struct lu_env *env, struct obd_export *exp, return 0; } +EXPORT_SYMBOL(osc_set_info_async); static int osc_reconnect(const struct lu_env *env, struct obd_export *exp, struct obd_device *obd, @@ -2702,9 +2700,9 @@ static int osc_disconnect(struct obd_export *exp) return rc; } -static int osc_ldlm_resource_invalidate(struct cfs_hash *hs, - struct cfs_hash_bd *bd, - struct hlist_node *hnode, void *arg) +int osc_ldlm_resource_invalidate(struct cfs_hash *hs, + struct cfs_hash_bd *bd, + struct hlist_node *hnode, void *arg) { struct ldlm_resource *res = cfs_hash_object(hs, hnode); struct osc_object *osc = NULL; @@ -2734,6 +2732,7 @@ static int osc_ldlm_resource_invalidate(struct cfs_hash *hs, return 0; } +EXPORT_SYMBOL(osc_ldlm_resource_invalidate); static int osc_import_event(struct obd_device *obd, struct obd_import *imp, @@ -2842,14 +2841,11 @@ static int brw_queue_work(const struct lu_env *env, void *data) return 0; } -int osc_setup(struct obd_device *obd, struct lustre_cfg *lcfg) +int osc_setup_common(struct obd_device *obd, struct lustre_cfg *lcfg) { struct client_obd *cli = &obd->u.cli; void *handler; int rc; - int adding; - int added; - int req_count; rc = ptlrpcd_addref(); if (rc) @@ -2862,7 +2858,7 @@ int osc_setup(struct obd_device *obd, struct lustre_cfg *lcfg) handler = ptlrpcd_alloc_work(cli->cl_import, brw_queue_work, cli); if (IS_ERR(handler)) { rc = PTR_ERR(handler); - goto out_client_setup; + goto out_ptlrpcd_work; } cli->cl_writeback_work = handler; @@ -2880,6 +2876,37 @@ int osc_setup(struct obd_device *obd, struct lustre_cfg *lcfg) cli->cl_grant_shrink_interval = GRANT_SHRINK_INTERVAL; + INIT_LIST_HEAD(&cli->cl_grant_shrink_list); + return 0; + +out_ptlrpcd_work: + if (cli->cl_writeback_work) { + ptlrpcd_destroy_work(cli->cl_writeback_work); + cli->cl_writeback_work = NULL; + } + if (cli->cl_lru_work) { + ptlrpcd_destroy_work(cli->cl_lru_work); + cli->cl_lru_work = NULL; + } + client_obd_cleanup(obd); +out_ptlrpcd: + ptlrpcd_decref(); + return rc; +} +EXPORT_SYMBOL(osc_setup_common); + +int osc_setup(struct obd_device *obd, struct lustre_cfg *lcfg) +{ + struct client_obd *cli = &obd->u.cli; + int adding; + int added; + int req_count; + int rc; + + rc = osc_setup_common(obd, lcfg); + if (rc < 0) + return rc; + rc = osc_tunables_init(obd); if (rc) goto out_quota; @@ -2910,23 +2937,10 @@ int osc_setup(struct obd_device *obd, struct lustre_cfg *lcfg) out_quota: osc_quota_cleanup(obd); -out_ptlrpcd_work: - if (cli->cl_writeback_work) { - ptlrpcd_destroy_work(cli->cl_writeback_work); - cli->cl_writeback_work = NULL; - } - if (cli->cl_lru_work) { - ptlrpcd_destroy_work(cli->cl_lru_work); - cli->cl_lru_work = NULL; - } -out_client_setup: - client_obd_cleanup(obd); -out_ptlrpcd: - ptlrpcd_decref(); return rc; } -static int osc_precleanup(struct obd_device *obd) +int osc_precleanup_common(struct obd_device *obd) { struct client_obd *cli = &obd->u.cli; @@ -2951,12 +2965,20 @@ static int osc_precleanup(struct obd_device *obd) } obd_cleanup_client_import(obd); + return 0; +} +EXPORT_SYMBOL(osc_precleanup_common); + +static int osc_precleanup(struct obd_device *obd) +{ + osc_precleanup_common(obd); + ptlrpc_lprocfs_unregister_obd(obd); lprocfs_obd_cleanup(obd); return 0; } -static int osc_cleanup(struct obd_device *obd) +int osc_cleanup_common(struct obd_device *obd) { struct client_obd *cli = &obd->u.cli; int rc; @@ -2984,6 +3006,7 @@ static int osc_cleanup(struct obd_device *obd) ptlrpcd_decref(); return rc; } +EXPORT_SYMBOL(osc_cleanup_common); int osc_process_config_base(struct obd_device *obd, struct lustre_cfg *lcfg) { @@ -3001,7 +3024,7 @@ static int osc_process_config(struct obd_device *obd, u32 len, void *buf) .owner = THIS_MODULE, .setup = osc_setup, .precleanup = osc_precleanup, - .cleanup = osc_cleanup, + .cleanup = osc_cleanup_common, .add_conn = client_import_add_conn, .del_conn = client_import_del_conn, .connect = client_connect_import, From patchwork Mon Sep 30 18:55:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167149 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D3B2316B1 for ; Mon, 30 Sep 2019 19:00:35 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BBDFF224D5 for ; Mon, 30 Sep 2019 19:00:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BBDFF224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id AE1155C4075; Mon, 30 Sep 2019 11:58:43 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id C84B75C3546 for ; Mon, 30 Sep 2019 11:57:12 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id B44F610055FA; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id B1E0CB5; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:05 -0400 Message-Id: <1569869810-23848-47-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 046/151] lustre: mdc: add IO methods to the MDC X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mikhal Pershin , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mikhal Pershin Add CL IO methods specific for MDC. Update and export related OSC functions to be used from MDC. WC-bug-id: https://jira.whamcloud.com/browse/LU-3285 Lustre-commit: 59bd024b7117 ("LU-3285 mdc: add IO methods to the MDC") Signed-off-by: Mikhal Pershin Reviewed-on: https://review.whamcloud.com/28015 Reviewed-by: Jinshan Xiong Reviewed-by: Bobi Jam Signed-off-by: James Simmons --- fs/lustre/include/lustre_osc.h | 39 +++++++++-- fs/lustre/mdc/mdc_dev.c | 155 ++++++++++++++++++++++++++++++++++++++--- fs/lustre/osc/osc_cache.c | 4 ++ fs/lustre/osc/osc_dev.c | 2 + fs/lustre/osc/osc_internal.h | 13 +++- fs/lustre/osc/osc_io.c | 76 +++++++++++--------- fs/lustre/osc/osc_page.c | 1 + fs/lustre/osc/osc_request.c | 25 ++++--- 8 files changed, 253 insertions(+), 62 deletions(-) diff --git a/fs/lustre/include/lustre_osc.h b/fs/lustre/include/lustre_osc.h index 5723261..290f3c9 100644 --- a/fs/lustre/include/lustre_osc.h +++ b/fs/lustre/include/lustre_osc.h @@ -447,18 +447,19 @@ struct osc_page { #define OSC_FLAGS (ASYNC_URGENT | ASYNC_READY) +/* osc_page.c */ int osc_page_init(const struct lu_env *env, struct cl_object *obj, struct cl_page *page, pgoff_t ind); void osc_index2policy(union ldlm_policy_data *policy, const struct cl_object *obj, pgoff_t start, pgoff_t end); -int osc_lvb_print(const struct lu_env *env, void *cookie, - lu_printer_t p, const struct ost_lvb *lvb); - void osc_lru_add_batch(struct client_obd *cli, struct list_head *list); void osc_page_submit(const struct lu_env *env, struct osc_page *opg, enum cl_req_type crt, int brw_flags); +int lru_queue_work(const struct lu_env *env, void *data); + +/* osc_cache.c */ int osc_set_async_flags(struct osc_object *obj, struct osc_page *opg, u32 async_flags); int osc_prep_async_page(struct osc_object *osc, struct osc_page *ops, @@ -480,7 +481,6 @@ int osc_cache_writeback_range(const struct lu_env *env, struct osc_object *obj, pgoff_t start, pgoff_t end, int hp, int discard); int osc_cache_wait_range(const struct lu_env *env, struct osc_object *obj, pgoff_t start, pgoff_t end); - int __osc_io_unplug(const struct lu_env *env, struct client_obd *cli, struct osc_object *osc, int async); @@ -539,6 +539,37 @@ int osc_set_info_async(const struct lu_env *env, struct obd_export *exp, int osc_ldlm_resource_invalidate(struct cfs_hash *hs, struct cfs_hash_bd *bd, struct hlist_node *hnode, void *arg); +int osc_punch_send(struct obd_export *exp, struct obdo *oa, + obd_enqueue_update_f upcall, void *cookie); + +/* osc_io.c */ +int osc_io_submit(const struct lu_env *env, const struct cl_io_slice *ios, + enum cl_req_type crt, struct cl_2queue *queue); +int osc_io_commit_async(const struct lu_env *env, + const struct cl_io_slice *ios, + struct cl_page_list *qin, int from, int to, + cl_commit_cbt cb); +int osc_io_iter_init(const struct lu_env *env, const struct cl_io_slice *ios); +void osc_io_iter_fini(const struct lu_env *env, + const struct cl_io_slice *ios); +int osc_io_write_iter_init(const struct lu_env *env, + const struct cl_io_slice *ios); +void osc_io_write_iter_fini(const struct lu_env *env, + const struct cl_io_slice *ios); +int osc_io_fault_start(const struct lu_env *env, const struct cl_io_slice *ios); +void osc_io_setattr_end(const struct lu_env *env, + const struct cl_io_slice *slice); +int osc_io_read_start(const struct lu_env *env, + const struct cl_io_slice *slice); +int osc_io_write_start(const struct lu_env *env, + const struct cl_io_slice *slice); +void osc_io_end(const struct lu_env *env, const struct cl_io_slice *slice); + +int osc_io_fsync_start(const struct lu_env *env, + const struct cl_io_slice *slice); +void osc_io_fsync_end(const struct lu_env *env, + const struct cl_io_slice *slice); + /**************************************************************************** * * Accessors and type conversions. diff --git a/fs/lustre/mdc/mdc_dev.c b/fs/lustre/mdc/mdc_dev.c index aa86d93..77e152e 100644 --- a/fs/lustre/mdc/mdc_dev.c +++ b/fs/lustre/mdc/mdc_dev.c @@ -38,23 +38,156 @@ #include "mdc_internal.h" -int mdc_page_init(const struct lu_env *env, struct cl_object *obj, - struct cl_page *page, pgoff_t index) -{ - return -ENOTSUPP; -} - int mdc_lock_init(const struct lu_env *env, struct cl_object *obj, struct cl_lock *lock, const struct cl_io *unused) { - return -ENOTSUPP; + return 0; +} + +/** + * IO operations. + * + * An implementation of cl_io_operations specific methods for MDC layer. + * + */ +static int mdc_async_upcall(void *a, int rc) +{ + struct osc_async_cbargs *args = a; + + args->opc_rc = rc; + complete(&args->opc_sync); + return 0; +} + +static int mdc_io_setattr_start(const struct lu_env *env, + const struct cl_io_slice *slice) +{ + struct cl_io *io = slice->cis_io; + struct osc_io *oio = cl2osc_io(env, slice); + struct cl_object *obj = slice->cis_obj; + struct lov_oinfo *loi = cl2osc(obj)->oo_oinfo; + struct cl_attr *attr = &osc_env_info(env)->oti_attr; + struct obdo *oa = &oio->oi_oa; + struct osc_async_cbargs *cbargs = &oio->oi_cbarg; + u64 size = io->u.ci_setattr.sa_attr.lvb_size; + unsigned int ia_valid = io->u.ci_setattr.sa_avalid; + enum op_xvalid ia_xvalid = io->u.ci_setattr.sa_xvalid; + int rc; + + /* silently ignore non-truncate setattr for Data-on-MDT object */ + if (cl_io_is_trunc(io)) { + /* truncate cache dirty pages first */ + rc = osc_cache_truncate_start(env, cl2osc(obj), size, + &oio->oi_trunc); + if (rc < 0) + return rc; + } + + if (oio->oi_lockless == 0) { + cl_object_attr_lock(obj); + rc = cl_object_attr_get(env, obj, attr); + if (rc == 0) { + struct ost_lvb *lvb = &io->u.ci_setattr.sa_attr; + unsigned int cl_valid = 0; + + if (ia_valid & ATTR_SIZE) { + attr->cat_size = attr->cat_kms = size; + cl_valid = (CAT_SIZE | CAT_KMS); + } + if (ia_valid & ATTR_MTIME_SET) { + attr->cat_mtime = lvb->lvb_mtime; + cl_valid |= CAT_MTIME; + } + if (ia_valid & ATTR_ATIME_SET) { + attr->cat_atime = lvb->lvb_atime; + cl_valid |= CAT_ATIME; + } + if (ia_xvalid & OP_XVALID_CTIME_SET) { + attr->cat_ctime = lvb->lvb_ctime; + cl_valid |= CAT_CTIME; + } + rc = cl_object_attr_update(env, obj, attr, cl_valid); + } + cl_object_attr_unlock(obj); + if (rc < 0) + return rc; + } + + if (!(ia_valid & ATTR_SIZE)) + return 0; + + memset(oa, 0, sizeof(*oa)); + oa->o_oi = loi->loi_oi; + oa->o_mtime = attr->cat_mtime; + oa->o_atime = attr->cat_atime; + oa->o_ctime = attr->cat_ctime; + + oa->o_size = size; + oa->o_blocks = OBD_OBJECT_EOF; + oa->o_valid = OBD_MD_FLID | OBD_MD_FLGROUP | OBD_MD_FLATIME | + OBD_MD_FLCTIME | OBD_MD_FLMTIME | OBD_MD_FLSIZE | + OBD_MD_FLBLOCKS; + if (oio->oi_lockless) { + oa->o_flags = OBD_FL_SRVLOCK; + oa->o_valid |= OBD_MD_FLFLAGS; + } + + init_completion(&cbargs->opc_sync); + + rc = osc_punch_send(osc_export(cl2osc(obj)), oa, + mdc_async_upcall, cbargs); + cbargs->opc_rpc_sent = rc == 0; + return rc; } -int mdc_io_init(const struct lu_env *env, - struct cl_object *obj, struct cl_io *io) +static struct cl_io_operations mdc_io_ops = { + .op = { + [CIT_READ] = { + .cio_iter_init = osc_io_iter_init, + .cio_iter_fini = osc_io_iter_fini, + .cio_start = osc_io_read_start, + }, + [CIT_WRITE] = { + .cio_iter_init = osc_io_write_iter_init, + .cio_iter_fini = osc_io_write_iter_fini, + .cio_start = osc_io_write_start, + .cio_end = osc_io_end, + }, + [CIT_SETATTR] = { + .cio_iter_init = osc_io_iter_init, + .cio_iter_fini = osc_io_iter_fini, + .cio_start = mdc_io_setattr_start, + .cio_end = osc_io_setattr_end, + }, + /* no support for data version so far */ + [CIT_DATA_VERSION] = { + .cio_start = NULL, + .cio_end = NULL, + }, + [CIT_FAULT] = { + .cio_iter_init = osc_io_iter_init, + .cio_iter_fini = osc_io_iter_fini, + .cio_start = osc_io_fault_start, + .cio_end = osc_io_end, + }, + [CIT_FSYNC] = { + .cio_start = osc_io_fsync_start, + .cio_end = osc_io_fsync_end, + }, + }, + .cio_submit = osc_io_submit, + .cio_commit_async = osc_io_commit_async, +}; + +int mdc_io_init(const struct lu_env *env, struct cl_object *obj, + struct cl_io *io) { - return -ENOTSUPP; + struct osc_io *oio = osc_env_io(env); + + CL_IO_SLICE_CLEAN(oio, oi_cl); + cl_io_slice_add(io, &oio->oi_cl, obj, &mdc_io_ops); + return 0; } /** @@ -78,7 +211,7 @@ static void mdc_req_attr_set(const struct lu_env *env, struct cl_object *obj, } static const struct cl_object_operations mdc_ops = { - .coo_page_init = mdc_page_init, + .coo_page_init = osc_page_init, .coo_lock_init = mdc_lock_init, .coo_io_init = mdc_io_init, .coo_attr_get = osc_attr_get, diff --git a/fs/lustre/osc/osc_cache.c b/fs/lustre/osc/osc_cache.c index 65fdbc1..cda8791 100644 --- a/fs/lustre/osc/osc_cache.c +++ b/fs/lustre/osc/osc_cache.c @@ -2309,6 +2309,7 @@ int osc_prep_async_page(struct osc_object *osc, struct osc_page *ops, return 0; } +EXPORT_SYMBOL(osc_prep_async_page); int osc_queue_async_io(const struct lu_env *env, struct cl_io *io, struct osc_page *ops) @@ -2798,6 +2799,7 @@ int osc_cache_truncate_start(const struct lu_env *env, struct osc_object *obj, return result; } +EXPORT_SYMBOL(osc_cache_truncate_start); /** * Called after osc_io_setattr_end to add oio->oi_trunc back to cache. @@ -2883,6 +2885,7 @@ int osc_cache_wait_range(const struct lu_env *env, struct osc_object *obj, OSC_IO_DEBUG(obj, "sync file range.\n"); return result; } +EXPORT_SYMBOL(osc_cache_wait_range); /** * Called to write out a range of osc object. @@ -3004,6 +3007,7 @@ int osc_cache_writeback_range(const struct lu_env *env, struct osc_object *obj, OSC_IO_DEBUG(obj, "pageout [%lu, %lu], %d.\n", start, end, result); return result; } +EXPORT_SYMBOL(osc_cache_writeback_range); /** * Returns a list of pages by a given [start, end] of @obj. diff --git a/fs/lustre/osc/osc_dev.c b/fs/lustre/osc/osc_dev.c index 7178e6b..fb1d31c 100644 --- a/fs/lustre/osc/osc_dev.c +++ b/fs/lustre/osc/osc_dev.c @@ -126,6 +126,7 @@ struct lu_context_key osc_key = { .lct_init = osc_key_init, .lct_fini = osc_key_fini }; +EXPORT_SYMBOL(osc_key); static void *osc_session_init(const struct lu_context *ctx, struct lu_context_key *key) @@ -151,6 +152,7 @@ struct lu_context_key osc_session_key = { .lct_init = osc_session_init, .lct_fini = osc_session_fini }; +EXPORT_SYMBOL(osc_session_key); /* type constructor/destructor: osc_type_{init,fini,start,stop}(). */ LU_TYPE_INIT_FINI(osc, &osc_key, &osc_session_key); diff --git a/fs/lustre/osc/osc_internal.h b/fs/lustre/osc/osc_internal.h index 873556a..f9ab069 100644 --- a/fs/lustre/osc/osc_internal.h +++ b/fs/lustre/osc/osc_internal.h @@ -65,9 +65,6 @@ int osc_match_base(struct obd_export *exp, struct ldlm_res_id *res_id, int osc_setattr_async(struct obd_export *exp, struct obdo *oa, obd_enqueue_update_f upcall, void *cookie, struct ptlrpc_request_set *rqset); -int osc_punch_base(struct obd_export *exp, struct obdo *oa, - obd_enqueue_update_f upcall, void *cookie, - struct ptlrpc_request_set *rqset); int osc_sync_base(struct osc_object *exp, struct obdo *oa, obd_enqueue_update_f upcall, void *cookie, struct ptlrpc_request_set *rqset); @@ -169,4 +166,14 @@ unsigned long osc_cache_shrink_count(struct shrinker *sk, unsigned long osc_cache_shrink_scan(struct shrinker *sk, struct shrink_control *sc); +static inline void osc_set_io_portal(struct ptlrpc_request *req) +{ + struct obd_import *imp = req->rq_import; + + /* Distinguish OSC from MDC here to use OST or MDS portal */ + if (OCD_HAS_FLAG(&imp->imp_connect_data, IBITS)) + req->rq_request_portal = MDS_IO_PORTAL; + else + req->rq_request_portal = OST_IO_PORTAL; +} #endif /* OSC_INTERNAL_H */ diff --git a/fs/lustre/osc/osc_io.c b/fs/lustre/osc/osc_io.c index f7169e9..176ebe8 100644 --- a/fs/lustre/osc/osc_io.c +++ b/fs/lustre/osc/osc_io.c @@ -104,9 +104,8 @@ static int osc_io_read_ahead(const struct lu_env *env, * or, if page is already submitted, changes osc flags through * osc_set_async_flags(). */ -static int osc_io_submit(const struct lu_env *env, - const struct cl_io_slice *ios, - enum cl_req_type crt, struct cl_2queue *queue) +int osc_io_submit(const struct lu_env *env, const struct cl_io_slice *ios, + enum cl_req_type crt, struct cl_2queue *queue) { struct cl_page *page; struct cl_page *tmp; @@ -212,6 +211,7 @@ static int osc_io_submit(const struct lu_env *env, CDEBUG(D_INFO, "%d/%d %d\n", qin->pl_nr, qout->pl_nr, result); return qout->pl_nr > 0 ? 0 : result; } +EXPORT_SYMBOL(osc_io_submit); /** * This is called when a page is accessed within file in a way that creates @@ -258,10 +258,10 @@ static void osc_page_touch_at(const struct lu_env *env, cl_object_attr_unlock(obj); } -static int osc_io_commit_async(const struct lu_env *env, - const struct cl_io_slice *ios, - struct cl_page_list *qin, int from, int to, - cl_commit_cbt cb) +int osc_io_commit_async(const struct lu_env *env, + const struct cl_io_slice *ios, + struct cl_page_list *qin, int from, int to, + cl_commit_cbt cb) { struct cl_io *io = ios->cis_io; struct osc_io *oio = cl2osc_io(env, ios); @@ -331,9 +331,9 @@ static int osc_io_commit_async(const struct lu_env *env, CDEBUG(D_INFO, "%d %d\n", qin->pl_nr, result); return result; } +EXPORT_SYMBOL(osc_io_commit_async); -static int osc_io_iter_init(const struct lu_env *env, - const struct cl_io_slice *ios) +int osc_io_iter_init(const struct lu_env *env, const struct cl_io_slice *ios) { struct osc_object *osc = cl2osc(ios->cis_obj); struct obd_import *imp = osc_cli(osc)->cl_import; @@ -351,9 +351,10 @@ static int osc_io_iter_init(const struct lu_env *env, return rc; } +EXPORT_SYMBOL(osc_io_iter_init); -static int osc_io_write_iter_init(const struct lu_env *env, - const struct cl_io_slice *ios) +int osc_io_write_iter_init(const struct lu_env *env, + const struct cl_io_slice *ios) { struct cl_io *io = ios->cis_io; struct osc_io *oio = osc_env_io(env); @@ -371,9 +372,10 @@ static int osc_io_write_iter_init(const struct lu_env *env, return osc_io_iter_init(env, ios); } +EXPORT_SYMBOL(osc_io_write_iter_init); -static void osc_io_iter_fini(const struct lu_env *env, - const struct cl_io_slice *ios) +void osc_io_iter_fini(const struct lu_env *env, + const struct cl_io_slice *ios) { struct osc_io *oio = osc_env_io(env); @@ -386,9 +388,10 @@ static void osc_io_iter_fini(const struct lu_env *env, wake_up_all(&osc->oo_io_waitq); } } +EXPORT_SYMBOL(osc_io_iter_fini); -static void osc_io_write_iter_fini(const struct lu_env *env, - const struct cl_io_slice *ios) +void osc_io_write_iter_fini(const struct lu_env *env, + const struct cl_io_slice *ios) { struct osc_io *oio = osc_env_io(env); struct osc_object *osc = cl2osc(ios->cis_obj); @@ -401,9 +404,9 @@ static void osc_io_write_iter_fini(const struct lu_env *env, osc_io_iter_fini(env, ios); } +EXPORT_SYMBOL(osc_io_write_iter_fini); -static int osc_io_fault_start(const struct lu_env *env, - const struct cl_io_slice *ios) +int osc_io_fault_start(const struct lu_env *env, const struct cl_io_slice *ios) { struct cl_io *io; struct cl_fault_io *fio; @@ -422,6 +425,7 @@ static int osc_io_fault_start(const struct lu_env *env, fio->ft_index, fio->ft_nob); return 0; } +EXPORT_SYMBOL(osc_io_fault_start); static int osc_async_upcall(void *a, int rc) { @@ -563,9 +567,8 @@ static int osc_io_setattr_start(const struct lu_env *env, init_completion(&cbargs->opc_sync); if (ia_avalid & ATTR_SIZE) - result = osc_punch_base(osc_export(cl2osc(obj)), - oa, osc_async_upcall, - cbargs, PTLRPCD_SET); + result = osc_punch_send(osc_export(cl2osc(obj)), + oa, osc_async_upcall, cbargs); else result = osc_setattr_async(osc_export(cl2osc(obj)), oa, osc_async_upcall, @@ -575,8 +578,8 @@ static int osc_io_setattr_start(const struct lu_env *env, return result; } -static void osc_io_setattr_end(const struct lu_env *env, - const struct cl_io_slice *slice) +void osc_io_setattr_end(const struct lu_env *env, + const struct cl_io_slice *slice) { struct cl_io *io = slice->cis_io; struct osc_io *oio = cl2osc_io(env, slice); @@ -608,6 +611,7 @@ static void osc_io_setattr_end(const struct lu_env *env, oio->oi_trunc = NULL; } } +EXPORT_SYMBOL(osc_io_setattr_end); struct osc_data_version_args { struct osc_io *dva_oio; @@ -710,10 +714,10 @@ static void osc_io_data_version_end(const struct lu_env *env, } } -static int osc_io_read_start(const struct lu_env *env, - const struct cl_io_slice *slice) +int osc_io_read_start(const struct lu_env *env, + const struct cl_io_slice *slice) { - struct cl_object *obj = slice->cis_obj; + struct cl_object *obj = slice->cis_obj; struct cl_attr *attr = &osc_env_info(env)->oti_attr; int rc = 0; @@ -725,11 +729,12 @@ static int osc_io_read_start(const struct lu_env *env, } return rc; } +EXPORT_SYMBOL(osc_io_read_start); -static int osc_io_write_start(const struct lu_env *env, - const struct cl_io_slice *slice) +int osc_io_write_start(const struct lu_env *env, + const struct cl_io_slice *slice) { - struct cl_object *obj = slice->cis_obj; + struct cl_object *obj = slice->cis_obj; struct cl_attr *attr = &osc_env_info(env)->oti_attr; int rc = 0; @@ -742,6 +747,7 @@ static int osc_io_write_start(const struct lu_env *env, return rc; } +EXPORT_SYMBOL(osc_io_write_start); static int osc_fsync_ost(const struct lu_env *env, struct osc_object *obj, struct cl_fsync_io *fio) @@ -769,8 +775,8 @@ static int osc_fsync_ost(const struct lu_env *env, struct osc_object *obj, return rc; } -static int osc_io_fsync_start(const struct lu_env *env, - const struct cl_io_slice *slice) +int osc_io_fsync_start(const struct lu_env *env, + const struct cl_io_slice *slice) { struct cl_io *io = slice->cis_io; struct cl_fsync_io *fio = &io->u.ci_fsync; @@ -808,9 +814,10 @@ static int osc_io_fsync_start(const struct lu_env *env, return result; } +EXPORT_SYMBOL(osc_io_fsync_start); -static void osc_io_fsync_end(const struct lu_env *env, - const struct cl_io_slice *slice) +void osc_io_fsync_end(const struct lu_env *env, + const struct cl_io_slice *slice) { struct cl_fsync_io *fio = &slice->cis_io->u.ci_fsync; struct cl_object *obj = slice->cis_obj; @@ -830,6 +837,7 @@ static void osc_io_fsync_end(const struct lu_env *env, } slice->cis_io->ci_result = result; } +EXPORT_SYMBOL(osc_io_fsync_end); static int osc_io_ladvise_start(const struct lu_env *env, const struct cl_io_slice *slice) @@ -905,8 +913,7 @@ static void osc_io_ladvise_end(const struct lu_env *env, slice->cis_io->ci_result = result; } -static void osc_io_end(const struct lu_env *env, - const struct cl_io_slice *slice) +void osc_io_end(const struct lu_env *env, const struct cl_io_slice *slice) { struct osc_io *oio = cl2osc_io(env, slice); @@ -915,6 +922,7 @@ static void osc_io_end(const struct lu_env *env, oio->oi_active = NULL; } } +EXPORT_SYMBOL(osc_io_end); static const struct cl_io_operations osc_io_ops = { .op = { diff --git a/fs/lustre/osc/osc_page.c b/fs/lustre/osc/osc_page.c index 3576b34..45b18f1 100644 --- a/fs/lustre/osc/osc_page.c +++ b/fs/lustre/osc/osc_page.c @@ -276,6 +276,7 @@ int osc_page_init(const struct lu_env *env, struct cl_object *obj, return result; } +EXPORT_SYMBOL(osc_page_init); /** * Helper function called by osc_io_submit() for every page in an immediate diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index e804554..275bd12 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -410,30 +410,32 @@ static int osc_create(const struct lu_env *env, struct obd_export *exp, return rc; } -int osc_punch_base(struct obd_export *exp, struct obdo *oa, - obd_enqueue_update_f upcall, void *cookie, - struct ptlrpc_request_set *rqset) +int osc_punch_send(struct obd_export *exp, struct obdo *oa, + obd_enqueue_update_f upcall, void *cookie) { struct ptlrpc_request *req; struct osc_setattr_args *sa; + struct obd_import *imp = class_exp2cliimp(exp); struct ost_body *body; int rc; - req = ptlrpc_request_alloc(class_exp2cliimp(exp), &RQF_OST_PUNCH); + req = ptlrpc_request_alloc(imp, &RQF_OST_PUNCH); if (!req) return -ENOMEM; rc = ptlrpc_request_pack(req, LUSTRE_OST_VERSION, OST_PUNCH); - if (rc) { + if (rc < 0) { ptlrpc_request_free(req); return rc; } - req->rq_request_portal = OST_IO_PORTAL; /* bug 7198 */ + + osc_set_io_portal(req); + ptlrpc_at_set_req_timeout(req); body = req_capsule_client_get(&req->rq_pill, &RMF_OST_BODY); - LASSERT(body); - lustre_set_wire_obdo(&req->rq_import->imp_connect_data, &body->oa, + + lustre_set_wire_obdo(&imp->imp_connect_data, &body->oa, oa); ptlrpc_request_set_replen(req); @@ -444,10 +446,12 @@ int osc_punch_base(struct obd_export *exp, struct obdo *oa, sa->sa_oa = oa; sa->sa_upcall = upcall; sa->sa_cookie = cookie; - ptlrpc_set_add_req(rqset, req); + + ptlrpcd_add_req(req); return 0; } +EXPORT_SYMBOL(osc_punch_send); static int osc_sync_interpret(const struct lu_env *env, struct ptlrpc_request *req, @@ -1157,7 +1161,8 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli, ptlrpc_request_free(req); return rc; } - req->rq_request_portal = OST_IO_PORTAL; /* bug 7198 */ + osc_set_io_portal(req); + ptlrpc_at_set_req_timeout(req); /* ask ptlrpc not to resend on EINPROGRESS since BRWs have their own * retry logic From patchwork Mon Sep 30 18:55:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167159 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 62183912 for ; Mon, 30 Sep 2019 19:00:59 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 49EBA224EF for ; Mon, 30 Sep 2019 19:00:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 49EBA224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id D06235C3DEF; Mon, 30 Sep 2019 11:58:52 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 2946D5C3546 for ; Mon, 30 Sep 2019 11:57:13 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id B746910055FB; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id B5088BD; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:06 -0400 Message-Id: <1569869810-23848-48-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 047/151] lustre: lvbo: pass lock as parameter to lvbo_update() X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mikhal Pershin , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mikhal Pershin LVBO lvbo_update call was using ldlm_resource as parameter, the ldlm_lock parameter is added to recognize DoM resource in MDT LVBO functions and perform DoM specific actions. WC-bug-id: https://jira.whamcloud.com/browse/LU-3285 Lustre-commit: 978428ed0476 ("LU-3285 lvbo: pass lock as parameter to lvbo_update()") Signed-off-by: Mikhal Pershin Reviewed-on: https://review.whamcloud.com/28016 Reviewed-by: Jinshan Xiong Reviewed-by: Lai Siyao Signed-off-by: James Simmons --- fs/lustre/include/lustre_dlm.h | 26 ++++++++++++++++---------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/fs/lustre/include/lustre_dlm.h b/fs/lustre/include/lustre_dlm.h index b13e0cc..ce287b7 100644 --- a/fs/lustre/include/lustre_dlm.h +++ b/fs/lustre/include/lustre_dlm.h @@ -279,9 +279,8 @@ struct ldlm_pool { */ struct ldlm_valblock_ops { int (*lvbo_init)(struct ldlm_resource *res); - int (*lvbo_update)(struct ldlm_resource *res, - struct ptlrpc_request *r, - int increase); + int (*lvbo_update)(struct ldlm_resource *res, struct ldlm_lock *lock, + struct ptlrpc_request *r, int increase); int (*lvbo_free)(struct ldlm_resource *res); /* Return size of lvb data appropriate RPC size can be reserved */ int (*lvbo_size)(struct ldlm_lock *lock); @@ -1094,17 +1093,24 @@ static inline struct ldlm_lock *ldlm_handle2lock(const struct lustre_handle *h) * Update Lock Value Block Operations (LVBO) on a resource taking into account * data from request @r */ -static inline int ldlm_res_lvbo_update(struct ldlm_resource *res, - struct ptlrpc_request *r, int increase) +static inline int ldlm_lvbo_update(struct ldlm_resource *res, + struct ldlm_lock *lock, + struct ptlrpc_request *req, int increase) { - if (ldlm_res_to_ns(res)->ns_lvbo && - ldlm_res_to_ns(res)->ns_lvbo->lvbo_update) { - return ldlm_res_to_ns(res)->ns_lvbo->lvbo_update(res, r, - increase); - } + struct ldlm_namespace *ns = ldlm_res_to_ns(res); + + if (ns->ns_lvbo && ns->ns_lvbo->lvbo_update) + return ns->ns_lvbo->lvbo_update(res, lock, req, increase); + return 0; } +static inline int ldlm_res_lvbo_update(struct ldlm_resource *res, + struct ptlrpc_request *req, int increase) +{ + return ldlm_lvbo_update(res, NULL, req, increase); +} + int ldlm_error2errno(enum ldlm_error error); #if LUSTRE_TRACKS_LOCK_EXP_REFS From patchwork Mon Sep 30 18:55:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167081 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E72CA16B1 for ; Mon, 30 Sep 2019 18:58:11 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CF4EF224D5 for ; Mon, 30 Sep 2019 18:58:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CF4EF224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 92B155C3CA0; Mon, 30 Sep 2019 11:57:39 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 6B3645C3427 for ; Mon, 30 Sep 2019 11:57:13 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id BAB2B10055FC; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id B9260A9; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:07 -0400 Message-Id: <1569869810-23848-49-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 048/151] lustre: mds: add IO locking to the MDC and MDT X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mikhal Pershin , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mikhal Pershin - introduce new DOM inodebit for Data-on-MDT files. - add IO lock and glimpse handling at MDT along with needed LVB updates for it. - a MDC is updated to exclude DOM bit from ELC and to handle LVB changes due to glimpse on MDT. - add CLIO locking at MDC, it uses IBITS lock to protect data at MDT and a MDC handles such locks to convert them into proper CLIO locks. WC-bug-id: https://jira.whamcloud.com/browse/LU-3285 Lustre-commit: 4f35c341f22b ("LU-3285 mds: add IO locking to the MDC and MDT") Signed-off-by: Mikhal Pershin Reviewed-on: https://review.whamcloud.com/28018 Lustre-commit: 138b3eee3b61 ("LU-3285 osc: remove wrongly applied assertion.") Signed-off-by: Mikhal Pershin Reviewed-on: https://review.whamcloud.com/29811 Reviewed-by: Jinshan Xiong Reviewed-by: Bobi Jam eviewed-by: Andreas Dilger Signed-off-by: James Simmons --- fs/lustre/include/lustre_dlm.h | 11 +- fs/lustre/include/lustre_dlm_flags.h | 2 +- fs/lustre/include/lustre_osc.h | 147 ++++- fs/lustre/ldlm/ldlm_internal.h | 1 + fs/lustre/ldlm/ldlm_request.c | 4 +- fs/lustre/llite/file.c | 5 +- fs/lustre/llite/llite_lib.c | 1 + fs/lustre/lmv/lmv_obd.c | 6 +- fs/lustre/mdc/mdc_dev.c | 960 ++++++++++++++++++++++++++++++++- fs/lustre/mdc/mdc_internal.h | 2 + fs/lustre/mdc/mdc_reint.c | 5 +- fs/lustre/osc/osc_cache.c | 12 +- fs/lustre/osc/osc_internal.h | 30 +- fs/lustre/osc/osc_io.c | 3 +- fs/lustre/osc/osc_lock.c | 122 ++--- fs/lustre/osc/osc_object.c | 31 +- fs/lustre/osc/osc_request.c | 25 +- include/uapi/linux/lustre/lustre_idl.h | 4 + 18 files changed, 1201 insertions(+), 170 deletions(-) diff --git a/fs/lustre/include/lustre_dlm.h b/fs/lustre/include/lustre_dlm.h index ce287b7..8f92225 100644 --- a/fs/lustre/include/lustre_dlm.h +++ b/fs/lustre/include/lustre_dlm.h @@ -723,7 +723,10 @@ struct ldlm_lock { /** Private storage for lock user. Opaque to LDLM. */ void *l_ast_data; - + /* Separate ost_lvb used mostly by Data-on-MDT for now. + * It is introduced to don't mix with layout lock data. + */ + struct ost_lvb l_ost_lvb; /* * Server-side-only members. */ @@ -870,6 +873,12 @@ static inline bool ldlm_has_layout(struct ldlm_lock *lock) lock->l_policy_data.l_inodebits.bits & MDS_INODELOCK_LAYOUT; } +static inline bool ldlm_has_dom(struct ldlm_lock *lock) +{ + return lock->l_resource->lr_type == LDLM_IBITS && + lock->l_policy_data.l_inodebits.bits & MDS_INODELOCK_DOM; +} + static inline char * ldlm_ns_name(struct ldlm_namespace *ns) { diff --git a/fs/lustre/include/lustre_dlm_flags.h b/fs/lustre/include/lustre_dlm_flags.h index 87cda36..2413309 100644 --- a/fs/lustre/include/lustre_dlm_flags.h +++ b/fs/lustre/include/lustre_dlm_flags.h @@ -393,7 +393,7 @@ /** l_flags bits marked as "ast" bits */ #define LDLM_FL_AST_MASK (LDLM_FL_FLOCK_DEADLOCK |\ - LDLM_FL_AST_DISCARD_DATA) + LDLM_FL_DISCARD_DATA) /** l_flags bits marked as "blocked" bits */ #define LDLM_FL_BLOCKED_MASK (LDLM_FL_BLOCK_GRANTED |\ diff --git a/fs/lustre/include/lustre_osc.h b/fs/lustre/include/lustre_osc.h index 290f3c9..ecca719 100644 --- a/fs/lustre/include/lustre_osc.h +++ b/fs/lustre/include/lustre_osc.h @@ -190,6 +190,73 @@ struct osc_thread_info { struct lu_buf oti_ladvise_buf; }; +static inline u64 osc_enq2ldlm_flags(u32 enqflags) +{ + u64 result = 0; + + CDEBUG(D_DLMTRACE, "flags: %x\n", enqflags); + + LASSERT((enqflags & ~CEF_MASK) == 0); + + if (enqflags & CEF_NONBLOCK) + result |= LDLM_FL_BLOCK_NOWAIT; + if (enqflags & CEF_GLIMPSE) + result |= LDLM_FL_HAS_INTENT; + if (enqflags & CEF_DISCARD_DATA) + result |= LDLM_FL_AST_DISCARD_DATA; + if (enqflags & CEF_PEEK) + result |= LDLM_FL_TEST_LOCK; + if (enqflags & CEF_LOCK_MATCH) + result |= LDLM_FL_MATCH_LOCK; + if (enqflags & CEF_LOCK_NO_EXPAND) + result |= LDLM_FL_NO_EXPANSION; + if (enqflags & CEF_SPECULATIVE) + result |= LDLM_FL_SPECULATIVE; + return result; +} + +typedef int (*osc_enqueue_upcall_f)(void *cookie, struct lustre_handle *lockh, + int rc); + +struct osc_enqueue_args { + struct obd_export *oa_exp; + enum ldlm_type oa_type; + enum ldlm_mode oa_mode; + u64 *oa_flags; + osc_enqueue_upcall_f oa_upcall; + void *oa_cookie; + struct ost_lvb *oa_lvb; + struct lustre_handle oa_lockh; + bool oa_speculative; +}; + +/** + * Bit flags for osc_dlm_lock_at_pageoff(). + */ +enum osc_dap_flags { + /** + * Just check if the desired lock exists, it won't hold reference + * count on lock. + */ + OSC_DAP_FL_TEST_LOCK = BIT(0), + /** + * Return the lock even if it is being canceled. + */ + OSC_DAP_FL_CANCELING = BIT(1), +}; + +/* + * The set of operations which are different for MDC and OSC objects + */ +struct osc_object_operations { + void (*oto_build_res_name)(struct osc_object *osc, + struct ldlm_res_id *resname); + struct ldlm_lock* (*oto_dlmlock_at_pgoff)(const struct lu_env *env, + struct osc_object *obj, + pgoff_t index, + enum osc_dap_flags dap_flags); +}; + struct osc_object { struct cl_object oo_cl; struct lov_oinfo *oo_oinfo; @@ -243,9 +310,24 @@ struct osc_object { atomic_t oo_nr_ios; wait_queue_head_t oo_io_waitq; + const struct osc_object_operations *oo_obj_ops; bool oo_initialized; }; +static inline void osc_build_res_name(struct osc_object *osc, + struct ldlm_res_id *resname) +{ + return osc->oo_obj_ops->oto_build_res_name(osc, resname); +} + +static inline struct ldlm_lock *osc_dlmlock_at_pgoff(const struct lu_env *env, + struct osc_object *obj, + pgoff_t index, + enum osc_dap_flags flags) +{ + return obj->oo_obj_ops->oto_dlmlock_at_pgoff(env, obj, index, flags); +} + static inline void osc_object_lock(struct osc_object *obj) { spin_lock(&obj->oo_lock); @@ -264,6 +346,18 @@ static inline void osc_object_unlock(struct osc_object *obj) #define assert_osc_object_is_locked(obj) \ assert_spin_locked(&obj->oo_lock) +static inline void osc_object_set_contended(struct osc_object *obj) +{ + obj->oo_contention_time = ktime_get(); + /* mb(); */ + obj->oo_contended = 1; +} + +static inline void osc_object_clear_contended(struct osc_object *obj) +{ + obj->oo_contended = 0; +} + /* * Lock "micro-states" for osc layer. */ @@ -340,7 +434,8 @@ struct osc_lock { enum osc_lock_state ols_state; /* lock value block */ struct ost_lvb ols_lvb; - + /** Lockless operations to be used by lockless lock */ + const struct cl_lock_operations *ols_lockless_ops; /* * true, if ldlm_lock_addref() was called against * osc_lock::ols_lock. This is used for sanity checking. @@ -393,6 +488,11 @@ struct osc_lock { ols_speculative:1; }; +static inline int osc_lock_is_lockless(const struct osc_lock *ols) +{ + return (ols->ols_cl.cls_ops == ols->ols_lockless_ops); +} + /* * Page state private for osc layer. */ @@ -498,11 +598,14 @@ static inline void osc_io_unplug(const struct lu_env *env, (void)__osc_io_unplug(env, cli, osc, 0); } -void osc_object_set_contended(struct osc_object *obj); -void osc_object_clear_contended(struct osc_object *obj); -int osc_object_is_contended(struct osc_object *obj); +typedef bool (*osc_page_gang_cbt)(const struct lu_env *, struct cl_io *, + struct osc_page *, void *); +bool osc_page_gang_lookup(const struct lu_env *env, struct cl_io *io, + struct osc_object *osc, pgoff_t start, pgoff_t end, + osc_page_gang_cbt cb, void *cbdata); -int osc_lock_is_lockless(const struct osc_lock *olck); +bool osc_discard_cb(const struct lu_env *env, struct cl_io *io, + struct osc_page *ops, void *cbdata); /* osc_dev.c */ int osc_device_init(const struct lu_env *env, struct lu_device *d, @@ -527,6 +630,10 @@ int osc_attr_update(const struct lu_env *env, struct cl_object *obj, int osc_object_glimpse(const struct lu_env *env, const struct cl_object *obj, struct ost_lvb *lvb); int osc_object_invalidate(const struct lu_env *env, struct osc_object *osc); +int osc_object_is_contended(struct osc_object *obj); +int osc_object_find_cbdata(const struct lu_env *env, struct cl_object *obj, + ldlm_iterator_t iter, void *data); +int osc_object_prune(const struct lu_env *env, struct cl_object *obj); /* osc_request.c */ void osc_init_grant(struct client_obd *cli, struct obd_connect_data *ocd); @@ -564,11 +671,27 @@ int osc_io_read_start(const struct lu_env *env, int osc_io_write_start(const struct lu_env *env, const struct cl_io_slice *slice); void osc_io_end(const struct lu_env *env, const struct cl_io_slice *slice); - int osc_io_fsync_start(const struct lu_env *env, const struct cl_io_slice *slice); void osc_io_fsync_end(const struct lu_env *env, const struct cl_io_slice *slice); +void osc_read_ahead_release(const struct lu_env *env, void *cbdata); + +/* osc_lock.c */ +void osc_lock_to_lockless(const struct lu_env *env, struct osc_lock *ols, + int force); +void osc_lock_wake_waiters(const struct lu_env *env, struct osc_object *osc, + struct osc_lock *oscl); +int osc_lock_enqueue_wait(const struct lu_env *env, struct osc_object *obj, + struct osc_lock *oscl); +void osc_lock_set_writer(const struct lu_env *env, const struct cl_io *io, + struct cl_object *obj, struct osc_lock *oscl); +int osc_lock_print(const struct lu_env *env, void *cookie, + lu_printer_t p, const struct cl_lock_slice *slice); +void osc_lock_cancel(const struct lu_env *env, + const struct cl_lock_slice *slice); +void osc_lock_fini(const struct lu_env *env, struct cl_lock_slice *slice); +int osc_ldlm_glimpse_ast(struct ldlm_lock *dlmlock, void *data); /**************************************************************************** * @@ -828,18 +951,6 @@ struct osc_extent { unsigned int oe_mppr; }; -int osc_extent_finish(const struct lu_env *env, struct osc_extent *ext, - int sent, int rc); -void osc_extent_release(const struct lu_env *env, struct osc_extent *ext); - -int osc_lock_discard_pages(const struct lu_env *env, struct osc_object *osc, - pgoff_t start, pgoff_t end, bool discard_pages); - -typedef bool (*osc_page_gang_cbt)(const struct lu_env *, struct cl_io *, - struct osc_page *, void *); -bool osc_page_gang_lookup(const struct lu_env *env, struct cl_io *io, - struct osc_object *osc, pgoff_t start, pgoff_t end, - osc_page_gang_cbt cb, void *cbdata); /* @} osc */ #endif /* LUSTRE_OSC_H */ diff --git a/fs/lustre/ldlm/ldlm_internal.h b/fs/lustre/ldlm/ldlm_internal.h index c3788c2..275d823 100644 --- a/fs/lustre/ldlm/ldlm_internal.h +++ b/fs/lustre/ldlm/ldlm_internal.h @@ -37,6 +37,7 @@ extern struct list_head ldlm_srv_namespace_list; extern struct mutex ldlm_cli_namespace_lock; extern struct list_head ldlm_cli_active_namespace_list; +extern struct kmem_cache *ldlm_glimpse_work_kmem; static inline int ldlm_namespace_nr_read(enum ldlm_side client) { diff --git a/fs/lustre/ldlm/ldlm_request.c b/fs/lustre/ldlm/ldlm_request.c index 4185d42..09be016 100644 --- a/fs/lustre/ldlm/ldlm_request.c +++ b/fs/lustre/ldlm/ldlm_request.c @@ -1488,8 +1488,8 @@ static int ldlm_prepare_lru_list(struct ldlm_namespace *ns, lock->l_flags |= LDLM_FL_CBPENDING | LDLM_FL_CANCELING; if ((flags & LDLM_LRU_FLAG_CLEANUP) && - lock->l_resource->lr_type == LDLM_EXTENT && - lock->l_granted_mode == LCK_PR) + (lock->l_resource->lr_type == LDLM_EXTENT || + ldlm_has_dom(lock)) && lock->l_granted_mode == LCK_PR) ldlm_set_discard_data(lock); /* We can't re-add to l_lru as it confuses the diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index c393856..0852e37 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -1054,8 +1054,11 @@ int ll_merge_attr(const struct lu_env *env, struct inode *inode) rc = cl_object_attr_get(env, obj, attr); cl_object_attr_unlock(obj); - if (rc != 0) + if (rc != 0) { + if (rc == -ENODATA) + rc = 0; goto out_size_unlock; + } if (atime < attr->cat_atime) atime = attr->cat_atime; diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index 418f464..a02ac03 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -189,6 +189,7 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt) data->ocd_connect_flags = OBD_CONNECT_IBITS | OBD_CONNECT_NODEVOH | OBD_CONNECT_ATTRFID | OBD_CONNECT_GRANT | OBD_CONNECT_VERSION | OBD_CONNECT_BRW_SIZE | + OBD_CONNECT_SRVLOCK | OBD_CONNECT_TRUNCLOCK| OBD_CONNECT_CANCELSET | OBD_CONNECT_FID | OBD_CONNECT_AT | OBD_CONNECT_LOV_V3 | OBD_CONNECT_VBR | OBD_CONNECT_FULL20 | diff --git a/fs/lustre/lmv/lmv_obd.c b/fs/lustre/lmv/lmv_obd.c index aabd043..73ab7b6 100644 --- a/fs/lustre/lmv/lmv_obd.c +++ b/fs/lustre/lmv/lmv_obd.c @@ -1914,7 +1914,7 @@ static int lmv_rename(struct obd_export *exp, struct md_op_data *op_data, return rc; rc = lmv_early_cancel(exp, NULL, op_data, src_tgt->ltd_idx, - LCK_EX, MDS_INODELOCK_FULL, + LCK_EX, MDS_INODELOCK_ELC, MF_MDC_CANCEL_FID3); if (rc) return rc; @@ -1928,7 +1928,7 @@ static int lmv_rename(struct obd_export *exp, struct md_op_data *op_data, struct lmv_tgt_desc *tgt; rc = lmv_early_cancel(exp, NULL, op_data, src_tgt->ltd_idx, - LCK_EX, MDS_INODELOCK_FULL, + LCK_EX, MDS_INODELOCK_ELC, MF_MDC_CANCEL_FID4); if (rc) return rc; @@ -2472,7 +2472,7 @@ static int lmv_unlink(struct obd_export *exp, struct md_op_data *op_data, } rc = lmv_early_cancel(exp, NULL, op_data, tgt->ltd_idx, LCK_EX, - MDS_INODELOCK_FULL, MF_MDC_CANCEL_FID3); + MDS_INODELOCK_ELC, MF_MDC_CANCEL_FID3); if (rc != 0) return rc; diff --git a/fs/lustre/mdc/mdc_dev.c b/fs/lustre/mdc/mdc_dev.c index 77e152e..3a7afab 100644 --- a/fs/lustre/mdc/mdc_dev.c +++ b/fs/lustre/mdc/mdc_dev.c @@ -38,10 +38,877 @@ #include "mdc_internal.h" -int mdc_lock_init(const struct lu_env *env, - struct cl_object *obj, struct cl_lock *lock, - const struct cl_io *unused) +static void mdc_lock_build_policy(const struct lu_env *env, + union ldlm_policy_data *policy) { + memset(policy, 0, sizeof(*policy)); + policy->l_inodebits.bits = MDS_INODELOCK_DOM; +} + +int mdc_ldlm_glimpse_ast(struct ldlm_lock *dlmlock, void *data) +{ + return osc_ldlm_glimpse_ast(dlmlock, data); +} + +static void mdc_lock_build_einfo(const struct lu_env *env, + const struct cl_lock *lock, + struct osc_object *osc, + struct ldlm_enqueue_info *einfo) +{ + einfo->ei_type = LDLM_IBITS; + einfo->ei_mode = osc_cl_lock2ldlm(lock->cll_descr.cld_mode); + einfo->ei_cb_bl = mdc_ldlm_blocking_ast; + einfo->ei_cb_cp = ldlm_completion_ast; + einfo->ei_cb_gl = mdc_ldlm_glimpse_ast; + einfo->ei_cbdata = osc; /* value to be put into ->l_ast_data */ +} + +static int mdc_set_dom_lock_data(struct ldlm_lock *lock, void *data) +{ + int set = 0; + + LASSERT(lock); + + lock_res_and_lock(lock); + + if (!lock->l_ast_data) + lock->l_ast_data = data; + if (lock->l_ast_data == data) + set = 1; + + unlock_res_and_lock(lock); + + return set; +} + +int mdc_dom_lock_match(struct obd_export *exp, struct ldlm_res_id *res_id, + enum ldlm_type type, union ldlm_policy_data *policy, + enum ldlm_mode mode, u64 *flags, void *data, + struct lustre_handle *lockh, int unref) +{ + struct obd_device *obd = exp->exp_obd; + u64 lflags = *flags; + enum ldlm_mode rc; + + rc = ldlm_lock_match(obd->obd_namespace, lflags, + res_id, type, policy, mode, lockh, unref); + if (rc == 0 || lflags & LDLM_FL_TEST_LOCK) + return rc; + + if (data) { + struct ldlm_lock *lock = ldlm_handle2lock(lockh); + + LASSERT(lock); + if (!mdc_set_dom_lock_data(lock, data)) { + ldlm_lock_decref(lockh, rc); + rc = 0; + } + LDLM_LOCK_PUT(lock); + } + return rc; +} + +/** + * Finds an existing lock covering a page with given index. + * Copy of osc_obj_dlmlock_at_pgoff() but for DoM IBITS lock. + */ +struct ldlm_lock *mdc_dlmlock_at_pgoff(const struct lu_env *env, + struct osc_object *obj, pgoff_t index, + enum osc_dap_flags dap_flags) +{ + struct osc_thread_info *info = osc_env_info(env); + struct ldlm_res_id *resname = &info->oti_resname; + union ldlm_policy_data *policy = &info->oti_policy; + struct lustre_handle lockh; + struct ldlm_lock *lock = NULL; + enum ldlm_mode mode; + u64 flags; + + fid_build_reg_res_name(lu_object_fid(osc2lu(obj)), resname); + mdc_lock_build_policy(env, policy); + + flags = LDLM_FL_BLOCK_GRANTED | LDLM_FL_CBPENDING; + if (dap_flags & OSC_DAP_FL_TEST_LOCK) + flags |= LDLM_FL_TEST_LOCK; + +again: + /* Next, search for already existing extent locks that will cover us */ + /* If we're trying to read, we also search for an existing PW lock. The + * VFS and page cache already protect us locally, so lots of readers/ + * writers can share a single PW lock. + */ + mode = mdc_dom_lock_match(osc_export(obj), resname, LDLM_IBITS, policy, + LCK_PR | LCK_PW, &flags, obj, &lockh, + dap_flags & OSC_DAP_FL_CANCELING); + if (mode) { + lock = ldlm_handle2lock(&lockh); + /* RACE: the lock is cancelled so let's try again */ + if (!lock) + goto again; + } + + return lock; +} + +/** + * Check if page @page is covered by an extra lock or discard it. + */ +static bool mdc_check_and_discard_cb(const struct lu_env *env, struct cl_io *io, + struct osc_page *ops, void *cbdata) +{ + struct osc_thread_info *info = osc_env_info(env); + struct osc_object *osc = cbdata; + pgoff_t index; + + index = osc_index(ops); + if (index >= info->oti_fn_index) { + struct ldlm_lock *tmp; + struct cl_page *page = ops->ops_cl.cpl_page; + + /* refresh non-overlapped index */ + tmp = mdc_dlmlock_at_pgoff(env, osc, index, + OSC_DAP_FL_TEST_LOCK); + if (tmp) { + info->oti_fn_index = CL_PAGE_EOF; + LDLM_LOCK_PUT(tmp); + } else if (cl_page_own(env, io, page) == 0) { + /* discard the page */ + cl_page_discard(env, io, page); + cl_page_disown(env, io, page); + } else { + LASSERT(page->cp_state == CPS_FREEING); + } + } + + info->oti_next_index = index + 1; + return true; +} + +/** + * Discard pages protected by the given lock. This function traverses radix + * tree to find all covering pages and discard them. If a page is being covered + * by other locks, it should remain in cache. + * + * If error happens on any step, the process continues anyway (the reasoning + * behind this being that lock cancellation cannot be delayed indefinitely). + */ +static int mdc_lock_discard_pages(const struct lu_env *env, + struct osc_object *osc, + pgoff_t start, pgoff_t end, + bool discard) +{ + struct osc_thread_info *info = osc_env_info(env); + struct cl_io *io = &info->oti_io; + osc_page_gang_cbt cb; + bool res; + int result; + + io->ci_obj = cl_object_top(osc2cl(osc)); + io->ci_ignore_layout = 1; + result = cl_io_init(env, io, CIT_MISC, io->ci_obj); + if (result) + goto out; + + cb = discard ? osc_discard_cb : mdc_check_and_discard_cb; + info->oti_fn_index = info->oti_next_index = start; + do { + res = osc_page_gang_lookup(env, io, osc, info->oti_next_index, + end, cb, (void *)osc); + if (info->oti_next_index > end) + break; + + } while (!res); +out: + cl_io_fini(env, io); + return result; +} + +static int mdc_lock_flush(const struct lu_env *env, struct osc_object *obj, + pgoff_t start, pgoff_t end, enum cl_lock_mode mode, + bool discard) +{ + int result = 0; + int rc; + + if (mode == CLM_WRITE) { + result = osc_cache_writeback_range(env, obj, start, end, 1, + discard); + CDEBUG(D_CACHE, "object %p: [%lu -> %lu] %d pages were %s.\n", + obj, start, end, result, + discard ? "discarded" : "written back"); + if (result > 0) + result = 0; + } + + rc = mdc_lock_discard_pages(env, obj, start, end, discard); + if (result == 0 && rc < 0) + result = rc; + + return result; +} + +void mdc_lock_lockless_cancel(const struct lu_env *env, + const struct cl_lock_slice *slice) +{ + struct osc_lock *ols = cl2osc_lock(slice); + struct osc_object *osc = cl2osc(slice->cls_obj); + struct cl_lock_descr *descr = &slice->cls_lock->cll_descr; + int rc; + + LASSERT(!ols->ols_dlmlock); + rc = mdc_lock_flush(env, osc, descr->cld_start, descr->cld_end, + descr->cld_mode, 0); + if (rc != 0) + CERROR("Pages for lockless lock %p were not purged(%d)\n", + ols, rc); + + osc_lock_wake_waiters(env, osc, ols); +} + +/** + * Helper for osc_dlm_blocking_ast() handling discrepancies between cl_lock + * and ldlm_lock caches. + */ +static int mdc_dlm_blocking_ast0(const struct lu_env *env, + struct ldlm_lock *dlmlock, + void *data, int flag) +{ + struct cl_object *obj = NULL; + int result = 0; + bool discard; + enum cl_lock_mode mode = CLM_READ; + + LASSERT(flag == LDLM_CB_CANCELING); + LASSERT(dlmlock); + + lock_res_and_lock(dlmlock); + if (dlmlock->l_granted_mode != dlmlock->l_req_mode) { + dlmlock->l_ast_data = NULL; + unlock_res_and_lock(dlmlock); + return 0; + } + + discard = ldlm_is_discard_data(dlmlock); + if (dlmlock->l_granted_mode & (LCK_PW | LCK_GROUP)) + mode = CLM_WRITE; + + if (dlmlock->l_ast_data) { + obj = osc2cl(dlmlock->l_ast_data); + dlmlock->l_ast_data = NULL; + cl_object_get(obj); + } + unlock_res_and_lock(dlmlock); + + /* if l_ast_data is NULL, the dlmlock was enqueued by AGL or + * the object has been destroyed. + */ + if (obj) { + struct cl_attr *attr = &osc_env_info(env)->oti_attr; + + /* Destroy pages covered by the extent of the DLM lock */ + result = mdc_lock_flush(env, cl2osc(obj), cl_index(obj, 0), + CL_PAGE_EOF, mode, discard); + /* Losing a lock, set KMS to 0. + * NB: assumed that DOM lock covers whole data on MDT. + */ + /* losing a lock, update kms */ + lock_res_and_lock(dlmlock); + cl_object_attr_lock(obj); + attr->cat_kms = 0; + cl_object_attr_update(env, obj, attr, CAT_KMS); + cl_object_attr_unlock(obj); + unlock_res_and_lock(dlmlock); + cl_object_put(env, obj); + } + return result; +} + +int mdc_ldlm_blocking_ast(struct ldlm_lock *dlmlock, + struct ldlm_lock_desc *new, void *data, int flag) +{ + int rc = 0; + + switch (flag) { + case LDLM_CB_BLOCKING: { + struct lustre_handle lockh; + + ldlm_lock2handle(dlmlock, &lockh); + rc = ldlm_cli_cancel(&lockh, LCF_ASYNC); + if (rc == -ENODATA) + rc = 0; + break; + } + case LDLM_CB_CANCELING: { + struct lu_env *env; + u16 refcheck; + + /* + * This can be called in the context of outer IO, e.g., + * + * osc_enqueue_base()->... + * ->ldlm_prep_elc_req()->... + * ->ldlm_cancel_callback()->... + * ->osc_ldlm_blocking_ast() + * + * new environment has to be created to not corrupt outer + * context. + */ + env = cl_env_get(&refcheck); + if (IS_ERR(env)) { + rc = PTR_ERR(env); + break; + } + + rc = mdc_dlm_blocking_ast0(env, dlmlock, data, flag); + cl_env_put(env, &refcheck); + break; + } + default: + LBUG(); + } + return rc; +} + +/** + * Updates object attributes from a lock value block (lvb) received together + * with the DLM lock reply from the server. + * This can be optimized to not update attributes when lock is a result of a + * local match. + * + * Called under lock and resource spin-locks. + */ +static void mdc_lock_lvb_update(const struct lu_env *env, + struct osc_object *osc, + struct ldlm_lock *dlmlock, + struct ost_lvb *lvb) +{ + struct cl_object *obj = osc2cl(osc); + struct lov_oinfo *oinfo = osc->oo_oinfo; + struct cl_attr *attr = &osc_env_info(env)->oti_attr; + unsigned int valid = CAT_BLOCKS | CAT_ATIME | CAT_CTIME | CAT_MTIME | + CAT_SIZE; + + if (!lvb) { + LASSERT(dlmlock); + lvb = &dlmlock->l_ost_lvb; + } + cl_lvb2attr(attr, lvb); + + cl_object_attr_lock(obj); + if (dlmlock) { + u64 size; + + check_res_locked(dlmlock->l_resource); + size = lvb->lvb_size; + + if (size >= oinfo->loi_kms) { + LDLM_DEBUG(dlmlock, + "lock acquired, setting rss=%llu, kms=%llu", + lvb->lvb_size, size); + valid |= CAT_KMS; + attr->cat_kms = size; + } else { + LDLM_DEBUG(dlmlock, + "lock acquired, setting rss=%llu, leaving kms=%llu, end=%llu", + lvb->lvb_size, oinfo->loi_kms, + dlmlock->l_policy_data.l_extent.end); + } + } + cl_object_attr_update(env, obj, attr, valid); + cl_object_attr_unlock(obj); +} + +static void mdc_lock_granted(const struct lu_env *env, struct osc_lock *oscl, + struct lustre_handle *lockh, bool lvb_update) +{ + struct ldlm_lock *dlmlock; + + dlmlock = ldlm_handle2lock_long(lockh, 0); + LASSERT(dlmlock); + + /* lock reference taken by ldlm_handle2lock_long() is + * owned by osc_lock and released in osc_lock_detach() + */ + lu_ref_add(&dlmlock->l_reference, "osc_lock", oscl); + oscl->ols_has_ref = 1; + + LASSERT(!oscl->ols_dlmlock); + oscl->ols_dlmlock = dlmlock; + + /* This may be a matched lock for glimpse request, do not hold + * lock reference in that case. + */ + if (!oscl->ols_glimpse) { + /* hold a refc for non glimpse lock which will + * be released in osc_lock_cancel() + */ + lustre_handle_copy(&oscl->ols_handle, lockh); + ldlm_lock_addref(lockh, oscl->ols_einfo.ei_mode); + oscl->ols_hold = 1; + } + + /* Lock must have been granted. */ + lock_res_and_lock(dlmlock); + if (dlmlock->l_granted_mode == dlmlock->l_req_mode) { + struct cl_lock_descr *descr = &oscl->ols_cl.cls_lock->cll_descr; + + /* extend the lock extent, otherwise it will have problem when + * we decide whether to grant a lockless lock. + */ + descr->cld_mode = osc_ldlm2cl_lock(dlmlock->l_granted_mode); + descr->cld_start = cl_index(descr->cld_obj, 0); + descr->cld_end = CL_PAGE_EOF; + + /* no lvb update for matched lock */ + if (lvb_update) { + LASSERT(oscl->ols_flags & LDLM_FL_LVB_READY); + mdc_lock_lvb_update(env, cl2osc(oscl->ols_cl.cls_obj), + dlmlock, NULL); + } + } + unlock_res_and_lock(dlmlock); + + LASSERT(oscl->ols_state != OLS_GRANTED); + oscl->ols_state = OLS_GRANTED; +} + +/** + * Lock upcall function that is executed either when a reply to ENQUEUE rpc is + * received from a server, or after osc_enqueue_base() matched a local DLM + * lock. + */ +static int mdc_lock_upcall(void *cookie, struct lustre_handle *lockh, + int errcode) +{ + struct osc_lock *oscl = cookie; + struct cl_lock_slice *slice = &oscl->ols_cl; + struct lu_env *env; + int rc; + + env = cl_env_percpu_get(); + /* should never happen, similar to osc_ldlm_blocking_ast(). */ + LASSERT(!IS_ERR(env)); + + rc = ldlm_error2errno(errcode); + if (oscl->ols_state == OLS_ENQUEUED) { + oscl->ols_state = OLS_UPCALL_RECEIVED; + } else if (oscl->ols_state == OLS_CANCELLED) { + rc = -EIO; + } else { + CERROR("Impossible state: %d\n", oscl->ols_state); + LBUG(); + } + + CDEBUG(D_INODE, "rc %d, err %d\n", rc, errcode); + if (rc == 0) + mdc_lock_granted(env, oscl, lockh, errcode == ELDLM_OK); + + /* Error handling, some errors are tolerable. */ + if (oscl->ols_locklessable && rc == -EUSERS) { + /* This is a tolerable error, turn this lock into + * lockless lock. + */ + osc_object_set_contended(cl2osc(slice->cls_obj)); + LASSERT(slice->cls_ops != oscl->ols_lockless_ops); + + /* Change this lock to ldlmlock-less lock. */ + osc_lock_to_lockless(env, oscl, 1); + oscl->ols_state = OLS_GRANTED; + rc = 0; + } else if (oscl->ols_glimpse && rc == -ENAVAIL) { + LASSERT(oscl->ols_flags & LDLM_FL_LVB_READY); + mdc_lock_lvb_update(env, cl2osc(slice->cls_obj), + NULL, &oscl->ols_lvb); + /* Hide the error. */ + rc = 0; + } + + if (oscl->ols_owner) + cl_sync_io_note(env, oscl->ols_owner, rc); + cl_env_percpu_put(env); + + return rc; +} + +int mdc_fill_lvb(struct ptlrpc_request *req, struct ost_lvb *lvb) +{ + struct mdt_body *body; + + body = req_capsule_server_get(&req->rq_pill, &RMF_MDT_BODY); + if (!body) + return -EPROTO; + + lvb->lvb_mtime = body->mbo_mtime; + lvb->lvb_atime = body->mbo_atime; + lvb->lvb_ctime = body->mbo_ctime; + lvb->lvb_blocks = body->mbo_blocks; + lvb->lvb_size = body->mbo_size; + return 0; +} + +int mdc_enqueue_fini(struct ptlrpc_request *req, osc_enqueue_upcall_f upcall, + void *cookie, struct lustre_handle *lockh, + enum ldlm_mode mode, u64 *flags, int errcode) +{ + struct osc_lock *ols = cookie; + struct ldlm_lock *lock; + int rc = 0; + + /* The request was created before ldlm_cli_enqueue call. */ + if (errcode == ELDLM_LOCK_ABORTED) { + struct ldlm_reply *rep; + + rep = req_capsule_server_get(&req->rq_pill, &RMF_DLM_REP); + LASSERT(rep); + + rep->lock_policy_res2 = + ptlrpc_status_ntoh(rep->lock_policy_res2); + if (rep->lock_policy_res2) + errcode = rep->lock_policy_res2; + + rc = mdc_fill_lvb(req, &ols->ols_lvb); + *flags |= LDLM_FL_LVB_READY; + } else if (errcode == ELDLM_OK) { + /* Callers have references, should be valid always */ + lock = ldlm_handle2lock(lockh); + LASSERT(lock); + + rc = mdc_fill_lvb(req, &lock->l_ost_lvb); + LDLM_LOCK_PUT(lock); + *flags |= LDLM_FL_LVB_READY; + } + + /* Call the update callback. */ + rc = (*upcall)(cookie, lockh, rc < 0 ? rc : errcode); + + /* release the reference taken in ldlm_cli_enqueue() */ + if (errcode == ELDLM_LOCK_MATCHED) + errcode = ELDLM_OK; + if (errcode == ELDLM_OK && lustre_handle_is_used(lockh)) + ldlm_lock_decref(lockh, mode); + + return rc; +} + +int mdc_enqueue_interpret(const struct lu_env *env, struct ptlrpc_request *req, + struct osc_enqueue_args *aa, int rc) +{ + struct ldlm_lock *lock; + struct lustre_handle *lockh = &aa->oa_lockh; + enum ldlm_mode mode = aa->oa_mode; + + LASSERT(!aa->oa_speculative); + + /* ldlm_cli_enqueue is holding a reference on the lock, so it must + * be valid. + */ + lock = ldlm_handle2lock(lockh); + LASSERTF(lock, + "lockh %#llx, req %p, aa %p - client evicted?\n", + lockh->cookie, req, aa); + + /* Take an additional reference so that a blocking AST that + * ldlm_cli_enqueue_fini() might post for a failed lock, is guaranteed + * to arrive after an upcall has been executed by + * osc_enqueue_fini(). + */ + ldlm_lock_addref(lockh, mode); + + /* Let cl_lock_state_wait fail with -ERESTARTSYS to unuse sublocks. */ + OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_ENQUEUE_HANG, 2); + + /* Let CP AST to grant the lock first. */ + OBD_FAIL_TIMEOUT(OBD_FAIL_OSC_CP_ENQ_RACE, 1); + + /* Complete obtaining the lock procedure. */ + rc = ldlm_cli_enqueue_fini(aa->oa_exp, req, aa->oa_type, 1, + aa->oa_mode, aa->oa_flags, NULL, 0, + lockh, rc); + /* Complete mdc stuff. */ + rc = mdc_enqueue_fini(req, aa->oa_upcall, aa->oa_cookie, lockh, mode, + aa->oa_flags, rc); + + OBD_FAIL_TIMEOUT(OBD_FAIL_OSC_CP_CANCEL_RACE, 10); + + ldlm_lock_decref(lockh, mode); + LDLM_LOCK_PUT(lock); + return rc; +} + +/* When enqueuing asynchronously, locks are not ordered, we can obtain a lock + * from the 2nd OSC before a lock from the 1st one. This does not deadlock with + * other synchronous requests, however keeping some locks and trying to obtain + * others may take a considerable amount of time in a case of ost failure; and + * when other sync requests do not get released lock from a client, the client + * is excluded from the cluster -- such scenarious make the life difficult, so + * release locks just after they are obtained. + */ +int mdc_enqueue_send(struct obd_export *exp, struct ldlm_res_id *res_id, + u64 *flags, union ldlm_policy_data *policy, + struct ost_lvb *lvb, int kms_valid, + osc_enqueue_upcall_f upcall, void *cookie, + struct ldlm_enqueue_info *einfo, int async) +{ + struct obd_device *obd = exp->exp_obd; + struct lustre_handle lockh = { 0 }; + struct ptlrpc_request *req = NULL; + struct ldlm_intent *lit; + enum ldlm_mode mode; + bool glimpse = *flags & LDLM_FL_HAS_INTENT; + u64 match_flags = *flags; + int rc; + + if (!kms_valid) + goto no_match; + + mode = einfo->ei_mode; + if (einfo->ei_mode == LCK_PR) + mode |= LCK_PW; + + if (!glimpse) + match_flags |= LDLM_FL_BLOCK_GRANTED; + mode = ldlm_lock_match(obd->obd_namespace, match_flags, res_id, + einfo->ei_type, policy, mode, &lockh, 0); + if (mode) { + struct ldlm_lock *matched; + + if (*flags & LDLM_FL_TEST_LOCK) + return ELDLM_OK; + + matched = ldlm_handle2lock(&lockh); + if (!mdc_set_dom_lock_data(matched, einfo->ei_cbdata)) { + ldlm_lock_decref(&lockh, mode); + LDLM_LOCK_PUT(matched); + } else { + *flags |= LDLM_FL_LVB_READY; + + /* We already have a lock, and it's referenced. */ + (*upcall)(cookie, &lockh, ELDLM_LOCK_MATCHED); + + ldlm_lock_decref(&lockh, mode); + LDLM_LOCK_PUT(matched); + return ELDLM_OK; + } + } + +no_match: + if (*flags & (LDLM_FL_TEST_LOCK | LDLM_FL_MATCH_LOCK)) + return -ENOLCK; + + req = ptlrpc_request_alloc(class_exp2cliimp(exp), &RQF_LDLM_INTENT); + if (!req) + return -ENOMEM; + + rc = ldlm_prep_enqueue_req(exp, req, NULL, 0); + if (rc < 0) { + ptlrpc_request_free(req); + return rc; + } + + /* pack the intent */ + lit = req_capsule_client_get(&req->rq_pill, &RMF_LDLM_INTENT); + lit->opc = glimpse ? IT_GLIMPSE : IT_BRW; + + req_capsule_set_size(&req->rq_pill, &RMF_MDT_MD, RCL_SERVER, 0); + req_capsule_set_size(&req->rq_pill, &RMF_ACL, RCL_SERVER, 0); + ptlrpc_request_set_replen(req); + + /* users of mdc_enqueue() can pass this flag for ldlm_lock_match() */ + *flags &= ~LDLM_FL_BLOCK_GRANTED; + /* All MDC IO locks are intents */ + *flags |= LDLM_FL_HAS_INTENT; + rc = ldlm_cli_enqueue(exp, &req, einfo, res_id, policy, flags, NULL, + 0, LVB_T_NONE, &lockh, async); + if (async) { + if (!rc) { + struct osc_enqueue_args *aa; + + aa = ptlrpc_req_async_args(req); + aa->oa_exp = exp; + aa->oa_mode = einfo->ei_mode; + aa->oa_type = einfo->ei_type; + lustre_handle_copy(&aa->oa_lockh, &lockh); + aa->oa_upcall = upcall; + aa->oa_cookie = cookie; + aa->oa_speculative = false; + aa->oa_flags = flags; + aa->oa_lvb = lvb; + + req->rq_interpret_reply = + (ptlrpc_interpterer_t)mdc_enqueue_interpret; + ptlrpcd_add_req(req); + } else { + ptlrpc_req_finished(req); + } + return rc; + } + + rc = mdc_enqueue_fini(req, upcall, cookie, &lockh, einfo->ei_mode, + flags, rc); + ptlrpc_req_finished(req); + return rc; +} + +/** + * Implementation of cl_lock_operations::clo_enqueue() method for osc + * layer. This initiates ldlm enqueue: + * + * - cancels conflicting locks early (osc_lock_enqueue_wait()); + * + * - calls osc_enqueue_base() to do actual enqueue. + * + * osc_enqueue_base() is supplied with an upcall function that is executed + * when lock is received either after a local cached ldlm lock is matched, or + * when a reply from the server is received. + * + * This function does not wait for the network communication to complete. + */ +static int mdc_lock_enqueue(const struct lu_env *env, + const struct cl_lock_slice *slice, + struct cl_io *unused, struct cl_sync_io *anchor) +{ + struct osc_thread_info *info = osc_env_info(env); + struct osc_io *oio = osc_env_io(env); + struct osc_object *osc = cl2osc(slice->cls_obj); + struct osc_lock *oscl = cl2osc_lock(slice); + struct cl_lock *lock = slice->cls_lock; + struct ldlm_res_id *resname = &info->oti_resname; + union ldlm_policy_data *policy = &info->oti_policy; + osc_enqueue_upcall_f upcall = mdc_lock_upcall; + void *cookie = (void *)oscl; + bool async = false; + int result; + + LASSERTF(ergo(oscl->ols_glimpse, lock->cll_descr.cld_mode <= CLM_READ), + "lock = %p, ols = %p\n", lock, oscl); + + if (oscl->ols_state == OLS_GRANTED) + return 0; + + /* Lockahead is not supported on MDT yet */ + if (oscl->ols_flags & LDLM_FL_NO_EXPANSION) { + result = -EOPNOTSUPP; + return result; + } + + if (oscl->ols_flags & LDLM_FL_TEST_LOCK) + goto enqueue_base; + + if (oscl->ols_glimpse) { + LASSERT(equi(oscl->ols_speculative, !anchor)); + async = true; + goto enqueue_base; + } + + result = osc_lock_enqueue_wait(env, osc, oscl); + if (result < 0) + goto out; + + /* we can grant lockless lock right after all conflicting locks + * are canceled. + */ + if (osc_lock_is_lockless(oscl)) { + oscl->ols_state = OLS_GRANTED; + oio->oi_lockless = 1; + return 0; + } + +enqueue_base: + oscl->ols_state = OLS_ENQUEUED; + if (anchor) { + atomic_inc(&anchor->csi_sync_nr); + oscl->ols_owner = anchor; + } + + /** + * DLM lock's ast data must be osc_object; + * DLM's enqueue callback set to osc_lock_upcall() with cookie as + * osc_lock. + */ + fid_build_reg_res_name(lu_object_fid(osc2lu(osc)), resname); + mdc_lock_build_policy(env, policy); + LASSERT(!oscl->ols_speculative); + result = mdc_enqueue_send(osc_export(osc), resname, &oscl->ols_flags, + policy, &oscl->ols_lvb, + osc->oo_oinfo->loi_kms_valid, + upcall, cookie, &oscl->ols_einfo, async); + if (result == 0) { + if (osc_lock_is_lockless(oscl)) { + oio->oi_lockless = 1; + } else if (!async) { + LASSERT(oscl->ols_state == OLS_GRANTED); + LASSERT(oscl->ols_hold); + LASSERT(oscl->ols_dlmlock); + } + } +out: + if (result < 0) { + oscl->ols_state = OLS_CANCELLED; + osc_lock_wake_waiters(env, osc, oscl); + + if (anchor) + cl_sync_io_note(env, anchor, result); + } + return result; +} + +static const struct cl_lock_operations mdc_lock_lockless_ops = { + .clo_fini = osc_lock_fini, + .clo_enqueue = mdc_lock_enqueue, + .clo_cancel = mdc_lock_lockless_cancel, + .clo_print = osc_lock_print +}; + +static const struct cl_lock_operations mdc_lock_ops = { + .clo_fini = osc_lock_fini, + .clo_enqueue = mdc_lock_enqueue, + .clo_cancel = osc_lock_cancel, + .clo_print = osc_lock_print, +}; + +int mdc_lock_init(const struct lu_env *env, struct cl_object *obj, + struct cl_lock *lock, const struct cl_io *io) +{ + struct osc_lock *ols; + u32 enqflags = lock->cll_descr.cld_enq_flags; + u64 flags = osc_enq2ldlm_flags(enqflags); + + /* Ignore AGL for Data-on-MDT, stat returns size data */ + if ((enqflags & CEF_SPECULATIVE) != 0) + return 0; + + ols = kmem_cache_zalloc(osc_lock_kmem, GFP_NOFS); + if (!ols) + return -ENOMEM; + + ols->ols_state = OLS_NEW; + spin_lock_init(&ols->ols_lock); + INIT_LIST_HEAD(&ols->ols_waiting_list); + INIT_LIST_HEAD(&ols->ols_wait_entry); + INIT_LIST_HEAD(&ols->ols_nextlock_oscobj); + ols->ols_lockless_ops = &mdc_lock_lockless_ops; + + ols->ols_flags = flags; + ols->ols_speculative = !!(enqflags & CEF_SPECULATIVE); + + if (ols->ols_flags & LDLM_FL_HAS_INTENT) { + ols->ols_flags |= LDLM_FL_BLOCK_GRANTED; + ols->ols_glimpse = 1; + } + mdc_lock_build_einfo(env, lock, cl2osc(obj), &ols->ols_einfo); + + cl_lock_slice_add(lock, &ols->ols_cl, obj, &mdc_lock_ops); + + if (!(enqflags & CEF_MUST)) + osc_lock_to_lockless(env, ols, (enqflags & CEF_NEVER)); + if (ols->ols_locklessable && !(enqflags & CEF_DISCARD_DATA)) + ols->ols_flags |= LDLM_FL_DENY_ON_CONTENTION; + + if (io->ci_type == CIT_WRITE || cl_io_is_mkwrite(io)) + osc_lock_set_writer(env, io, obj, ols); + + LDLM_DEBUG_NOLOCK("lock %p, mdc lock %p, flags %llx\n", + lock, ols, ols->ols_flags); return 0; } @@ -141,6 +1008,33 @@ static int mdc_io_setattr_start(const struct lu_env *env, return rc; } +static int mdc_io_read_ahead(const struct lu_env *env, + const struct cl_io_slice *ios, + pgoff_t start, struct cl_read_ahead *ra) +{ + struct osc_object *osc = cl2osc(ios->cis_obj); + struct ldlm_lock *dlmlock; + + dlmlock = mdc_dlmlock_at_pgoff(env, osc, start, 0); + if (!dlmlock) + return -ENODATA; + + if (dlmlock->l_req_mode != LCK_PR) { + struct lustre_handle lockh; + + ldlm_lock2handle(dlmlock, &lockh); + ldlm_lock_addref(&lockh, LCK_PR); + ldlm_lock_decref(&lockh, dlmlock->l_req_mode); + } + + ra->cra_rpc_size = osc_cli(osc)->cl_max_pages_per_rpc; + ra->cra_end = CL_PAGE_EOF; + ra->cra_release = osc_read_ahead_release; + ra->cra_cbdata = dlmlock; + + return 0; +} + static struct cl_io_operations mdc_io_ops = { .op = { [CIT_READ] = { @@ -176,6 +1070,7 @@ static int mdc_io_setattr_start(const struct lu_env *env, .cio_end = osc_io_fsync_end, }, }, + .cio_read_ahead = mdc_io_read_ahead, .cio_submit = osc_io_submit, .cio_commit_async = osc_io_commit_async, }; @@ -190,6 +1085,12 @@ int mdc_io_init(const struct lu_env *env, struct cl_object *obj, return 0; } +static void mdc_build_res_name(struct osc_object *osc, + struct ldlm_res_id *resname) +{ + fid_build_reg_res_name(lu_object_fid(osc2lu(osc)), resname); +} + /** * Implementation of struct cl_req_operations::cro_attr_set() for MDC * layer. MDC is responsible for struct obdo::o_id and struct obdo::o_seq @@ -208,16 +1109,66 @@ static void mdc_req_attr_set(const struct lu_env *env, struct cl_object *obj, if (flags & OBD_MD_FLID) attr->cra_oa->o_valid |= OBD_MD_FLID; + + if (flags & OBD_MD_FLHANDLE) { + struct ldlm_lock *lock; /* _some_ lock protecting @apage */ + struct osc_page *opg; + + opg = osc_cl_page_osc(attr->cra_page, cl2osc(obj)); + lock = mdc_dlmlock_at_pgoff(env, cl2osc(obj), osc_index(opg), + OSC_DAP_FL_TEST_LOCK | OSC_DAP_FL_CANCELING); + if (!lock && !opg->ops_srvlock) { + struct ldlm_resource *res; + struct ldlm_res_id *resname; + + CL_PAGE_DEBUG(D_ERROR, env, attr->cra_page, + "uncovered page!\n"); + + resname = &osc_env_info(env)->oti_resname; + mdc_build_res_name(cl2osc(obj), resname); + res = ldlm_resource_get( + osc_export(cl2osc(obj))->exp_obd->obd_namespace, + NULL, resname, LDLM_IBITS, 0); + ldlm_resource_dump(D_ERROR, res); + + libcfs_debug_dumpstack(NULL); + LBUG(); + } + + /* check for lockless io. */ + if (lock) { + attr->cra_oa->o_handle = lock->l_remote_handle; + attr->cra_oa->o_valid |= OBD_MD_FLHANDLE; + LDLM_LOCK_PUT(lock); + } + } +} + +static int mdc_attr_get(const struct lu_env *env, struct cl_object *obj, + struct cl_attr *attr) +{ + struct lov_oinfo *oinfo = cl2osc(obj)->oo_oinfo; + + if (OST_LVB_IS_ERR(oinfo->loi_lvb.lvb_blocks)) + return OST_LVB_GET_ERR(oinfo->loi_lvb.lvb_blocks); + + return osc_attr_get(env, obj, attr); } static const struct cl_object_operations mdc_ops = { .coo_page_init = osc_page_init, .coo_lock_init = mdc_lock_init, .coo_io_init = mdc_io_init, - .coo_attr_get = osc_attr_get, + .coo_attr_get = mdc_attr_get, .coo_attr_update = osc_attr_update, .coo_glimpse = osc_object_glimpse, .coo_req_attr_set = mdc_req_attr_set, + .coo_prune = osc_object_prune, +}; + +static const struct osc_object_operations mdc_object_ops = { + .oto_build_res_name = mdc_build_res_name, + .oto_dlmlock_at_pgoff = mdc_dlmlock_at_pgoff, }; static int mdc_object_init(const struct lu_env *env, struct lu_object *obj, @@ -260,6 +1211,7 @@ struct lu_object *mdc_object_alloc(const struct lu_env *env, lu_object_init(obj, NULL, dev); osc->oo_cl.co_ops = &mdc_ops; obj->lo_ops = &mdc_lu_obj_ops; + osc->oo_obj_ops = &mdc_object_ops; osc->oo_initialized = false; } else { obj = NULL; diff --git a/fs/lustre/mdc/mdc_internal.h b/fs/lustre/mdc/mdc_internal.h index 943b383..6e69dfe 100644 --- a/fs/lustre/mdc/mdc_internal.h +++ b/fs/lustre/mdc/mdc_internal.h @@ -154,5 +154,7 @@ static inline unsigned long hash_x_index(u64 hash, int hash64) /* mdc_dev.c */ extern struct lu_device_type mdc_device_type; +int mdc_ldlm_blocking_ast(struct ldlm_lock *dlmlock, + struct ldlm_lock_desc *new, void *data, int flag); #endif diff --git a/fs/lustre/mdc/mdc_reint.c b/fs/lustre/mdc/mdc_reint.c index e0e7b00..87dabaf 100644 --- a/fs/lustre/mdc/mdc_reint.c +++ b/fs/lustre/mdc/mdc_reint.c @@ -270,9 +270,10 @@ int mdc_unlink(struct obd_export *exp, struct md_op_data *op_data, MDS_INODELOCK_UPDATE); if ((op_data->op_flags & MF_MDC_CANCEL_FID3) && (fid_is_sane(&op_data->op_fid3))) + /* don't cancel DoM lock which may cause data flush */ count += mdc_resource_get_unused(exp, &op_data->op_fid3, &cancels, LCK_EX, - MDS_INODELOCK_FULL); + MDS_INODELOCK_ELC); req = ptlrpc_request_alloc(class_exp2cliimp(exp), &RQF_MDS_REINT_UNLINK); if (!req) { @@ -373,7 +374,7 @@ int mdc_rename(struct obd_export *exp, struct md_op_data *op_data, (fid_is_sane(&op_data->op_fid4))) count += mdc_resource_get_unused(exp, &op_data->op_fid4, &cancels, LCK_EX, - MDS_INODELOCK_FULL); + MDS_INODELOCK_ELC); req = ptlrpc_request_alloc(class_exp2cliimp(exp), op_data->op_cli_flags & CLI_MIGRATE ? diff --git a/fs/lustre/osc/osc_cache.c b/fs/lustre/osc/osc_cache.c index cda8791..32785e4 100644 --- a/fs/lustre/osc/osc_cache.c +++ b/fs/lustre/osc/osc_cache.c @@ -248,7 +248,9 @@ static int __osc_extent_sanity_check(struct osc_extent *ext, goto out; } - if (ext->oe_dlmlock && !ldlm_is_failed(ext->oe_dlmlock)) { + if (ext->oe_dlmlock && + ext->oe_dlmlock->l_resource->lr_type == LDLM_EXTENT && + !ldlm_is_failed(ext->oe_dlmlock)) { struct ldlm_extent *extent; extent = &ext->oe_dlmlock->l_policy_data.l_extent; @@ -3096,6 +3098,7 @@ bool osc_page_gang_lookup(const struct lu_env *env, struct cl_io *io, return res; } +EXPORT_SYMBOL(osc_page_gang_lookup); /** * Check if page @page is covered by an extra lock or discard it. @@ -3140,8 +3143,8 @@ static bool check_and_discard_cb(const struct lu_env *env, struct cl_io *io, return true; } -static bool discard_cb(const struct lu_env *env, struct cl_io *io, - struct osc_page *ops, void *cbdata) +bool osc_discard_cb(const struct lu_env *env, struct cl_io *io, + struct osc_page *ops, void *cbdata) { struct osc_thread_info *info = osc_env_info(env); struct cl_page *page = ops->ops_cl.cpl_page; @@ -3163,6 +3166,7 @@ static bool discard_cb(const struct lu_env *env, struct cl_io *io, return true; } +EXPORT_SYMBOL(osc_discard_cb); /** * Discard pages protected by the given lock. This function traverses radix @@ -3186,7 +3190,7 @@ int osc_lock_discard_pages(const struct lu_env *env, struct osc_object *osc, if (result != 0) goto out; - cb = discard ? discard_cb : check_and_discard_cb; + cb = discard ? osc_discard_cb : check_and_discard_cb; info->oti_fn_index = start; info->oti_next_index = start; diff --git a/fs/lustre/osc/osc_internal.h b/fs/lustre/osc/osc_internal.h index f9ab069..8f89443 100644 --- a/fs/lustre/osc/osc_internal.h +++ b/fs/lustre/osc/osc_internal.h @@ -45,9 +45,11 @@ int osc_shrink_grant_to_target(struct client_obd *cli, u64 target_bytes); void osc_update_next_shrink(struct client_obd *cli); int lru_queue_work(const struct lu_env *env, void *data); - -typedef int (*osc_enqueue_upcall_f)(void *cookie, struct lustre_handle *lockh, - int rc); +int osc_extent_finish(const struct lu_env *env, struct osc_extent *ext, + int sent, int rc); +void osc_extent_release(const struct lu_env *env, struct osc_extent *ext); +int osc_lock_discard_pages(const struct lu_env *env, struct osc_object *osc, + pgoff_t start, pgoff_t end, bool discard); int osc_enqueue_base(struct obd_export *exp, struct ldlm_res_id *res_id, u64 *flags, union ldlm_policy_data *policy, @@ -136,24 +138,10 @@ int osc_quotactl(struct obd_device *unused, struct obd_export *exp, void osc_dec_unstable_pages(struct ptlrpc_request *req); bool osc_over_unstable_soft_limit(struct client_obd *cli); -/* - * Bit flags for osc_dlm_lock_at_pageoff(). - */ -enum osc_dap_flags { - /* - * Just check if the desired lock exists, it won't hold reference - * count on lock. - */ - OSC_DAP_FL_TEST_LOCK = BIT(0), - /* - * Return the lock even if it is being canceled. - */ - OSC_DAP_FL_CANCELING = BIT(1), -}; - -struct ldlm_lock *osc_dlmlock_at_pgoff(const struct lu_env *env, - struct osc_object *obj, pgoff_t index, - enum osc_dap_flags flags); +struct ldlm_lock *osc_obj_dlmlock_at_pgoff(const struct lu_env *env, + struct osc_object *obj, + pgoff_t index, + enum osc_dap_flags flags); int osc_object_invalidate(const struct lu_env *env, struct osc_object *osc); diff --git a/fs/lustre/osc/osc_io.c b/fs/lustre/osc/osc_io.c index 176ebe8..0a67089 100644 --- a/fs/lustre/osc/osc_io.c +++ b/fs/lustre/osc/osc_io.c @@ -57,7 +57,7 @@ static void osc_io_fini(const struct lu_env *env, const struct cl_io_slice *io) { } -static void osc_read_ahead_release(const struct lu_env *env, void *cbdata) +void osc_read_ahead_release(const struct lu_env *env, void *cbdata) { struct ldlm_lock *dlmlock = cbdata; struct lustre_handle lockh; @@ -66,6 +66,7 @@ static void osc_read_ahead_release(const struct lu_env *env, void *cbdata) ldlm_lock_decref(&lockh, LCK_PR); LDLM_LOCK_PUT(dlmlock); } +EXPORT_SYMBOL(osc_read_ahead_release); static int osc_io_read_ahead(const struct lu_env *env, const struct cl_io_slice *ios, diff --git a/fs/lustre/osc/osc_lock.c b/fs/lustre/osc/osc_lock.c index f976230..42dd654 100644 --- a/fs/lustre/osc/osc_lock.c +++ b/fs/lustre/osc/osc_lock.c @@ -48,22 +48,6 @@ * @{ */ -/***************************************************************************** - * - * Type conversions. - * - */ - -static const struct cl_lock_operations osc_lock_ops; -static const struct cl_lock_operations osc_lock_lockless_ops; -static void osc_lock_to_lockless(const struct lu_env *env, - struct osc_lock *ols, int force); - -int osc_lock_is_lockless(const struct osc_lock *olck) -{ - return (olck->ols_cl.cls_ops == &osc_lock_lockless_ops); -} - /** * Returns a weak pointer to the ldlm lock identified by a handle. Returned * pointer cannot be dereferenced, as lock is not protected from concurrent @@ -133,8 +117,7 @@ static int osc_lock_invariant(struct osc_lock *ols) * */ -static void osc_lock_fini(const struct lu_env *env, - struct cl_lock_slice *slice) +void osc_lock_fini(const struct lu_env *env, struct cl_lock_slice *slice) { struct osc_lock *ols = cl2osc_lock(slice); @@ -143,6 +126,7 @@ static void osc_lock_fini(const struct lu_env *env, kmem_cache_free(osc_lock_kmem, ols); } +EXPORT_SYMBOL(osc_lock_fini); static void osc_lock_build_policy(const struct lu_env *env, const struct cl_lock *lock, @@ -154,31 +138,6 @@ static void osc_lock_build_policy(const struct lu_env *env, policy->l_extent.gid = d->cld_gid; } -static u64 osc_enq2ldlm_flags(u32 enqflags) -{ - u64 result = 0; - - CDEBUG(D_DLMTRACE, "flags: %x\n", enqflags); - - LASSERT((enqflags & ~CEF_MASK) == 0); - - if (enqflags & CEF_NONBLOCK) - result |= LDLM_FL_BLOCK_NOWAIT; - if (enqflags & CEF_GLIMPSE) - result |= LDLM_FL_HAS_INTENT; - if (enqflags & CEF_DISCARD_DATA) - result |= LDLM_FL_AST_DISCARD_DATA; - if (enqflags & CEF_PEEK) - result |= LDLM_FL_TEST_LOCK; - if (enqflags & CEF_LOCK_MATCH) - result |= LDLM_FL_MATCH_LOCK; - if (enqflags & CEF_LOCK_NO_EXPAND) - result |= LDLM_FL_NO_EXPANSION; - if (enqflags & CEF_SPECULATIVE) - result |= LDLM_FL_SPECULATIVE; - return result; -} - /** * Updates object attributes from a lock value block (lvb) received together * with the DLM lock reply from the server. Copy of osc_update_enqueue() @@ -330,7 +289,7 @@ static int osc_lock_upcall(void *cookie, struct lustre_handle *lockh, * lockless lock. */ osc_object_set_contended(cl2osc(slice->cls_obj)); - LASSERT(slice->cls_ops == &osc_lock_ops); + LASSERT(slice->cls_ops != oscl->ols_lockless_ops); /* Change this lock to ldlmlock-less lock. */ osc_lock_to_lockless(env, oscl, 1); @@ -576,7 +535,7 @@ static int osc_ldlm_blocking_ast(struct ldlm_lock *dlmlock, return result; } -static int osc_ldlm_glimpse_ast(struct ldlm_lock *dlmlock, void *data) +int osc_ldlm_glimpse_ast(struct ldlm_lock *dlmlock, void *data) { struct ptlrpc_request *req = data; struct lu_env *env; @@ -639,6 +598,7 @@ static int osc_ldlm_glimpse_ast(struct ldlm_lock *dlmlock, void *data) req->rq_status = result; return result; } +EXPORT_SYMBOL(osc_ldlm_glimpse_ast); static bool weigh_cb(const struct lu_env *env, struct cl_io *io, struct osc_page *ops, void *cbdata) @@ -768,46 +728,46 @@ static void osc_lock_build_einfo(const struct lu_env *env, * Additional policy can be implemented here, e.g., never do lockless-io * for large extents. */ -static void osc_lock_to_lockless(const struct lu_env *env, - struct osc_lock *ols, int force) +void osc_lock_to_lockless(const struct lu_env *env, + struct osc_lock *ols, int force) { struct cl_lock_slice *slice = &ols->ols_cl; + struct osc_io *oio = osc_env_io(env); + struct cl_io *io = oio->oi_cl.cis_io; + struct cl_object *obj = slice->cls_obj; + struct osc_object *oob = cl2osc(obj); + const struct osc_device *osd = lu2osc_dev(obj->co_lu.lo_dev); + struct obd_connect_data *ocd; LASSERT(ols->ols_state == OLS_NEW || ols->ols_state == OLS_UPCALL_RECEIVED); if (force) { ols->ols_locklessable = 1; - slice->cls_ops = &osc_lock_lockless_ops; + slice->cls_ops = ols->ols_lockless_ops; } else { - struct osc_io *oio = osc_env_io(env); - struct cl_io *io = oio->oi_cl.cis_io; - struct cl_object *obj = slice->cls_obj; - struct osc_object *oob = cl2osc(obj); - const struct osc_device *osd = lu2osc_dev(obj->co_lu.lo_dev); - struct obd_connect_data *ocd; - LASSERT(io->ci_lockreq == CILR_MANDATORY || io->ci_lockreq == CILR_MAYBE || io->ci_lockreq == CILR_NEVER); ocd = &class_exp2cliimp(osc_export(oob))->imp_connect_data; ols->ols_locklessable = (io->ci_type != CIT_SETATTR) && - (io->ci_lockreq == CILR_MAYBE) && - (ocd->ocd_connect_flags & OBD_CONNECT_SRVLOCK); + (io->ci_lockreq == CILR_MAYBE) && + (ocd->ocd_connect_flags & + OBD_CONNECT_SRVLOCK); if (io->ci_lockreq == CILR_NEVER || - /* lockless IO */ + /* lockless IO */ (ols->ols_locklessable && osc_object_is_contended(oob)) || - /* lockless truncate */ - (cl_io_is_trunc(io) && - (ocd->ocd_connect_flags & OBD_CONNECT_TRUNCLOCK) && - osd->od_lockless_truncate)) { + /* lockless truncate */ + (cl_io_is_trunc(io) && osd->od_lockless_truncate && + (ocd->ocd_connect_flags & OBD_CONNECT_TRUNCLOCK))) { ols->ols_locklessable = 1; - slice->cls_ops = &osc_lock_lockless_ops; + slice->cls_ops = ols->ols_lockless_ops; } } LASSERT(ergo(ols->ols_glimpse, !osc_lock_is_lockless(ols))); } +EXPORT_SYMBOL(osc_lock_to_lockless); static bool osc_lock_compatible(const struct osc_lock *qing, const struct osc_lock *qed) @@ -832,9 +792,8 @@ static bool osc_lock_compatible(const struct osc_lock *qing, return false; } -static void osc_lock_wake_waiters(const struct lu_env *env, - struct osc_object *osc, - struct osc_lock *oscl) +void osc_lock_wake_waiters(const struct lu_env *env, struct osc_object *osc, + struct osc_lock *oscl) { struct osc_lock *scan; @@ -852,10 +811,10 @@ static void osc_lock_wake_waiters(const struct lu_env *env, } spin_unlock(&oscl->ols_lock); } +EXPORT_SYMBOL(osc_lock_wake_waiters); -static int osc_lock_enqueue_wait(const struct lu_env *env, - struct osc_object *obj, - struct osc_lock *oscl) +int osc_lock_enqueue_wait(const struct lu_env *env, struct osc_object *obj, + struct osc_lock *oscl) { struct osc_lock *tmp_oscl; struct cl_lock_descr *need = &oscl->ols_cl.cls_lock->cll_descr; @@ -909,6 +868,7 @@ static int osc_lock_enqueue_wait(const struct lu_env *env, return rc; } +EXPORT_SYMBOL(osc_lock_enqueue_wait); /** * Implementation of cl_lock_operations::clo_enqueue() method for osc @@ -1082,8 +1042,8 @@ static void osc_lock_detach(const struct lu_env *env, struct osc_lock *olck) * * - cancels ldlm lock (ldlm_cli_cancel()). */ -static void osc_lock_cancel(const struct lu_env *env, - const struct cl_lock_slice *slice) +void osc_lock_cancel(const struct lu_env *env, + const struct cl_lock_slice *slice) { struct osc_object *obj = cl2osc(slice->cls_obj); struct osc_lock *oscl = cl2osc_lock(slice); @@ -1096,9 +1056,10 @@ static void osc_lock_cancel(const struct lu_env *env, osc_lock_wake_waiters(env, obj, oscl); } +EXPORT_SYMBOL(osc_lock_cancel); -static int osc_lock_print(const struct lu_env *env, void *cookie, - lu_printer_t p, const struct cl_lock_slice *slice) +int osc_lock_print(const struct lu_env *env, void *cookie, + lu_printer_t p, const struct cl_lock_slice *slice) { struct osc_lock *lock = cl2osc_lock(slice); @@ -1108,6 +1069,7 @@ static int osc_lock_print(const struct lu_env *env, void *cookie, osc_lvb_print(env, cookie, p, &lock->ols_lvb); return 0; } +EXPORT_SYMBOL(osc_lock_print); static const struct cl_lock_operations osc_lock_ops = { .clo_fini = osc_lock_fini, @@ -1141,9 +1103,8 @@ static void osc_lock_lockless_cancel(const struct lu_env *env, .clo_print = osc_lock_print }; -static void osc_lock_set_writer(const struct lu_env *env, - const struct cl_io *io, - struct cl_object *obj, struct osc_lock *oscl) +void osc_lock_set_writer(const struct lu_env *env, const struct cl_io *io, + struct cl_object *obj, struct osc_lock *oscl) { struct cl_lock_descr *descr = &oscl->ols_cl.cls_lock->cll_descr; pgoff_t io_start; @@ -1172,6 +1133,7 @@ static void osc_lock_set_writer(const struct lu_env *env, oio->oi_write_osclock = oscl; } } +EXPORT_SYMBOL(osc_lock_set_writer); int osc_lock_init(const struct lu_env *env, struct cl_object *obj, struct cl_lock *lock, @@ -1189,6 +1151,7 @@ int osc_lock_init(const struct lu_env *env, INIT_LIST_HEAD(&oscl->ols_waiting_list); INIT_LIST_HEAD(&oscl->ols_wait_entry); INIT_LIST_HEAD(&oscl->ols_nextlock_oscobj); + oscl->ols_lockless_ops = &osc_lock_lockless_ops; /* Speculative lock requests must be either no_expand or glimpse * request (CEF_GLIMPSE). non-glimpse no_expand speculative extent @@ -1228,9 +1191,10 @@ int osc_lock_init(const struct lu_env *env, * Finds an existing lock covering given index and optionally different from a * given \a except lock. */ -struct ldlm_lock *osc_dlmlock_at_pgoff(const struct lu_env *env, - struct osc_object *obj, pgoff_t index, - enum osc_dap_flags dap_flags) +struct ldlm_lock *osc_obj_dlmlock_at_pgoff(const struct lu_env *env, + struct osc_object *obj, + pgoff_t index, + enum osc_dap_flags dap_flags) { struct osc_thread_info *info = osc_env_info(env); struct ldlm_res_id *resname = &info->oti_resname; diff --git a/fs/lustre/osc/osc_object.c b/fs/lustre/osc/osc_object.c index d34e6f2..76e7e33 100644 --- a/fs/lustre/osc/osc_object.c +++ b/fs/lustre/osc/osc_object.c @@ -50,6 +50,16 @@ * Object operations. * */ +static void osc_obj_build_res_name(struct osc_object *osc, + struct ldlm_res_id *resname) +{ + ostid_build_res_name(&osc->oo_oinfo->loi_oi, resname); +} + +static const struct osc_object_operations osc_object_ops = { + .oto_build_res_name = osc_obj_build_res_name, + .oto_dlmlock_at_pgoff = osc_obj_dlmlock_at_pgoff, +}; int osc_object_init(const struct lu_env *env, struct lu_object *obj, const struct lu_object_conf *conf) @@ -78,6 +88,8 @@ int osc_object_init(const struct lu_env *env, struct lu_object *obj, spin_lock_init(&osc->oo_ol_spin); INIT_LIST_HEAD(&osc->oo_ol_list); + LASSERT(osc->oo_obj_ops); + cl_object_page_init(lu2cl(obj), sizeof(struct osc_page)); return 0; @@ -189,7 +201,7 @@ static int osc_object_ast_clear(struct ldlm_lock *lock, void *data) return LDLM_ITER_CONTINUE; } -static int osc_object_prune(const struct lu_env *env, struct cl_object *obj) +int osc_object_prune(const struct lu_env *env, struct cl_object *obj) { struct osc_object *osc = cl2osc(obj); struct ldlm_res_id *resname = &osc_env_info(env)->oti_resname; @@ -197,11 +209,12 @@ static int osc_object_prune(const struct lu_env *env, struct cl_object *obj) /* DLM locks don't hold a reference of osc_object so we have to * clear it before the object is being destroyed. */ - ostid_build_res_name(&osc->oo_oinfo->loi_oi, resname); + osc_build_res_name(osc, resname); ldlm_resource_iterate(osc_export(osc)->exp_obd->obd_namespace, resname, osc_object_ast_clear, osc); return 0; } +EXPORT_SYMBOL(osc_object_prune); static int osc_object_fiemap(const struct lu_env *env, struct cl_object *obj, struct ll_fiemap_info_key *fmkey, @@ -291,18 +304,6 @@ static int osc_object_fiemap(const struct lu_env *env, struct cl_object *obj, return rc; } -void osc_object_set_contended(struct osc_object *obj) -{ - obj->oo_contention_time = jiffies; - /* mb(); */ - obj->oo_contended = 1; -} - -void osc_object_clear_contended(struct osc_object *obj) -{ - obj->oo_contended = 0; -} - int osc_object_is_contended(struct osc_object *obj) { struct osc_device *dev = lu2osc_dev(obj->oo_cl.co_lu.lo_dev); @@ -327,6 +328,7 @@ int osc_object_is_contended(struct osc_object *obj) } return 1; } +EXPORT_SYMBOL(osc_object_is_contended); /** * Implementation of struct cl_object_operations::coo_req_attr_set() for osc @@ -438,6 +440,7 @@ struct lu_object *osc_object_alloc(const struct lu_env *env, lu_object_init(obj, NULL, dev); osc->oo_cl.co_ops = &osc_ops; obj->lo_ops = &osc_lu_obj_ops; + osc->oo_obj_ops = &osc_object_ops; } else { obj = NULL; } diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index 275bd12..7e3f954 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -97,18 +97,6 @@ struct osc_ladvise_args { void *la_cookie; }; -struct osc_enqueue_args { - struct obd_export *oa_exp; - enum ldlm_type oa_type; - enum ldlm_mode oa_mode; - u64 *oa_flags; - osc_enqueue_upcall_f oa_upcall; - void *oa_cookie; - struct ost_lvb *oa_lvb; - struct lustre_handle oa_lockh; - unsigned int oa_speculative; -}; - static void osc_release_ppga(struct brw_page **ppga, u32 count); static int brw_interpret(const struct lu_env *env, struct ptlrpc_request *req, void *data, int rc); @@ -2042,10 +2030,10 @@ static int osc_set_lock_data(struct ldlm_lock *lock, void *data) return set; } -static int osc_enqueue_fini(struct ptlrpc_request *req, - osc_enqueue_upcall_f upcall, void *cookie, - struct lustre_handle *lockh, enum ldlm_mode mode, - u64 *flags, int speculative, int errcode) +int osc_enqueue_fini(struct ptlrpc_request *req, osc_enqueue_upcall_f upcall, + void *cookie, struct lustre_handle *lockh, + enum ldlm_mode mode, u64 *flags, int speculative, + int errcode) { bool intent = *flags & LDLM_FL_HAS_INTENT; int rc; @@ -2077,9 +2065,8 @@ static int osc_enqueue_fini(struct ptlrpc_request *req, return rc; } -static int osc_enqueue_interpret(const struct lu_env *env, - struct ptlrpc_request *req, - struct osc_enqueue_args *aa, int rc) +int osc_enqueue_interpret(const struct lu_env *env, struct ptlrpc_request *req, + struct osc_enqueue_args *aa, int rc) { struct ldlm_lock *lock; struct lustre_handle *lockh = &aa->oa_lockh; diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index bcbe557..57a869f 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -1434,6 +1434,8 @@ enum mdt_reint_cmd { #define MDS_INODELOCK_MAXSHIFT 6 /* This FULL lock is useful to take on unlink sort of operations */ #define MDS_INODELOCK_FULL ((1 << (MDS_INODELOCK_MAXSHIFT + 1)) - 1) +/* DOM lock shouldn't be canceled early, use this macro for ELC */ +#define MDS_INODELOCK_ELC (MDS_INODELOCK_FULL & ~MDS_INODELOCK_DOM) /* NOTE: until Lustre 1.8.7/2.1.1 the fid_ver() was packed into name[2], * but was moved into name[1] along with the OID to avoid consuming the @@ -2153,6 +2155,8 @@ enum ldlm_intent_flags { IT_QUOTA_DQACQ = 0x00000800, IT_QUOTA_CONN = 0x00001000, IT_SETXATTR = 0x00002000, + IT_GLIMPSE = 0x00004000, + IT_BRW = 0x00008000, }; struct ldlm_intent { From patchwork Mon Sep 30 18:55:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167163 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3D710912 for ; Mon, 30 Sep 2019 19:01:08 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 25B7E224EF for ; Mon, 30 Sep 2019 19:01:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 25B7E224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 65A9F5C3E2F; Mon, 30 Sep 2019 11:58:56 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id C09325C3B4D for ; Mon, 30 Sep 2019 11:57:13 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id BE93010055FD; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id BC862B4; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:08 -0400 Message-Id: <1569869810-23848-50-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 049/151] lustre: mdc: add IO stats in mdc X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mikhal Pershin , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mikhal Pershin Add IO statistic into MDC like in OSC WC-bug-id: https://jira.whamcloud.com/browse/LU-3285 Lustre-commit: 2f103489f614 ("LU-3285 mdc: add IO stats in mdc") Signed-off-by: Mikhal Pershin Reviewed-on: https://review.whamcloud.com/28019 Reviewed-by: Jinshan Xiong Reviewed-by: Andreas Dilger Signed-off-by: James Simmons --- fs/lustre/include/lustre_dlm.h | 29 ++++ fs/lustre/include/lustre_osc.h | 2 + fs/lustre/ldlm/ldlm_request.c | 22 +++ fs/lustre/mdc/lproc_mdc.c | 318 ++++++++++++++++++++++++++++++++++++++++- fs/lustre/obdclass/genops.c | 2 + fs/lustre/osc/osc_internal.h | 2 - fs/lustre/osc/osc_page.c | 1 + 7 files changed, 367 insertions(+), 9 deletions(-) diff --git a/fs/lustre/include/lustre_dlm.h b/fs/lustre/include/lustre_dlm.h index 8f92225..feef43a 100644 --- a/fs/lustre/include/lustre_dlm.h +++ b/fs/lustre/include/lustre_dlm.h @@ -1206,6 +1206,35 @@ void ldlm_namespace_free_prior(struct ldlm_namespace *ns, void ldlm_debugfs_setup(void); void ldlm_debugfs_cleanup(void); +static inline void ldlm_svc_get_eopc(const struct ldlm_request *dlm_req, + struct lprocfs_stats *srv_stats) +{ + int lock_type = 0, op = 0; + + lock_type = dlm_req->lock_desc.l_resource.lr_type; + + switch (lock_type) { + case LDLM_PLAIN: + op = PTLRPC_LAST_CNTR + LDLM_PLAIN_ENQUEUE; + break; + case LDLM_EXTENT: + op = PTLRPC_LAST_CNTR + LDLM_EXTENT_ENQUEUE; + break; + case LDLM_FLOCK: + op = PTLRPC_LAST_CNTR + LDLM_FLOCK_ENQUEUE; + break; + case LDLM_IBITS: + op = PTLRPC_LAST_CNTR + LDLM_IBITS_ENQUEUE; + break; + default: + op = 0; + break; + } + + if (op != 0) + lprocfs_counter_incr(srv_stats, op); +} + /* resource.c - internal */ struct ldlm_resource *ldlm_resource_get(struct ldlm_namespace *ns, struct ldlm_resource *parent, diff --git a/fs/lustre/include/lustre_osc.h b/fs/lustre/include/lustre_osc.h index ecca719..2a16197 100644 --- a/fs/lustre/include/lustre_osc.h +++ b/fs/lustre/include/lustre_osc.h @@ -558,6 +558,8 @@ void osc_index2policy(union ldlm_policy_data *policy, void osc_page_submit(const struct lu_env *env, struct osc_page *opg, enum cl_req_type crt, int brw_flags); int lru_queue_work(const struct lu_env *env, void *data); +long osc_lru_shrink(const struct lu_env *env, struct client_obd *cli, + long target, bool force); /* osc_cache.c */ int osc_set_async_flags(struct osc_object *obj, struct osc_page *opg, diff --git a/fs/lustre/ldlm/ldlm_request.c b/fs/lustre/ldlm/ldlm_request.c index 09be016..6089ac4 100644 --- a/fs/lustre/ldlm/ldlm_request.c +++ b/fs/lustre/ldlm/ldlm_request.c @@ -754,6 +754,28 @@ int ldlm_cli_enqueue(struct obd_export *exp, struct ptlrpc_request **reqp, body->lock_flags = ldlm_flags_to_wire(*flags); body->lock_handle[0] = *lockh; + /* extended LDLM opcodes in client stats */ + if (exp->exp_obd->obd_svc_stats != NULL) { + bool glimpse = *flags & LDLM_FL_HAS_INTENT; + + /* OST glimpse has no intent buffer */ + if (req_capsule_has_field(&req->rq_pill, &RMF_LDLM_INTENT, + RCL_CLIENT)) { + struct ldlm_intent *it; + + it = req_capsule_client_get(&req->rq_pill, + &RMF_LDLM_INTENT); + glimpse = (it && (it->opc == IT_GLIMPSE)); + } + + if (!glimpse) + ldlm_svc_get_eopc(body, exp->exp_obd->obd_svc_stats); + else + lprocfs_counter_incr(exp->exp_obd->obd_svc_stats, + PTLRPC_LAST_CNTR + + LDLM_GLIMPSE_ENQUEUE); + } + if (async) { LASSERT(reqp); return 0; diff --git a/fs/lustre/mdc/lproc_mdc.c b/fs/lustre/mdc/lproc_mdc.c index ffc1085..9092a97 100644 --- a/fs/lustre/mdc/lproc_mdc.c +++ b/fs/lustre/mdc/lproc_mdc.c @@ -35,8 +35,175 @@ #include #include #include +#include +#include + #include "mdc_internal.h" +static int mdc_max_dirty_mb_seq_show(struct seq_file *m, void *v) +{ + struct obd_device *dev = m->private; + struct client_obd *cli = &dev->u.cli; + unsigned long val; + + spin_lock(&cli->cl_loi_list_lock); + val = cli->cl_dirty_max_pages >> (20 - PAGE_SHIFT); + spin_unlock(&cli->cl_loi_list_lock); + + seq_printf(m, "%lu\n", val); + return 0; +} + +static ssize_t mdc_max_dirty_mb_seq_write(struct file *file, + const char __user *buffer, + size_t count, loff_t *off) +{ + struct seq_file *sfl = file->private_data; + struct obd_device *dev = sfl->private; + struct client_obd *cli = &dev->u.cli; + __s64 pages_number; + int rc; + + rc = lprocfs_write_frac_u64_helper(buffer, count, &pages_number, + 1 << (20 - PAGE_SHIFT)); + if (rc) + return rc; + + pages_number >>= PAGE_SHIFT; + + if (pages_number <= 0 || + pages_number >= OSC_MAX_DIRTY_MB_MAX << (20 - PAGE_SHIFT) || + pages_number > totalram_pages() / 4) /* 1/4 of RAM */ + return -ERANGE; + + spin_lock(&cli->cl_loi_list_lock); + cli->cl_dirty_max_pages = pages_number; + osc_wake_cache_waiters(cli); + spin_unlock(&cli->cl_loi_list_lock); + + return count; +} +LPROC_SEQ_FOPS(mdc_max_dirty_mb); + +static int mdc_cached_mb_seq_show(struct seq_file *m, void *v) +{ + struct obd_device *dev = m->private; + struct client_obd *cli = &dev->u.cli; + int shift = 20 - PAGE_SHIFT; + + seq_printf(m, "used_mb: %ld\n" + "busy_cnt: %ld\n" + "reclaim: %llu\n", + (atomic_long_read(&cli->cl_lru_in_list) + + atomic_long_read(&cli->cl_lru_busy)) >> shift, + atomic_long_read(&cli->cl_lru_busy), + cli->cl_lru_reclaim); + + return 0; +} + +/* shrink the number of caching pages to a specific number */ +static ssize_t +mdc_cached_mb_seq_write(struct file *file, const char __user *buffer, + size_t count, loff_t *off) +{ + struct seq_file *sfl = file->private_data; + struct obd_device *dev = sfl->private; + struct client_obd *cli = &dev->u.cli; + u64 pages_number; + long rc; + char kernbuf[128]; + + if (count >= sizeof(kernbuf)) + return -EINVAL; + + if (copy_from_user(kernbuf, buffer, count)) + return -EFAULT; + kernbuf[count] = 0; + + buffer += lprocfs_find_named_value(kernbuf, "used_mb:", &count) - + kernbuf; + rc = lprocfs_write_frac_u64_helper(buffer, count, &pages_number, + 1 << (20 - PAGE_SHIFT)); + if (rc) + return rc; + + pages_number >>= PAGE_SHIFT; + + if (pages_number < 0) + return -ERANGE; + + rc = atomic_long_read(&cli->cl_lru_in_list) - pages_number; + if (rc > 0) { + struct lu_env *env; + u16 refcheck; + + env = cl_env_get(&refcheck); + if (!IS_ERR(env)) { + (void)osc_lru_shrink(env, cli, rc, true); + cl_env_put(env, &refcheck); + } + } + + return count; +} +LPROC_SEQ_FOPS(mdc_cached_mb); + +static int mdc_contention_seconds_seq_show(struct seq_file *m, void *v) +{ + struct obd_device *obd = m->private; + struct osc_device *od = obd2osc_dev(obd); + + seq_printf(m, "%u\n", od->od_contention_time); + return 0; +} + +static ssize_t mdc_contention_seconds_seq_write(struct file *file, + const char __user *buffer, + size_t count, loff_t *off) +{ + struct seq_file *sfl = file->private_data; + struct obd_device *obd = sfl->private; + struct osc_device *od = obd2osc_dev(obd); + int rc; + char kernbuf[128]; + s64 val; + + if (count >= sizeof(kernbuf)) + return -EINVAL; + + if (copy_from_user(kernbuf, buffer, count)) + return -EFAULT; + kernbuf[count] = 0; + + rc = kstrtos64(kernbuf, count, &val); + if (rc) + return rc; + if (val < 0 || val > INT_MAX) + return -ERANGE; + + od->od_contention_time = val; + + return count; +} +LPROC_SEQ_FOPS(mdc_contention_seconds); + +static int mdc_unstable_stats_seq_show(struct seq_file *m, void *v) +{ + struct obd_device *dev = m->private; + struct client_obd *cli = &dev->u.cli; + long pages; + int mb; + + pages = atomic_long_read(&cli->cl_unstable_count); + mb = (pages * PAGE_SIZE) >> 20; + + seq_printf(m, "unstable_pages: %20ld\n" + "unstable_mb: %10d\n", pages, mb); + return 0; +} +LPROC_SEQ_FOPS_RO(mdc_unstable_stats); + static ssize_t active_show(struct kobject *kobj, struct attribute *attr, char *buf) { @@ -139,13 +306,6 @@ static ssize_t max_mod_rpcs_in_flight_store(struct kobject *kobj, #define mdc_conn_uuid_show conn_uuid_show LUSTRE_RO_ATTR(mdc_conn_uuid); -static int mdc_rpc_stats_seq_show(struct seq_file *seq, void *v) -{ - struct obd_device *dev = seq->private; - - return obd_mod_rpc_stats_seq_show(&dev->u.cli, seq); -} - static ssize_t mdc_rpc_stats_seq_write(struct file *file, const char __user *buf, size_t len, loff_t *off) @@ -156,10 +316,144 @@ static ssize_t mdc_rpc_stats_seq_write(struct file *file, lprocfs_oh_clear(&cli->cl_mod_rpcs_hist); + lprocfs_oh_clear(&cli->cl_read_rpc_hist); + lprocfs_oh_clear(&cli->cl_write_rpc_hist); + lprocfs_oh_clear(&cli->cl_read_page_hist); + lprocfs_oh_clear(&cli->cl_write_page_hist); + lprocfs_oh_clear(&cli->cl_read_offset_hist); + lprocfs_oh_clear(&cli->cl_write_offset_hist); + return len; } + +#define pct(a, b) (b ? a * 100 / b : 0) +static int mdc_rpc_stats_seq_show(struct seq_file *seq, void *v) +{ + struct obd_device *dev = seq->private; + struct client_obd *cli = &dev->u.cli; + unsigned long read_tot = 0, write_tot = 0, read_cum, write_cum; + int i; + + obd_mod_rpc_stats_seq_show(&dev->u.cli, seq); + + spin_lock(&cli->cl_loi_list_lock); + + seq_printf(seq, "\nread RPCs in flight: %d\n", + cli->cl_r_in_flight); + seq_printf(seq, "write RPCs in flight: %d\n", + cli->cl_w_in_flight); + seq_printf(seq, "pending write pages: %d\n", + atomic_read(&cli->cl_pending_w_pages)); + seq_printf(seq, "pending read pages: %d\n", + atomic_read(&cli->cl_pending_r_pages)); + + seq_puts(seq, "\n\t\t\tread\t\t\twrite\n"); + seq_puts(seq, "pages per rpc rpcs %% cum %% |"); + seq_puts(seq, " rpcs %% cum %%\n"); + + read_tot = lprocfs_oh_sum(&cli->cl_read_page_hist); + write_tot = lprocfs_oh_sum(&cli->cl_write_page_hist); + + read_cum = 0; + write_cum = 0; + for (i = 0; i < OBD_HIST_MAX; i++) { + unsigned long r = cli->cl_read_page_hist.oh_buckets[i]; + unsigned long w = cli->cl_write_page_hist.oh_buckets[i]; + + read_cum += r; + write_cum += w; + seq_printf(seq, "%d:\t\t%10lu %3lu %3lu | %10lu %3lu %3lu\n", + 1 << i, r, pct(r, read_tot), + pct(read_cum, read_tot), w, + pct(w, write_tot), + pct(write_cum, write_tot)); + if (read_cum == read_tot && write_cum == write_tot) + break; + } + + seq_puts(seq, "\n\t\t\tread\t\t\twrite\n"); + seq_puts(seq, "rpcs in flight rpcs %% cum %% |"); + seq_puts(seq, " rpcs %% cum %%\n"); + + read_tot = lprocfs_oh_sum(&cli->cl_read_rpc_hist); + write_tot = lprocfs_oh_sum(&cli->cl_write_rpc_hist); + + read_cum = 0; + write_cum = 0; + for (i = 0; i < OBD_HIST_MAX; i++) { + unsigned long r = cli->cl_read_rpc_hist.oh_buckets[i]; + unsigned long w = cli->cl_write_rpc_hist.oh_buckets[i]; + + read_cum += r; + write_cum += w; + seq_printf(seq, "%d:\t\t%10lu %3lu %3lu | %10lu %3lu %3lu\n", + i, r, pct(r, read_tot), pct(read_cum, read_tot), w, + pct(w, write_tot), pct(write_cum, write_tot)); + if (read_cum == read_tot && write_cum == write_tot) + break; + } + + seq_puts(seq, "\n\t\t\tread\t\t\twrite\n"); + seq_puts(seq, "offset rpcs %% cum %% |"); + seq_puts(seq, " rpcs %% cum %%\n"); + + read_tot = lprocfs_oh_sum(&cli->cl_read_offset_hist); + write_tot = lprocfs_oh_sum(&cli->cl_write_offset_hist); + + read_cum = 0; + write_cum = 0; + for (i = 0; i < OBD_HIST_MAX; i++) { + unsigned long r = cli->cl_read_offset_hist.oh_buckets[i]; + unsigned long w = cli->cl_write_offset_hist.oh_buckets[i]; + + read_cum += r; + write_cum += w; + seq_printf(seq, "%d:\t\t%10lu %3lu %3lu | %10lu %3lu %3lu\n", + (i == 0) ? 0 : 1 << (i - 1), + r, pct(r, read_tot), pct(read_cum, read_tot), + w, pct(w, write_tot), pct(write_cum, write_tot)); + if (read_cum == read_tot && write_cum == write_tot) + break; + } + spin_unlock(&cli->cl_loi_list_lock); + + return 0; +} +#undef pct LPROC_SEQ_FOPS(mdc_rpc_stats); +static int mdc_stats_seq_show(struct seq_file *seq, void *v) +{ + struct timespec64 now; + struct obd_device *dev = seq->private; + struct osc_stats *stats = &obd2osc_dev(dev)->od_stats; + + ktime_get_real_ts64(&now); + + seq_printf(seq, "snapshot_time: %lld.%09lu (secs.nsecs)\n", + (s64)now.tv_sec, now.tv_nsec); + seq_printf(seq, "lockless_write_bytes\t\t%llu\n", + stats->os_lockless_writes); + seq_printf(seq, "lockless_read_bytes\t\t%llu\n", + stats->os_lockless_reads); + seq_printf(seq, "lockless_truncate\t\t%llu\n", + stats->os_lockless_truncates); + return 0; +} + +static ssize_t mdc_stats_seq_write(struct file *file, + const char __user *buf, + size_t len, loff_t *off) +{ + struct seq_file *seq = file->private_data; + struct obd_device *dev = seq->private; + struct osc_stats *stats = &obd2osc_dev(dev)->od_stats; + + memset(stats, 0, sizeof(*stats)); + return len; +} +LPROC_SEQ_FOPS(mdc_stats); + LPROC_SEQ_FOPS_WR_ONLY(mdc, ping); LPROC_SEQ_FOPS_RO_TYPE(mdc, connect_flags); @@ -177,8 +471,14 @@ static ssize_t mdc_rpc_stats_seq_write(struct file *file, .fops = &mdc_connect_flags_fops }, { .name = "mds_server_uuid", .fops = &mdc_server_uuid_fops, }, + { .name = "max_dirty_mb", + .fops = &mdc_max_dirty_mb_fops }, + { .name = "mdc_cached_mb", + .fops = &mdc_cached_mb_fops }, { .name = "timeouts", .fops = &mdc_timeouts_fops }, + { .name = "contention_seconds", + .fops = &mdc_contention_seconds_fops }, { .name = "import", .fops = &mdc_import_fops }, { .name = "state", @@ -187,6 +487,10 @@ static ssize_t mdc_rpc_stats_seq_write(struct file *file, .fops = &mdc_pinger_recov_fops }, { .name = "rpc_stats", .fops = &mdc_rpc_stats_fops }, + { .name = "unstable_stats", + .fops = &mdc_unstable_stats_fops }, + { .name = "mdc_stats", + .fops = &mdc_stats_fops }, { NULL } }; diff --git a/fs/lustre/obdclass/genops.c b/fs/lustre/obdclass/genops.c index 68d267f..1ecceeb 100644 --- a/fs/lustre/obdclass/genops.c +++ b/fs/lustre/obdclass/genops.c @@ -1391,6 +1391,8 @@ int obd_set_max_rpcs_in_flight(struct client_obd *cli, u32 max) spin_lock(&cli->cl_loi_list_lock); old = cli->cl_max_rpcs_in_flight; cli->cl_max_rpcs_in_flight = max; + client_adjust_max_dirty(cli); + diff = max - old; /* We increase the max_rpcs_in_flight, then wakeup some waiters. */ diff --git a/fs/lustre/osc/osc_internal.h b/fs/lustre/osc/osc_internal.h index 8f89443..65f52f9 100644 --- a/fs/lustre/osc/osc_internal.h +++ b/fs/lustre/osc/osc_internal.h @@ -78,8 +78,6 @@ int osc_ladvise_base(struct obd_export *exp, struct obdo *oa, int osc_process_config_base(struct obd_device *obd, struct lustre_cfg *cfg); int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, struct list_head *ext_list, int cmd); -long osc_lru_shrink(const struct lu_env *env, struct client_obd *cli, - long target, bool force); unsigned long osc_lru_reserve(struct client_obd *cli, unsigned long npages); void osc_lru_unreserve(struct client_obd *cli, unsigned long npages); diff --git a/fs/lustre/osc/osc_page.c b/fs/lustre/osc/osc_page.c index 45b18f1..4e41b26 100644 --- a/fs/lustre/osc/osc_page.c +++ b/fs/lustre/osc/osc_page.c @@ -663,6 +663,7 @@ long osc_lru_shrink(const struct lu_env *env, struct client_obd *cli, } return count > 0 ? count : rc; } +EXPORT_SYMBOL(osc_lru_shrink); /** * Reclaim LRU pages by an IO thread. The caller wants to reclaim at least From patchwork Mon Sep 30 18:55:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167157 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5A8AA912 for ; Mon, 30 Sep 2019 19:00:54 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 41BF8224EF for ; Mon, 30 Sep 2019 19:00:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 41BF8224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id EE73B5C40B8; Mon, 30 Sep 2019 11:58:46 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 21BE65C344D for ; Mon, 30 Sep 2019 11:57:14 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id C424C1005643; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id C1EFDB5; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:09 -0400 Message-Id: <1569869810-23848-51-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 050/151] lustre: lov: add Data-on-MDT tests and fixes X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mikhal Pershin , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mikhal Pershin - fixes for DoM code for issues revealed by tests. WC-bug-id: https://jira.whamcloud.com/browse/LU-3285 Lustre-commit: a7625cd2f37a ("LU-3285 test: add Data-on-MDT tests and fixes") Signed-off-by: Mikhal Pershin Reviewed-on: https://review.whamcloud.com/28020 Reviewed-by: Jinshan Xiong Reviewed-by: Andreas Dilger Signed-off-by: James Simmons --- fs/lustre/lov/lov_object.c | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-) diff --git a/fs/lustre/lov/lov_object.c b/fs/lustre/lov/lov_object.c index 186b875..34575dc 100644 --- a/fs/lustre/lov/lov_object.c +++ b/fs/lustre/lov/lov_object.c @@ -431,8 +431,17 @@ static int lov_attr_get_dom(const struct lu_env *env, struct lov_object *lov, return OST_LVB_GET_ERR(loi->loi_lvb.lvb_blocks); cl_lvb2attr(attr, &loi->loi_lvb); - attr->cat_kms = attr->cat_size > loi->loi_kms ? attr->cat_size : - loi->loi_kms; + + /* DoM component size can be bigger than stripe size after + * client's setattr RPC, so do not count anything beyound + * component end. Alternatively, check that limit on server + * and do not allow size overflow there. + */ + if (attr->cat_size > lle->lle_extent.e_end) + attr->cat_size = lle->lle_extent.e_end; + + attr->cat_kms = attr->cat_size; + dom->lo_dom_r0.lo_attr_valid = 1; *lov_attr = attr; @@ -508,7 +517,6 @@ static int lov_init_dom(const struct lu_env *env, struct lov_device *dev, struct cl_device *mdcdev; struct lov_oinfo *loi = NULL; struct cl_object_conf *sconf = <i->lti_stripe_conf; - struct inode *inode = conf->coc_inode; u32 idx = 0; int rc; @@ -538,15 +546,6 @@ static int lov_init_dom(const struct lu_env *env, struct lov_device *dev, return -ENOMEM; fid_to_ostid(lu_object_fid(lov2lu(lov)), &loi->loi_oi); - /* Initialize lvb structure */ - loi->loi_lvb.lvb_mtime = inode->i_mtime.tv_sec; - loi->loi_lvb.lvb_atime = inode->i_atime.tv_sec; - loi->loi_lvb.lvb_ctime = inode->i_ctime.tv_sec; - loi->loi_lvb.lvb_blocks = inode->i_blocks; - loi->loi_lvb.lvb_size = i_size_read(inode); - if (loi->loi_lvb.lvb_size > lsme->lsme_stripe_size) - loi->loi_lvb.lvb_size = lsme->lsme_stripe_size; - loi_kms_set(loi, loi->loi_lvb.lvb_size); sconf->u.coc_oinfo = loi; again: @@ -843,6 +842,12 @@ static int lov_attr_get_composite(const struct lu_env *env, if (!lov_attr) continue; + CDEBUG(D_INODE, + "COMP ID #%i: s=%llu m=%llu a=%llu c=%llu b=%llu\n", + index - 1, lov_attr->cat_size, + lov_attr->cat_mtime, lov_attr->cat_atime, + lov_attr->cat_ctime, lov_attr->cat_blocks); + /* merge results */ attr->cat_blocks += lov_attr->cat_blocks; if (attr->cat_size < lov_attr->cat_size) From patchwork Mon Sep 30 18:55:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167087 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 780A116B1 for ; Mon, 30 Sep 2019 18:58:20 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 60549224D5 for ; Mon, 30 Sep 2019 18:58:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 60549224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id ECBF65C3BCC; Mon, 30 Sep 2019 11:57:43 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 622025C344D for ; Mon, 30 Sep 2019 11:57:14 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id C56521005644; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id C3A15BB; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:10 -0400 Message-Id: <1569869810-23848-52-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 051/151] lustre: mdc: use generic grant code at MDT X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mikhal Pershin , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mikhal Pershin Now that we have files on the MDT server we need to expand the mdc layer to handle grant space. Export osc_reconnect() and osc_disconnect() for the mdc layer. Indicate that we support grants on MDT with the client by adding OBD_CONNECT_GRANT_PARAM. WC--bug-id: https://jira.whamcloud.com/browse/LU-3285 Lustre-commit: 0697cf73c74d ("LU-3285 mdt: use generic grant code at MDT") Signed-off-by: Mikhal Pershin Reviewed-on: https://review.whamcloud.com/28021 Reviewed-by: Andreas Dilger Reviewed-by: Jinshan Xiong Signed-off-by: James Simmons --- fs/lustre/include/lustre_osc.h | 5 ++++- fs/lustre/llite/llite_lib.c | 3 ++- fs/lustre/mdc/mdc_request.c | 3 ++- fs/lustre/osc/osc_request.c | 12 ++++++------ 4 files changed, 14 insertions(+), 9 deletions(-) diff --git a/fs/lustre/include/lustre_osc.h b/fs/lustre/include/lustre_osc.h index 2a16197..3d83fa5 100644 --- a/fs/lustre/include/lustre_osc.h +++ b/fs/lustre/include/lustre_osc.h @@ -647,7 +647,10 @@ int osc_set_info_async(const struct lu_env *env, struct obd_export *exp, struct ptlrpc_request_set *set); int osc_ldlm_resource_invalidate(struct cfs_hash *hs, struct cfs_hash_bd *bd, struct hlist_node *hnode, void *arg); - +int osc_reconnect(const struct lu_env *env, struct obd_export *exp, + struct obd_device *obd, struct obd_uuid *cluuid, + struct obd_connect_data *data, void *localdata); +int osc_disconnect(struct obd_export *exp); int osc_punch_send(struct obd_export *exp, struct obdo *oa, obd_enqueue_update_f upcall, void *cookie); diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index a02ac03..beacaeb 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -205,7 +205,8 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt) OBD_CONNECT_DIR_STRIPE | OBD_CONNECT_BULK_MBITS | OBD_CONNECT_CKSUM | OBD_CONNECT_SUBTREE | - OBD_CONNECT_FLAGS2 | OBD_CONNECT_MULTIMODRPCS; + OBD_CONNECT_MULTIMODRPCS | + OBD_CONNECT_GRANT_PARAM | OBD_CONNECT_FLAGS2; data->ocd_connect_flags2 = 0; diff --git a/fs/lustre/mdc/mdc_request.c b/fs/lustre/mdc/mdc_request.c index 85eb0ec..655f1c2 100644 --- a/fs/lustre/mdc/mdc_request.c +++ b/fs/lustre/mdc/mdc_request.c @@ -2596,7 +2596,8 @@ int mdc_process_config(struct obd_device *obd, u32 len, void *buf) .add_conn = client_import_add_conn, .del_conn = client_import_del_conn, .connect = client_connect_import, - .disconnect = client_disconnect_export, + .reconnect = osc_reconnect, + .disconnect = osc_disconnect, .iocontrol = mdc_iocontrol, .set_info_async = mdc_set_info_async, .statfs = mdc_statfs, diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index 7e3f954..33555ed 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -2633,11 +2633,9 @@ int osc_set_info_async(const struct lu_env *env, struct obd_export *exp, } EXPORT_SYMBOL(osc_set_info_async); -static int osc_reconnect(const struct lu_env *env, - struct obd_export *exp, struct obd_device *obd, - struct obd_uuid *cluuid, - struct obd_connect_data *data, - void *localdata) +int osc_reconnect(const struct lu_env *env, struct obd_export *exp, + struct obd_device *obd, struct obd_uuid *cluuid, + struct obd_connect_data *data, void *localdata) { struct client_obd *cli = &obd->u.cli; @@ -2663,8 +2661,9 @@ static int osc_reconnect(const struct lu_env *env, return 0; } +EXPORT_SYMBOL(osc_reconnect); -static int osc_disconnect(struct obd_export *exp) +int osc_disconnect(struct obd_export *exp) { struct obd_device *obd = class_exp2obd(exp); int rc; @@ -2691,6 +2690,7 @@ static int osc_disconnect(struct obd_export *exp) osc_del_shrink_grant(&obd->u.cli); return rc; } +EXPORT_SYMBOL(osc_disconnect); int osc_ldlm_resource_invalidate(struct cfs_hash *hs, struct cfs_hash_bd *bd, From patchwork Mon Sep 30 18:55:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167155 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A800716B1 for ; Mon, 30 Sep 2019 19:00:53 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9050D224EF for ; Mon, 30 Sep 2019 19:00:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9050D224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 453345C3DB8; Mon, 30 Sep 2019 11:58:50 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id B82ED5C344D for ; Mon, 30 Sep 2019 11:57:14 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id C8D271005645; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id C74A2A9; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:11 -0400 Message-Id: <1569869810-23848-53-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 052/151] lustre: mds: combine DoM bit with other IBITS X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mikhal Pershin , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mikhal Pershin The DoM bit combining with other ibits opens way to several optimization, like getting IO lock in advance along with OPEN, size attribute caching at client and others. WC-bug-id: https://jira.whamcloud.com/browse/LU-3285 Lustre-commit: 238a309ea608 ("LU-3285 mds: combine DoM bit with other IBITS") Signed-off-by: Mikhal Pershin WC-bug-id: https://jira.whamcloud.com/browse/LU-3285 Reviewed-on: https://review.whamcloud.com/28024 Lustre-commit: f8929e6d0f3c ("LU-3285 mdc: remove wrong assertion from mdc_object_ast_clear") Signed-off-by: Mikhal Pershin Reviewed-on: https://review.whamcloud.com/30021 Reviewed-by: Jinshan Xiong Reviewed-by: Andreas Dilger Signed-off-by: James Simmons --- fs/lustre/include/cl_object.h | 2 + fs/lustre/include/lustre_dlm.h | 2 + fs/lustre/ldlm/ldlm_inodebits.c | 22 ++++++++ fs/lustre/ldlm/ldlm_internal.h | 1 + fs/lustre/ldlm/ldlm_lock.c | 2 +- fs/lustre/llite/namei.c | 57 ++++++++++++++++++--- fs/lustre/lov/lov_object.c | 8 ++- fs/lustre/mdc/mdc_dev.c | 94 +++++++++++++++++++++++----------- fs/lustre/mdc/mdc_internal.h | 3 +- fs/lustre/mdc/mdc_locks.c | 50 +++++++++++++++--- fs/lustre/ptlrpc/pack_generic.c | 4 +- fs/lustre/ptlrpc/wiretest.c | 16 +++--- include/uapi/linux/lustre/lustre_idl.h | 7 +-- 13 files changed, 209 insertions(+), 59 deletions(-) diff --git a/fs/lustre/include/cl_object.h b/fs/lustre/include/cl_object.h index c04f6c9..0894b2e 100644 --- a/fs/lustre/include/cl_object.h +++ b/fs/lustre/include/cl_object.h @@ -286,6 +286,8 @@ struct cl_layout { struct lu_buf cl_buf; /** size of layout in lov_mds_md format. */ size_t cl_size; + /** size of DoM component if exists or zero otherwise */ + u32 cl_dom_comp_size; /** Layout generation. */ u32 cl_layout_gen; /** whether layout is a composite one */ diff --git a/fs/lustre/include/lustre_dlm.h b/fs/lustre/include/lustre_dlm.h index feef43a..28b5cfa 100644 --- a/fs/lustre/include/lustre_dlm.h +++ b/fs/lustre/include/lustre_dlm.h @@ -1390,5 +1390,7 @@ static inline int ldlm_extent_contain(const struct ldlm_extent *ex1, return ex1->start <= ex2->start && ex1->end >= ex2->end; } +int ldlm_inodebits_drop(struct ldlm_lock *lock, __u64 to_drop); + #endif /** @} LDLM */ diff --git a/fs/lustre/ldlm/ldlm_inodebits.c b/fs/lustre/ldlm/ldlm_inodebits.c index 2926208..ea63d9d 100644 --- a/fs/lustre/ldlm/ldlm_inodebits.c +++ b/fs/lustre/ldlm/ldlm_inodebits.c @@ -67,3 +67,25 @@ void ldlm_ibits_policy_local_to_wire(const union ldlm_policy_data *lpolicy, memset(wpolicy, 0, sizeof(*wpolicy)); wpolicy->l_inodebits.bits = lpolicy->l_inodebits.bits; } + +int ldlm_inodebits_drop(struct ldlm_lock *lock, __u64 to_drop) +{ + check_res_locked(lock->l_resource); + + /* Just return if there are no conflicting bits */ + if ((lock->l_policy_data.l_inodebits.bits & to_drop) == 0) { + LDLM_WARN(lock, "try to drop unset bits %#llx/%#llx\n", + lock->l_policy_data.l_inodebits.bits, to_drop); + /* nothing to do */ + return 0; + } + + /* remove lock from a skiplist and put in the new place + * according with new inodebits + */ + ldlm_resource_unlink_lock(lock); + lock->l_policy_data.l_inodebits.bits &= ~to_drop; + ldlm_grant_lock_with_skiplist(lock); + return 0; +} +EXPORT_SYMBOL(ldlm_inodebits_drop); diff --git a/fs/lustre/ldlm/ldlm_internal.h b/fs/lustre/ldlm/ldlm_internal.h index 275d823..b9d2e9a 100644 --- a/fs/lustre/ldlm/ldlm_internal.h +++ b/fs/lustre/ldlm/ldlm_internal.h @@ -128,6 +128,7 @@ enum ldlm_desc_ast_t { LDLM_WORK_GL_AST }; +void ldlm_grant_lock_with_skiplist(struct ldlm_lock *lock); void ldlm_grant_lock(struct ldlm_lock *lock, struct list_head *work_list); int ldlm_fill_lvb(struct ldlm_lock *lock, struct req_capsule *pill, enum req_location loc, void *data, int size); diff --git a/fs/lustre/ldlm/ldlm_lock.c b/fs/lustre/ldlm/ldlm_lock.c index 235a137..6b4dd41 100644 --- a/fs/lustre/ldlm/ldlm_lock.c +++ b/fs/lustre/ldlm/ldlm_lock.c @@ -981,7 +981,7 @@ static void ldlm_granted_list_add_lock(struct ldlm_lock *lock, * Add a lock to granted list on a resource maintaining skiplist * correctness. */ -static void ldlm_grant_lock_with_skiplist(struct ldlm_lock *lock) +void ldlm_grant_lock_with_skiplist(struct ldlm_lock *lock) { struct sl_insert_point prev; diff --git a/fs/lustre/llite/namei.c b/fs/lustre/llite/namei.c index 9c9acf2..fb75441 100644 --- a/fs/lustre/llite/namei.c +++ b/fs/lustre/llite/namei.c @@ -177,6 +177,43 @@ int ll_test_inode_by_fid(struct inode *inode, void *opaque) return lu_fid_eq(&ll_i2info(inode)->lli_fid, opaque); } +int ll_dom_lock_cancel(struct inode *inode, struct ldlm_lock *lock) +{ + struct lu_env *env; + struct ll_inode_info *lli = ll_i2info(inode); + struct cl_layout clt = { .cl_layout_gen = 0, }; + int rc; + u16 refcheck; + + if (!lli->lli_clob) + return 0; + + env = cl_env_get(&refcheck); + if (IS_ERR(env)) + return PTR_ERR(env); + + rc = cl_object_layout_get(env, lli->lli_clob, &clt); + if (rc) { + CDEBUG(D_INODE, "Cannot get layout for "DFID"\n", + PFID(ll_inode2fid(inode))); + rc = -ENODATA; + } else if (clt.cl_dom_comp_size == 0) { + CDEBUG(D_INODE, "DOM lock without DOM layout for "DFID"\n", + PFID(ll_inode2fid(inode))); + rc = -EINVAL; + } else { + enum cl_fsync_mode mode; + loff_t end = clt.cl_dom_comp_size - 1; + + mode = ldlm_is_discard_data(lock) ? + CL_FSYNC_DISCARD : CL_FSYNC_LOCAL; + rc = cl_sync_file_range(inode, 0, end, mode, 1); + truncate_inode_pages_range(inode->i_mapping, 0, end); + } + cl_env_put(env, &refcheck); + return rc; +} + int ll_md_blocking_ast(struct ldlm_lock *lock, struct ldlm_lock_desc *desc, void *data, int flag) { @@ -196,11 +233,6 @@ int ll_md_blocking_ast(struct ldlm_lock *lock, struct ldlm_lock_desc *desc, struct inode *inode = ll_inode_from_resource_lock(lock); u64 bits = lock->l_policy_data.l_inodebits.bits; - /* Inode is set to lock->l_resource->lr_lvb_inode - * for mdc - bug 24555 - */ - LASSERT(!lock->l_ast_data); - if (!inode) break; @@ -250,9 +282,22 @@ int ll_md_blocking_ast(struct ldlm_lock *lock, struct ldlm_lock_desc *desc, } if (bits & (MDS_INODELOCK_LOOKUP | MDS_INODELOCK_UPDATE | - MDS_INODELOCK_LAYOUT | MDS_INODELOCK_PERM)) + MDS_INODELOCK_LAYOUT | MDS_INODELOCK_PERM | + MDS_INODELOCK_DOM)) ll_have_md_lock(inode, &bits, LCK_MINMODE); + if (bits & MDS_INODELOCK_DOM) { + rc = ll_dom_lock_cancel(inode, lock); + if (rc < 0) + CDEBUG(D_INODE, "cannot flush DoM data " + DFID": rc = %d\n", + PFID(ll_inode2fid(inode)), rc); + lock_res_and_lock(lock); + ldlm_set_kms_ignore(lock); + unlock_res_and_lock(lock); + bits &= ~MDS_INODELOCK_DOM; + } + if (bits & MDS_INODELOCK_LAYOUT) { struct cl_object_conf conf = { .coc_opc = OBJECT_CONF_INVALIDATE, diff --git a/fs/lustre/lov/lov_object.c b/fs/lustre/lov/lov_object.c index 34575dc..9fcf51b 100644 --- a/fs/lustre/lov/lov_object.c +++ b/fs/lustre/lov/lov_object.c @@ -1869,8 +1869,14 @@ static int lov_object_layout_get(const struct lu_env *env, cl->cl_size = lov_comp_md_size(lsm); cl->cl_layout_gen = lsm->lsm_layout_gen; - cl->cl_is_composite = lsm_is_composite(lsm->lsm_magic); + if (lsm_is_composite(lsm->lsm_magic)) { + struct lov_stripe_md_entry *lsme = lsm->lsm_entries[0]; + cl->cl_is_composite = true; + + if (lsme_is_dom(lsme)) + cl->cl_dom_comp_size = lsme->lsme_extent.e_end; + } rc = lov_lsm_pack(lsm, buf->lb_buf, buf->lb_len); lov_lsm_put(lsm); diff --git a/fs/lustre/mdc/mdc_dev.c b/fs/lustre/mdc/mdc_dev.c index 3a7afab..e28c863 100644 --- a/fs/lustre/mdc/mdc_dev.c +++ b/fs/lustre/mdc/mdc_dev.c @@ -63,16 +63,26 @@ static void mdc_lock_build_einfo(const struct lu_env *env, einfo->ei_cbdata = osc; /* value to be put into ->l_ast_data */ } -static int mdc_set_dom_lock_data(struct ldlm_lock *lock, void *data) +static void mdc_lock_lvb_update(const struct lu_env *env, + struct osc_object *osc, + struct ldlm_lock *dlmlock, + struct ost_lvb *lvb); + +static int mdc_set_dom_lock_data(const struct lu_env *env, + struct ldlm_lock *lock, void *data) { + struct osc_object *obj = data; int set = 0; LASSERT(lock); + LASSERT(lock->l_glimpse_ast == mdc_ldlm_glimpse_ast); lock_res_and_lock(lock); - - if (!lock->l_ast_data) + if (!lock->l_ast_data) { lock->l_ast_data = data; + mdc_lock_lvb_update(env, obj, lock, NULL); + } + if (lock->l_ast_data == data) set = 1; @@ -81,7 +91,8 @@ static int mdc_set_dom_lock_data(struct ldlm_lock *lock, void *data) return set; } -int mdc_dom_lock_match(struct obd_export *exp, struct ldlm_res_id *res_id, +int mdc_dom_lock_match(const struct lu_env *env, struct obd_export *exp, + struct ldlm_res_id *res_id, enum ldlm_type type, union ldlm_policy_data *policy, enum ldlm_mode mode, u64 *flags, void *data, struct lustre_handle *lockh, int unref) @@ -99,7 +110,7 @@ int mdc_dom_lock_match(struct obd_export *exp, struct ldlm_res_id *res_id, struct ldlm_lock *lock = ldlm_handle2lock(lockh); LASSERT(lock); - if (!mdc_set_dom_lock_data(lock, data)) { + if (!mdc_set_dom_lock_data(env, lock, data)) { ldlm_lock_decref(lockh, rc); rc = 0; } @@ -137,8 +148,8 @@ struct ldlm_lock *mdc_dlmlock_at_pgoff(const struct lu_env *env, * VFS and page cache already protect us locally, so lots of readers/ * writers can share a single PW lock. */ - mode = mdc_dom_lock_match(osc_export(obj), resname, LDLM_IBITS, policy, - LCK_PR | LCK_PW, &flags, obj, &lockh, + mode = mdc_dom_lock_match(env, osc_export(obj), resname, LDLM_IBITS, + policy, LCK_PR | LCK_PW, &flags, obj, &lockh, dap_flags & OSC_DAP_FL_CANCELING); if (mode) { lock = ldlm_handle2lock(&lockh); @@ -297,6 +308,7 @@ static int mdc_dlm_blocking_ast0(const struct lu_env *env, dlmlock->l_ast_data = NULL; cl_object_get(obj); } + ldlm_set_kms_ignore(dlmlock); unlock_res_and_lock(dlmlock); /* if l_ast_data is NULL, the dlmlock was enqueued by AGL or @@ -377,10 +389,8 @@ int mdc_ldlm_blocking_ast(struct ldlm_lock *dlmlock, * * Called under lock and resource spin-locks. */ -static void mdc_lock_lvb_update(const struct lu_env *env, - struct osc_object *osc, - struct ldlm_lock *dlmlock, - struct ost_lvb *lvb) +void mdc_lock_lvb_update(const struct lu_env *env, struct osc_object *osc, + struct ldlm_lock *dlmlock, struct ost_lvb *lvb) { struct cl_object *obj = osc2cl(osc); struct lov_oinfo *oinfo = osc->oo_oinfo; @@ -409,9 +419,8 @@ static void mdc_lock_lvb_update(const struct lu_env *env, attr->cat_kms = size; } else { LDLM_DEBUG(dlmlock, - "lock acquired, setting rss=%llu, leaving kms=%llu, end=%llu", - lvb->lvb_size, oinfo->loi_kms, - dlmlock->l_policy_data.l_extent.end); + "lock acquired, setting rss=%llu, leaving kms=%llu", + lvb->lvb_size, oinfo->loi_kms); } } cl_object_attr_update(env, obj, attr, valid); @@ -541,8 +550,9 @@ int mdc_fill_lvb(struct ptlrpc_request *req, struct ost_lvb *lvb) lvb->lvb_mtime = body->mbo_mtime; lvb->lvb_atime = body->mbo_atime; lvb->lvb_ctime = body->mbo_ctime; - lvb->lvb_blocks = body->mbo_blocks; - lvb->lvb_size = body->mbo_size; + lvb->lvb_blocks = body->mbo_dom_blocks; + lvb->lvb_size = body->mbo_dom_size; + return 0; } @@ -643,8 +653,9 @@ int mdc_enqueue_interpret(const struct lu_env *env, struct ptlrpc_request *req, * is excluded from the cluster -- such scenarious make the life difficult, so * release locks just after they are obtained. */ -int mdc_enqueue_send(struct obd_export *exp, struct ldlm_res_id *res_id, - u64 *flags, union ldlm_policy_data *policy, +int mdc_enqueue_send(const struct lu_env *env, struct obd_export *exp, + struct ldlm_res_id *res_id, u64 *flags, + union ldlm_policy_data *policy, struct ost_lvb *lvb, int kms_valid, osc_enqueue_upcall_f upcall, void *cookie, struct ldlm_enqueue_info *einfo, int async) @@ -658,9 +669,6 @@ int mdc_enqueue_send(struct obd_export *exp, struct ldlm_res_id *res_id, u64 match_flags = *flags; int rc; - if (!kms_valid) - goto no_match; - mode = einfo->ei_mode; if (einfo->ei_mode == LCK_PR) mode |= LCK_PW; @@ -676,10 +684,10 @@ int mdc_enqueue_send(struct obd_export *exp, struct ldlm_res_id *res_id, return ELDLM_OK; matched = ldlm_handle2lock(&lockh); - if (!mdc_set_dom_lock_data(matched, einfo->ei_cbdata)) { - ldlm_lock_decref(&lockh, mode); - LDLM_LOCK_PUT(matched); - } else { + if (ldlm_is_kms_ignore(matched)) + goto no_match; + + if (mdc_set_dom_lock_data(env, matched, einfo->ei_cbdata)) { *flags |= LDLM_FL_LVB_READY; /* We already have a lock, and it's referenced. */ @@ -689,9 +697,11 @@ int mdc_enqueue_send(struct obd_export *exp, struct ldlm_res_id *res_id, LDLM_LOCK_PUT(matched); return ELDLM_OK; } +no_match: + ldlm_lock_decref(&lockh, mode); + LDLM_LOCK_PUT(matched); } -no_match: if (*flags & (LDLM_FL_TEST_LOCK | LDLM_FL_MATCH_LOCK)) return -ENOLCK; @@ -828,9 +838,9 @@ static int mdc_lock_enqueue(const struct lu_env *env, fid_build_reg_res_name(lu_object_fid(osc2lu(osc)), resname); mdc_lock_build_policy(env, policy); LASSERT(!oscl->ols_speculative); - result = mdc_enqueue_send(osc_export(osc), resname, &oscl->ols_flags, - policy, &oscl->ols_lvb, - osc->oo_oinfo->loi_kms_valid, + result = mdc_enqueue_send(env, osc_export(osc), resname, + &oscl->ols_flags, policy, + &oscl->ols_lvb, osc->oo_oinfo->loi_kms_valid, upcall, cookie, &oscl->ols_einfo, async); if (result == 0) { if (osc_lock_is_lockless(oscl)) { @@ -1155,6 +1165,30 @@ static int mdc_attr_get(const struct lu_env *env, struct cl_object *obj, return osc_attr_get(env, obj, attr); } +static int mdc_object_ast_clear(struct ldlm_lock *lock, void *data) +{ + if ((!lock->l_ast_data && !ldlm_is_kms_ignore(lock)) || + (lock->l_ast_data == data)) { + lock->l_ast_data = NULL; + ldlm_set_kms_ignore(lock); + } + return LDLM_ITER_CONTINUE; +} + +int mdc_object_prune(const struct lu_env *env, struct cl_object *obj) +{ + struct osc_object *osc = cl2osc(obj); + struct ldlm_res_id *resname = &osc_env_info(env)->oti_resname; + + /* DLM locks don't hold a reference of osc_object so we have to + * clear it before the object is being destroyed. + */ + osc_build_res_name(osc, resname); + ldlm_resource_iterate(osc_export(osc)->exp_obd->obd_namespace, resname, + mdc_object_ast_clear, osc); + return 0; +} + static const struct cl_object_operations mdc_ops = { .coo_page_init = osc_page_init, .coo_lock_init = mdc_lock_init, @@ -1163,7 +1197,7 @@ static int mdc_attr_get(const struct lu_env *env, struct cl_object *obj, .coo_attr_update = osc_attr_update, .coo_glimpse = osc_object_glimpse, .coo_req_attr_set = mdc_req_attr_set, - .coo_prune = osc_object_prune, + .coo_prune = mdc_object_prune, }; static const struct osc_object_operations mdc_object_ops = { diff --git a/fs/lustre/mdc/mdc_internal.h b/fs/lustre/mdc/mdc_internal.h index 6e69dfe..6b282b2 100644 --- a/fs/lustre/mdc/mdc_internal.h +++ b/fs/lustre/mdc/mdc_internal.h @@ -156,5 +156,6 @@ static inline unsigned long hash_x_index(u64 hash, int hash64) extern struct lu_device_type mdc_device_type; int mdc_ldlm_blocking_ast(struct ldlm_lock *dlmlock, struct ldlm_lock_desc *new, void *data, int flag); - +int mdc_ldlm_glimpse_ast(struct ldlm_lock *dlmlock, void *data); +int mdc_fill_lvb(struct ptlrpc_request *req, struct ost_lvb *lvb); #endif diff --git a/fs/lustre/mdc/mdc_locks.c b/fs/lustre/mdc/mdc_locks.c index 4e6928e..9c4dcad 100644 --- a/fs/lustre/mdc/mdc_locks.c +++ b/fs/lustre/mdc/mdc_locks.c @@ -550,6 +550,7 @@ static int mdc_finish_enqueue(struct obd_export *exp, struct ldlm_request *lockreq; struct ldlm_reply *lockrep; struct ldlm_lock *lock; + struct mdt_body *body = NULL; void *lvb_data = NULL; u32 lvb_len = 0; @@ -611,8 +612,6 @@ static int mdc_finish_enqueue(struct obd_export *exp, /* We know what to expect, so we do any byte flipping required here */ if (it_has_reply_body(it)) { - struct mdt_body *body; - body = req_capsule_server_get(pill, &RMF_MDT_BODY); if (!body) { CERROR("Can't swab mdt_body\n"); @@ -699,7 +698,10 @@ static int mdc_finish_enqueue(struct obd_export *exp, * server. - Jinshan */ lock = ldlm_handle2lock(lockh); - if (lock && ldlm_has_layout(lock) && lvb_data && + if (!lock) + return rc; + + if (ldlm_has_layout(lock) && lvb_data && !(lockrep->lock_flags & LDLM_FL_BLOCKED_MASK)) { void *lmm; @@ -708,8 +710,8 @@ static int mdc_finish_enqueue(struct obd_export *exp, lmm = kvzalloc(lvb_len, GFP_NOFS); if (!lmm) { - LDLM_LOCK_PUT(lock); - return -ENOMEM; + rc = -ENOMEM; + goto out_lock; } memcpy(lmm, lvb_data, lvb_len); @@ -725,8 +727,25 @@ static int mdc_finish_enqueue(struct obd_export *exp, if (lmm) kvfree(lmm); } - if (lock) - LDLM_LOCK_PUT(lock); + + if (ldlm_has_dom(lock)) { + LASSERT(lock->l_glimpse_ast == mdc_ldlm_glimpse_ast); + + body = req_capsule_server_get(pill, &RMF_MDT_BODY); + if (!(body->mbo_valid & OBD_MD_DOM_SIZE)) { + LDLM_ERROR(lock, "%s: DoM lock without size.\n", + exp->exp_obd->obd_name); + rc = -EPROTO; + goto out_lock; + } + + LDLM_DEBUG(lock, "DoM lock is returned by: %s, size: %llu", + ldlm_it2str(it->it_op), body->mbo_dom_size); + + rc = mdc_fill_lvb(req, &lock->l_ost_lvb); + } +out_lock: + LDLM_LOCK_PUT(lock); return rc; } @@ -831,6 +850,14 @@ int mdc_enqueue_base(struct obd_export *exp, struct ldlm_enqueue_info *einfo, } } + /* With Data-on-MDT the glimpse callback is needed too. + * It is set here in advance but not in mdc_finish_enqueue() + * to avoid possible races. It is safe to have glimpse handler + * for non-DOM locks and costs nothing. + */ + if (!einfo->ei_cb_gl) + einfo->ei_cb_gl = mdc_ldlm_glimpse_ast; + rc = ldlm_cli_enqueue(exp, &req, einfo, &res_id, policy, &flags, NULL, 0, lvb_type, lockh, 0); if (!it) { @@ -1133,6 +1160,7 @@ int mdc_intent_lock(struct obd_export *exp, struct md_op_data *op_data, .ei_mode = it_to_lock_mode(it), .ei_cb_bl = cb_blocking, .ei_cb_cp = ldlm_completion_ast, + .ei_cb_gl = mdc_ldlm_glimpse_ast, }; struct lustre_handle lockh; int rc = 0; @@ -1258,6 +1286,14 @@ int mdc_intent_getattr_async(struct obd_export *exp, return rc; } + /* With Data-on-MDT the glimpse callback is needed too. + * It is set here in advance but not in mdc_finish_enqueue() + * to avoid possible races. It is safe to have glimpse handler + * for non-DOM locks and costs nothing. + */ + if (!minfo->mi_einfo.ei_cb_gl) + minfo->mi_einfo.ei_cb_gl = mdc_ldlm_glimpse_ast; + rc = ldlm_cli_enqueue(exp, &req, &minfo->mi_einfo, &res_id, &policy, &flags, NULL, 0, LVB_T_NONE, &minfo->mi_lockh, 1); if (rc < 0) { diff --git a/fs/lustre/ptlrpc/pack_generic.c b/fs/lustre/ptlrpc/pack_generic.c index 6d61cc7..a631dde 100644 --- a/fs/lustre/ptlrpc/pack_generic.c +++ b/fs/lustre/ptlrpc/pack_generic.c @@ -1758,8 +1758,8 @@ void lustre_swab_mdt_body(struct mdt_body *b) __swab32s(&b->mbo_uid_h); __swab32s(&b->mbo_gid_h); __swab32s(&b->mbo_projid); - BUILD_BUG_ON(offsetof(typeof(*b), mbo_padding_6) == 0); - BUILD_BUG_ON(offsetof(typeof(*b), mbo_padding_7) == 0); + __swab64s(&b->mbo_dom_size); + __swab64s(&b->mbo_dom_blocks); BUILD_BUG_ON(offsetof(typeof(*b), mbo_padding_8) == 0); BUILD_BUG_ON(offsetof(typeof(*b), mbo_padding_9) == 0); BUILD_BUG_ON(offsetof(typeof(*b), mbo_padding_10) == 0); diff --git a/fs/lustre/ptlrpc/wiretest.c b/fs/lustre/ptlrpc/wiretest.c index 2a38d1e..1092cc2 100644 --- a/fs/lustre/ptlrpc/wiretest.c +++ b/fs/lustre/ptlrpc/wiretest.c @@ -1982,14 +1982,14 @@ void lustre_assert_wire_constants(void) (long long)(int)offsetof(struct mdt_body, mbo_projid)); LASSERTF((int)sizeof(((struct mdt_body *)0)->mbo_projid) == 4, "found %lld\n", (long long)(int)sizeof(((struct mdt_body *)0)->mbo_projid)); - LASSERTF((int)offsetof(struct mdt_body, mbo_padding_6) == 176, "found %lld\n", - (long long)(int)offsetof(struct mdt_body, mbo_padding_6)); - LASSERTF((int)sizeof(((struct mdt_body *)0)->mbo_padding_6) == 8, "found %lld\n", - (long long)(int)sizeof(((struct mdt_body *)0)->mbo_padding_6)); - LASSERTF((int)offsetof(struct mdt_body, mbo_padding_7) == 184, "found %lld\n", - (long long)(int)offsetof(struct mdt_body, mbo_padding_7)); - LASSERTF((int)sizeof(((struct mdt_body *)0)->mbo_padding_7) == 8, "found %lld\n", - (long long)(int)sizeof(((struct mdt_body *)0)->mbo_padding_7)); + LASSERTF((int)offsetof(struct mdt_body, mbo_dom_size) == 176, "found %lld\n", + (long long)(int)offsetof(struct mdt_body, mbo_dom_size)); + LASSERTF((int)sizeof(((struct mdt_body *)0)->mbo_dom_size) == 8, "found %lld\n", + (long long)(int)sizeof(((struct mdt_body *)0)->mbo_dom_size)); + LASSERTF((int)offsetof(struct mdt_body, mbo_dom_blocks) == 184, "found %lld\n", + (long long)(int)offsetof(struct mdt_body, mbo_dom_blocks)); + LASSERTF((int)sizeof(((struct mdt_body *)0)->mbo_dom_blocks) == 8, "found %lld\n", + (long long)(int)sizeof(((struct mdt_body *)0)->mbo_dom_blocks)); LASSERTF((int)offsetof(struct mdt_body, mbo_padding_8) == 192, "found %lld\n", (long long)(int)offsetof(struct mdt_body, mbo_padding_8)); LASSERTF((int)sizeof(((struct mdt_body *)0)->mbo_padding_8) == 8, "found %lld\n", diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index 57a869f..2b585b6 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -1088,6 +1088,7 @@ static inline __u32 lov_mds_md_size(__u16 stripes, __u32 lmm_magic) #define OBD_MD_FLUID (0x00000200ULL) /* user ID */ #define OBD_MD_FLGID (0x00000400ULL) /* group ID */ #define OBD_MD_FLFLAGS (0x00000800ULL) /* flags word */ +#define OBD_MD_DOM_SIZE (0x00001000ULL) /* Data-on-MDT component size */ #define OBD_MD_FLNLINK (0x00002000ULL) /* link count */ #define OBD_MD_FLGENER (0x00004000ULL) /* generation number */ /*#define OBD_MD_FLINLINE (0x00008000ULL) inline data. used until 1.6.5 */ @@ -1547,9 +1548,9 @@ struct mdt_body { __u32 mbo_uid_h; /* high 32-bits of uid, for FUID */ __u32 mbo_gid_h; /* high 32-bits of gid, for FUID */ __u32 mbo_projid; /* also fix lustre_swab_mdt_body */ - __u64 mbo_padding_6; - __u64 mbo_padding_7; - __u64 mbo_padding_8; + __u64 mbo_dom_size; /* size of DOM component */ + __u64 mbo_dom_blocks; /* blocks consumed by DOM component */ + __u64 mbo_padding_8; /* also fix lustre_swab_mdt_body */ __u64 mbo_padding_9; __u64 mbo_padding_10; }; /* 216 */ From patchwork Mon Sep 30 18:55:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167091 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 01D0913BD for ; Mon, 30 Sep 2019 18:58:28 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DE492224D5 for ; Mon, 30 Sep 2019 18:58:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DE492224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8B2D55C3D04; Mon, 30 Sep 2019 11:57:47 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 1B42C5C344D for ; Mon, 30 Sep 2019 11:57:15 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id CC4FD1005646; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id CA55CB4; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:12 -0400 Message-Id: <1569869810-23848-54-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 053/151] lustre: llite: increase whole-file readahead to RPC size X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Andreas Dilger Increase the default whole-file readahead limit to match the current RPC size. That ensures that files smaller than the RPC size will be read in a single round-trip instead of sending multiple smaller RPCs. WC-bug-id: https://jira.whamcloud.com/browse/LU-7990 Lustre-commit: 627d0133d9d7 ("LU-7990 llite: increase whole-file readahead to RPC size") Signed-off-by: Andreas Dilger Reviewed-on: https://review.whamcloud.com/26955 Reviewed-by: Patrick Farrell Reviewed-by: Dmitry Eremin Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/llite_lib.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index beacaeb..07d1568 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -95,8 +95,7 @@ static struct ll_sb_info *ll_init_sbi(void) sbi->ll_ra_info.ra_max_pages_per_file = min(pages / 32, SBI_DEFAULT_READAHEAD_MAX); sbi->ll_ra_info.ra_max_pages = sbi->ll_ra_info.ra_max_pages_per_file; - sbi->ll_ra_info.ra_max_read_ahead_whole_pages = - SBI_DEFAULT_READAHEAD_WHOLE_MAX; + sbi->ll_ra_info.ra_max_read_ahead_whole_pages = -1; ll_generate_random_uuid(uuid); sprintf(sbi->ll_sb_uuid.uuid, "%pU", uuid); @@ -269,6 +268,12 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt) sbi->ll_md_exp->exp_connect_data = *data; + /* Don't change value if it was specified in the config log */ + if (sbi->ll_ra_info.ra_max_read_ahead_whole_pages == -1) + sbi->ll_ra_info.ra_max_read_ahead_whole_pages = + max_t(unsigned long, SBI_DEFAULT_READAHEAD_WHOLE_MAX, + (data->ocd_brw_size >> PAGE_SHIFT)); + err = obd_fid_init(sbi->ll_md_exp->exp_obd, sbi->ll_md_exp, LUSTRE_SEQ_METADATA); if (err) { From patchwork Mon Sep 30 18:55:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167161 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 260DE16B1 for ; Mon, 30 Sep 2019 19:01:07 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0DCA1224EF for ; Mon, 30 Sep 2019 19:01:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0DCA1224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id F3C2E5C3E06; Mon, 30 Sep 2019 11:58:53 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 5C9775C32B4 for ; Mon, 30 Sep 2019 11:57:15 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id CF879100585C; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id CD352BD; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:13 -0400 Message-Id: <1569869810-23848-55-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 054/151] lustre: ldlm: remove liblustre remnants X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Andreas Dilger Remove the last remnants of liblustre in the code. There are still some liblustre remnants in the code (e.g. blocked lock handling for LDLM_FL_CANCEL_ON_BLOCK), but that has more complex semantics and should be removed in a separate patch. WC-bug-id: https://jira.whamcloud.com/browse/LU-9452 Lustre-commit: 82e794e268be ("LU-9452 ldlm: remove MSG_CONNECT_LIBCLIENT support") Signed-off-by: Andreas Dilger Reviewed-on: https://review.whamcloud.com/26972 Reviewed-by: Dmitry Eremin Reviewed-by: James Simmons Reviewed-by: John L. Hammond Reviewed-by: Ben Evans Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_dlm_flags.h | 4 ++-- fs/lustre/llite/glimpse.c | 3 +-- fs/lustre/llite/lcommon_cl.c | 3 +-- fs/lustre/llite/lcommon_misc.c | 3 +-- fs/lustre/obdclass/cl_object.c | 12 +++++------- fs/lustre/ptlrpc/import.c | 12 ++++++------ fs/lustre/ptlrpc/niobuf.c | 2 +- 7 files changed, 17 insertions(+), 22 deletions(-) diff --git a/fs/lustre/include/lustre_dlm_flags.h b/fs/lustre/include/lustre_dlm_flags.h index 2413309..f1f454f 100644 --- a/fs/lustre/include/lustre_dlm_flags.h +++ b/fs/lustre/include/lustre_dlm_flags.h @@ -137,8 +137,8 @@ /** * Immediately cancel such locks when they block some other locks. Send * cancel notification to original lock holder, but expect no reply. This - * is for clients (like liblustre) that cannot be expected to reliably - * response to blocking AST. + * is for clients that cannot be expected to reliably response to blocking + * AST. */ #define LDLM_FL_CANCEL_ON_BLOCK 0x0000000000800000ULL /* bit 23 */ #define ldlm_is_cancel_on_block(_l) LDLM_TEST_FLAG((_l), 1ULL << 23) diff --git a/fs/lustre/llite/glimpse.c b/fs/lustre/llite/glimpse.c index 56a5094..ed5164a 100644 --- a/fs/lustre/llite/glimpse.c +++ b/fs/lustre/llite/glimpse.c @@ -30,8 +30,7 @@ * This file is part of Lustre, http://www.lustre.org/ * Lustre is a trademark of Sun Microsystems, Inc. * - * glimpse code shared between vvp and liblustre (and other Lustre clients in - * the future). + * glimpse code used by vvp (and other Lustre clients in the future). * * Author: Nikita Danilov * Author: Oleg Drokin diff --git a/fs/lustre/llite/lcommon_cl.c b/fs/lustre/llite/lcommon_cl.c index cc8d1b2..e859351 100644 --- a/fs/lustre/llite/lcommon_cl.c +++ b/fs/lustre/llite/lcommon_cl.c @@ -30,8 +30,7 @@ * This file is part of Lustre, http://www.lustre.org/ * Lustre is a trademark of Sun Microsystems, Inc. * - * cl code shared between vvp and liblustre (and other Lustre clients in the - * future). + * cl code used by vvp (and other Lustre clients in the future). * * Author: Nikita Danilov */ diff --git a/fs/lustre/llite/lcommon_misc.c b/fs/lustre/llite/lcommon_misc.c index 75156d8..cafcb4d 100644 --- a/fs/lustre/llite/lcommon_misc.c +++ b/fs/lustre/llite/lcommon_misc.c @@ -30,8 +30,7 @@ * This file is part of Lustre, http://www.lustre.org/ * Lustre is a trademark of Sun Microsystems, Inc. * - * cl code shared between vvp and liblustre (and other Lustre clients in the - * future). + * cl code used by vvp (and other Lustre clients in the future). * */ #define DEBUG_SUBSYSTEM S_LLITE diff --git a/fs/lustre/obdclass/cl_object.c b/fs/lustre/obdclass/cl_object.c index b4ea585..4a40121 100644 --- a/fs/lustre/obdclass/cl_object.c +++ b/fs/lustre/obdclass/cl_object.c @@ -515,18 +515,16 @@ int cl_site_stats_print(const struct cl_site *site, struct seq_file *m) /** * The most efficient way is to store cl_env pointer in task specific - * structures. On Linux, it wont' be easy to use task_struct->journal_info - * because Lustre code may call into other fs which has certain assumptions - * about journal_info. Currently following fields in task_struct are identified - * can be used for this purpose: - * - tux_info: only on RedHat kernel. - * - ... + * structures. On Linux, it isn't easy to use task_struct->journal_info + * because Lustre code may call into other fs during memory reclaim, which + * has certain assumptions about journal_info. There are not currently any + * fields in task_struct that can be used for this purpose. * \note As long as we use task_struct to store cl_env, we assume that once * called into Lustre, we'll never call into the other part of the kernel * which will use those fields in task_struct without explicitly exiting * Lustre. * - * If there's no space in task_struct is available, hash will be used. + * Since there's no space in task_struct is available, hash will be used. * bz20044, bz22683. */ diff --git a/fs/lustre/ptlrpc/import.c b/fs/lustre/ptlrpc/import.c index 3cb9b84..e1fc2ef 100644 --- a/fs/lustre/ptlrpc/import.c +++ b/fs/lustre/ptlrpc/import.c @@ -781,7 +781,7 @@ static int ptlrpc_connect_set_flags(struct obd_import *imp, * of macro arguments */ const char *older = "older than client. Consider upgrading server"; - const char *newer = "newer than client. Consider recompiling application"; + const char *newer = "newer than client. Consider upgrading client"; LCONSOLE_WARN("Server %s version (%d.%d.%d.%d) is much %s (%s)\n", obd2cli_tgt(imp->imp_obd), @@ -1237,14 +1237,14 @@ static int ptlrpc_connect_interpret(const struct lu_env *env, ocd = req_capsule_server_get(&request->rq_pill, &RMF_CONNECT_DATA); + /* Servers are not supposed to refuse connections from + * clients based on version, only connection feature + * flags. We should never see this from llite, but it + * may be useful for debugging in the future. + */ if (ocd && (ocd->ocd_connect_flags & OBD_CONNECT_VERSION) && (ocd->ocd_version != LUSTRE_VERSION_CODE)) { - /* - * Actually servers are only supposed to refuse - * connection from liblustre clients, so we - * should never see this from VFS context - */ LCONSOLE_ERROR_MSG(0x16a, "Server %s version (%d.%d.%d.%d) refused connection from this client with an incompatible version (%s). Client must be recompiled\n", obd2cli_tgt(imp->imp_obd), diff --git a/fs/lustre/ptlrpc/niobuf.c b/fs/lustre/ptlrpc/niobuf.c index f2b7902..7bf78ee 100644 --- a/fs/lustre/ptlrpc/niobuf.c +++ b/fs/lustre/ptlrpc/niobuf.c @@ -256,7 +256,7 @@ int ptlrpc_unregister_bulk(struct ptlrpc_request *req, int async) /* the unlink ensures the callback happens ASAP and is the last * one. If it fails, it must be because completion just happened, - * but we must still wait_event() in this case to give liblustre + * but we must still wait_event() in this case to give lustre * a chance to run client_bulk_callback() */ mdunlink_iterate_helper(desc->bd_mds, desc->bd_md_max_brw); From patchwork Mon Sep 30 18:55:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167165 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 51DE016B1 for ; Mon, 30 Sep 2019 19:01:11 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3A782224EF for ; Mon, 30 Sep 2019 19:01:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3A782224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 6C7AA5C40E3; Mon, 30 Sep 2019 11:58:57 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id B4E525C3A5E for ; Mon, 30 Sep 2019 11:57:15 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id D215E100585D; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id D0DFCB5; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:14 -0400 Message-Id: <1569869810-23848-56-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 055/151] lustre: misc: replace LASSERT() with BUILD_BUG_ON() X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Andreas Dilger Some code consistency checks are being done at runtime with LASSERT() when they could be done at compile time with BUILD_BUG_ON(). This might miss defects introduced into the code if that particular code path is not exercised during testing. Replace LASSERT() with BUILD_BUG_ON() in such cases. WC-bug-id: https://jira.whamcloud.com/browse/LU-10046 Lustre-commit: 558c93dc56dc ("LU-10046 misc: replace LASSERT() with CLASSERT()") Signed-off-by: Andreas Dilger Reviewed-on: https://review.whamcloud.com/29256 Reviewed-by: James Simmons Reviewed-by: Dmitry Eremin Reviewed-by: John L. Hammond Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/dir.c | 6 +++--- fs/lustre/ptlrpc/client.c | 2 +- fs/lustre/ptlrpc/lproc_ptlrpc.c | 1 + 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/fs/lustre/llite/dir.c b/fs/lustre/llite/dir.c index ba53c1e..8968110 100644 --- a/fs/lustre/llite/dir.c +++ b/fs/lustre/llite/dir.c @@ -1229,9 +1229,9 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) BUILD_BUG_ON(sizeof(struct lov_user_md_v3) <= sizeof(struct lov_comp_md_v1)); - LASSERT(sizeof(lumv3) == sizeof(*lumv3p)); - LASSERT(sizeof(lumv3.lmm_objects[0]) == - sizeof(lumv3p->lmm_objects[0])); + BUILD_BUG_ON(sizeof(lumv3) != sizeof(*lumv3p)); + BUILD_BUG_ON(sizeof(lumv3.lmm_objects[0]) != + sizeof(lumv3p->lmm_objects[0])); /* first try with v1 which is smaller than v3 */ if (copy_from_user(lumv1, lumv1p, sizeof(*lumv1))) return -EFAULT; diff --git a/fs/lustre/ptlrpc/client.c b/fs/lustre/ptlrpc/client.c index fc909a8..e0b2b91 100644 --- a/fs/lustre/ptlrpc/client.c +++ b/fs/lustre/ptlrpc/client.c @@ -2934,7 +2934,7 @@ int ptlrpc_replay_req(struct ptlrpc_request *req) LASSERT(req->rq_import->imp_state == LUSTRE_IMP_REPLAY); - LASSERT(sizeof(*aa) <= sizeof(req->rq_async_args)); + BUILD_BUG_ON(sizeof(*aa) > sizeof(req->rq_async_args)); aa = ptlrpc_req_async_args(req); memset(aa, 0, sizeof(*aa)); diff --git a/fs/lustre/ptlrpc/lproc_ptlrpc.c b/fs/lustre/ptlrpc/lproc_ptlrpc.c index bc5dc3f..937a413 100644 --- a/fs/lustre/ptlrpc/lproc_ptlrpc.c +++ b/fs/lustre/ptlrpc/lproc_ptlrpc.c @@ -449,6 +449,7 @@ static void nrs_policy_get_info_locked(struct ptlrpc_nrs_policy *policy, { assert_spin_locked(&policy->pol_nrs->nrs_lock); + BUILD_BUG_ON(sizeof(info->pi_arg) != sizeof(policy->pol_arg)); memcpy(info->pi_name, policy->pol_desc->pd_name, NRS_POL_NAME_MAX); info->pi_fallback = !!(policy->pol_flags & PTLRPC_NRS_FL_FALLBACK); From patchwork Mon Sep 30 18:55:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167169 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 52DA6912 for ; Mon, 30 Sep 2019 19:01:21 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3B75F224EF for ; Mon, 30 Sep 2019 19:01:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3B75F224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id EEBFB5C4121; Mon, 30 Sep 2019 11:59:00 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 057005C3B6E for ; Mon, 30 Sep 2019 11:57:16 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id D6031100585E; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id D3D39A9; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:15 -0400 Message-Id: <1569869810-23848-57-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 056/151] lustre: llite: check layout size after cl_object_layout_get X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mikhal Pershin , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mikhal Pershin Check that layout size is not zero before doing flush for DoM object. Also patch does that: - initializes all values in cl_layout explicitly in lov_object_layout_get(). - changes cl_dom_comp_size to u64 for future needs, e.g. importing an existing filesystem image directly. WC-bug-id: https://jira.whamcloud.com/browse/LU-3285 Lustre-commit: 877669a282ef ("LU-3285 llite: check layout size after cl_object_layout_get") Signed-off-by: Mikhal Pershin Reviewed-on: https://review.whamcloud.com/29810 Reviewed-by: Andreas Dilger Signed-off-by: James Simmons --- fs/lustre/include/cl_object.h | 2 +- fs/lustre/llite/namei.c | 3 +-- fs/lustre/lov/lov_object.c | 4 ++++ 3 files changed, 6 insertions(+), 3 deletions(-) diff --git a/fs/lustre/include/cl_object.h b/fs/lustre/include/cl_object.h index 0894b2e..c0d6452 100644 --- a/fs/lustre/include/cl_object.h +++ b/fs/lustre/include/cl_object.h @@ -287,7 +287,7 @@ struct cl_layout { /** size of layout in lov_mds_md format. */ size_t cl_size; /** size of DoM component if exists or zero otherwise */ - u32 cl_dom_comp_size; + u64 cl_dom_comp_size; /** Layout generation. */ u32 cl_layout_gen; /** whether layout is a composite one */ diff --git a/fs/lustre/llite/namei.c b/fs/lustre/llite/namei.c index fb75441..355a500 100644 --- a/fs/lustre/llite/namei.c +++ b/fs/lustre/llite/namei.c @@ -197,10 +197,9 @@ int ll_dom_lock_cancel(struct inode *inode, struct ldlm_lock *lock) CDEBUG(D_INODE, "Cannot get layout for "DFID"\n", PFID(ll_inode2fid(inode))); rc = -ENODATA; - } else if (clt.cl_dom_comp_size == 0) { + } else if (clt.cl_size == 0 || clt.cl_dom_comp_size == 0) { CDEBUG(D_INODE, "DOM lock without DOM layout for "DFID"\n", PFID(ll_inode2fid(inode))); - rc = -EINVAL; } else { enum cl_fsync_mode mode; loff_t end = clt.cl_dom_comp_size - 1; diff --git a/fs/lustre/lov/lov_object.c b/fs/lustre/lov/lov_object.c index 9fcf51b..3a7b38d 100644 --- a/fs/lustre/lov/lov_object.c +++ b/fs/lustre/lov/lov_object.c @@ -1869,6 +1869,7 @@ static int lov_object_layout_get(const struct lu_env *env, cl->cl_size = lov_comp_md_size(lsm); cl->cl_layout_gen = lsm->lsm_layout_gen; + cl->cl_dom_comp_size = 0; if (lsm_is_composite(lsm->lsm_magic)) { struct lov_stripe_md_entry *lsme = lsm->lsm_entries[0]; @@ -1876,7 +1877,10 @@ static int lov_object_layout_get(const struct lu_env *env, if (lsme_is_dom(lsme)) cl->cl_dom_comp_size = lsme->lsme_extent.e_end; + } else { + cl->cl_is_composite = false; } + rc = lov_lsm_pack(lsm, buf->lb_buf, buf->lb_len); lov_lsm_put(lsm); From patchwork Mon Sep 30 18:55:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167205 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E359B16C1 for ; Mon, 30 Sep 2019 19:03:22 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CC123224F0 for ; Mon, 30 Sep 2019 19:03:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CC123224F0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 80EBF21F4FF; Mon, 30 Sep 2019 11:59:39 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 5AC0A5C3B64 for ; Mon, 30 Sep 2019 11:57:16 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id D80DB100585F; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id D6C1EBB; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:16 -0400 Message-Id: <1569869810-23848-58-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 057/151] lustre: mdc: implement own mdc_io_fsync_start() X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mikhal Pershin , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mikhal Pershin DoM lock cancellation may happen in llite and call cl_sync_file_range() function to flush related data and it uses DOM component end as limit for data to be flushed. However related lock and extent are expanded to EOF and this is asserted in osc_cache_writeback_range(). To avoid this a MDC uses own version of cio_start for FSYNC and osc_cache_writeback_range() is called on whole DoM object no matter what start/end are supplied by upper layers. WC-bug-id: https://jira.whamcloud.com/browse/LU-3285 Lustre-commit: 1d2e9f42027c ("LU-3285 mdc: implement own mdc_io_fsync_start()") Signed-off-by: Mikhal Pershin Reviewed-on: https://review.whamcloud.com/29813 Reviewed-by: Andreas Dilger Reviewed-by: Jinshan Xiong Signed-off-by: James Simmons --- fs/lustre/include/lustre_osc.h | 4 ++-- fs/lustre/mdc/mdc_dev.c | 38 +++++++++++++++++++++++++++++++++++++- fs/lustre/osc/osc_io.c | 6 +++--- 3 files changed, 42 insertions(+), 6 deletions(-) diff --git a/fs/lustre/include/lustre_osc.h b/fs/lustre/include/lustre_osc.h index 3d83fa5..c3b8849 100644 --- a/fs/lustre/include/lustre_osc.h +++ b/fs/lustre/include/lustre_osc.h @@ -676,8 +676,8 @@ int osc_io_read_start(const struct lu_env *env, int osc_io_write_start(const struct lu_env *env, const struct cl_io_slice *slice); void osc_io_end(const struct lu_env *env, const struct cl_io_slice *slice); -int osc_io_fsync_start(const struct lu_env *env, - const struct cl_io_slice *slice); +int osc_fsync_ost(const struct lu_env *env, struct osc_object *obj, + struct cl_fsync_io *fio); void osc_io_fsync_end(const struct lu_env *env, const struct cl_io_slice *slice); void osc_read_ahead_release(const struct lu_env *env, void *cbdata); diff --git a/fs/lustre/mdc/mdc_dev.c b/fs/lustre/mdc/mdc_dev.c index e28c863..de21d7d 100644 --- a/fs/lustre/mdc/mdc_dev.c +++ b/fs/lustre/mdc/mdc_dev.c @@ -1045,6 +1045,42 @@ static int mdc_io_read_ahead(const struct lu_env *env, return 0; } +int mdc_io_fsync_start(const struct lu_env *env, + const struct cl_io_slice *slice) +{ + struct cl_io *io = slice->cis_io; + struct cl_fsync_io *fio = &io->u.ci_fsync; + struct cl_object *obj = slice->cis_obj; + struct osc_object *osc = cl2osc(obj); + int result = 0; + + /* a MDC lock always covers whole object, do sync for whole + * possible range despite of supplied start/end values. + */ + result = osc_cache_writeback_range(env, osc, 0, CL_PAGE_EOF, 0, + fio->fi_mode == CL_FSYNC_DISCARD); + if (result > 0) { + fio->fi_nr_written += result; + result = 0; + } + if (fio->fi_mode == CL_FSYNC_ALL) { + int rc; + + rc = osc_cache_wait_range(env, osc, 0, CL_PAGE_EOF); + if (result == 0) + result = rc; + /* Use OSC sync code because it is asynchronous. + * It is to be added into MDC and avoid the using of + * OST_SYNC at both MDC and MDT. + */ + rc = osc_fsync_ost(env, osc, fio); + if (result == 0) + result = rc; + } + + return result; +} + static struct cl_io_operations mdc_io_ops = { .op = { [CIT_READ] = { @@ -1076,7 +1112,7 @@ static int mdc_io_read_ahead(const struct lu_env *env, .cio_end = osc_io_end, }, [CIT_FSYNC] = { - .cio_start = osc_io_fsync_start, + .cio_start = mdc_io_fsync_start, .cio_end = osc_io_fsync_end, }, }, diff --git a/fs/lustre/osc/osc_io.c b/fs/lustre/osc/osc_io.c index 0a67089..d2e2f7f 100644 --- a/fs/lustre/osc/osc_io.c +++ b/fs/lustre/osc/osc_io.c @@ -750,8 +750,8 @@ int osc_io_write_start(const struct lu_env *env, } EXPORT_SYMBOL(osc_io_write_start); -static int osc_fsync_ost(const struct lu_env *env, struct osc_object *obj, - struct cl_fsync_io *fio) +int osc_fsync_ost(const struct lu_env *env, struct osc_object *obj, + struct cl_fsync_io *fio) { struct osc_io *oio = osc_env_io(env); struct obdo *oa = &oio->oi_oa; @@ -775,6 +775,7 @@ static int osc_fsync_ost(const struct lu_env *env, struct osc_object *obj, rc = osc_sync_base(obj, oa, osc_async_upcall, cbargs, PTLRPCD_SET); return rc; } +EXPORT_SYMBOL(osc_fsync_ost); int osc_io_fsync_start(const struct lu_env *env, const struct cl_io_slice *slice) @@ -815,7 +816,6 @@ int osc_io_fsync_start(const struct lu_env *env, return result; } -EXPORT_SYMBOL(osc_io_fsync_start); void osc_io_fsync_end(const struct lu_env *env, const struct cl_io_slice *slice) From patchwork Mon Sep 30 18:55:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167085 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CC5D413BD for ; Mon, 30 Sep 2019 18:58:19 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B4367224D5 for ; Mon, 30 Sep 2019 18:58:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B4367224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 4E9FA5C3CD0; Mon, 30 Sep 2019 11:57:43 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id B006D5C3B72 for ; Mon, 30 Sep 2019 11:57:16 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id DB14F1005860; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id D9CF7B4; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:17 -0400 Message-Id: <1569869810-23848-59-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 058/151] lustre: ldlm: migrate the rest of the code to 64 bit time X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: James Simmons , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" Replace the last cfs_time_current_sec() to avoid the overflow issues in 2038 with ktime_get_real_seconds(). Reduce the jiffies usage to the bare minimum which is useage for mod_timer() and schedule_timeout(). This makes the ldlm totally 64 bit time compliant. WC-bug-id: https://jira.whamcloud.com/browse/LU-9019 Lustre-commit: e920be681451 ("LU-9019 ldlm: migrate the rest of the code to 64 bit time") Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/29295 Reviewed-by: Dmitry Eremin Reviewed-by: Andreas Dilger Reviewed-by: John L. Hammond Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_dlm.h | 4 ++-- fs/lustre/include/lustre_net.h | 11 +++++------ fs/lustre/ldlm/ldlm_lock.c | 10 +++++----- fs/lustre/ldlm/ldlm_lockd.c | 2 +- fs/lustre/ldlm/ldlm_request.c | 14 +++++++------- fs/lustre/ldlm/ldlm_resource.c | 8 +++----- fs/lustre/ptlrpc/client.c | 4 ++-- fs/lustre/ptlrpc/import.c | 2 +- 8 files changed, 26 insertions(+), 29 deletions(-) diff --git a/fs/lustre/include/lustre_dlm.h b/fs/lustre/include/lustre_dlm.h index 28b5cfa..fbcd981 100644 --- a/fs/lustre/include/lustre_dlm.h +++ b/fs/lustre/include/lustre_dlm.h @@ -418,7 +418,7 @@ struct ldlm_namespace { * \see ldlm_namespace_dump. Increased by 10 seconds every time * it is called. */ - unsigned long ns_next_dump; + time64_t ns_next_dump; /** * LVB operations for this namespace. @@ -754,7 +754,7 @@ struct ldlm_lock { * under this lock. * \see ost_rw_prolong_locks */ - unsigned long l_callback_timeout; + time64_t l_callback_timeout; /** Local PID of process which created this lock. */ u32 l_pid; diff --git a/fs/lustre/include/lustre_net.h b/fs/lustre/include/lustre_net.h index a0a974c..8500db5 100644 --- a/fs/lustre/include/lustre_net.h +++ b/fs/lustre/include/lustre_net.h @@ -534,9 +534,9 @@ struct ptlrpc_cli_req { /** For bulk requests on client only: bulk descriptor */ struct ptlrpc_bulk_desc *cr_bulk; /** optional time limit for send attempts */ - long cr_delay_limit; + time64_t cr_delay_limit; /** time request was first queued */ - unsigned long cr_queued_time; + time64_t cr_queued_time; /** request sent in nanosseconds */ ktime_t cr_sent_ns; /** time for request really sent out */ @@ -867,8 +867,9 @@ struct ptlrpc_request { /** * service time estimate (secs) * If the request is not served by this time, it is marked as timed out. + * Do not change to time64_t since this is transmitted over the wire. */ - int rq_timeout; + time_t rq_timeout; /** * when request/reply sent (secs), or time when request should be sent */ @@ -2221,10 +2222,8 @@ static inline int ptlrpc_req_get_repsize(struct ptlrpc_request *req) static inline int ptlrpc_send_limit_expired(struct ptlrpc_request *req) { if (req->rq_delay_limit != 0 && - time_before(req->rq_queued_time + req->rq_delay_limit * HZ, - jiffies)) { + req->rq_queued_time + req->rq_delay_limit < ktime_get_seconds()) return 1; - } return 0; } diff --git a/fs/lustre/ldlm/ldlm_lock.c b/fs/lustre/ldlm/ldlm_lock.c index 6b4dd41..ef6c5e8 100644 --- a/fs/lustre/ldlm/ldlm_lock.c +++ b/fs/lustre/ldlm/ldlm_lock.c @@ -1996,7 +1996,7 @@ void _ldlm_lock_debug(struct ldlm_lock *lock, if (!resource) { libcfs_debug_msg(msgdata, - "%pV ns: \?\? lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: \?\? rrc=\?\? type: \?\?\? flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu lvb_type: %d\n", + "%pV ns: \?\? lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: \?\? rrc=\?\? type: \?\?\? flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lld lvb_type: %d\n", &vaf, lock, lock->l_handle.h_cookie, @@ -2016,7 +2016,7 @@ void _ldlm_lock_debug(struct ldlm_lock *lock, switch (resource->lr_type) { case LDLM_EXTENT: libcfs_debug_msg(msgdata, - "%pV ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " rrc: %d type: %s [%llu->%llu] (req %llu->%llu) flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu lvb_type: %d\n", + "%pV ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " rrc: %d type: %s [%llu->%llu] (req %llu->%llu) flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lld lvb_type: %d\n", &vaf, ldlm_lock_to_ns_name(lock), lock, lock->l_handle.h_cookie, @@ -2040,7 +2040,7 @@ void _ldlm_lock_debug(struct ldlm_lock *lock, case LDLM_FLOCK: libcfs_debug_msg(msgdata, - "%pV ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " rrc: %d type: %s pid: %d [%llu->%llu] flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu\n", + "%pV ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " rrc: %d type: %s pid: %d [%llu->%llu] flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lld\n", &vaf, ldlm_lock_to_ns_name(lock), lock, lock->l_handle.h_cookie, @@ -2062,7 +2062,7 @@ void _ldlm_lock_debug(struct ldlm_lock *lock, case LDLM_IBITS: libcfs_debug_msg(msgdata, - "%pV ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " bits %#llx rrc: %d type: %s flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu lvb_type: %d\n", + "%pV ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " bits %#llx rrc: %d type: %s flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lld lvb_type: %d\n", &vaf, ldlm_lock_to_ns_name(lock), lock, lock->l_handle.h_cookie, @@ -2083,7 +2083,7 @@ void _ldlm_lock_debug(struct ldlm_lock *lock, default: libcfs_debug_msg(msgdata, - "%pV ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " rrc: %d type: %s flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu lvb_type: %d\n", + "%pV ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " rrc: %d type: %s flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lld lvb_type: %d\n", &vaf, ldlm_lock_to_ns_name(lock), lock, lock->l_handle.h_cookie, diff --git a/fs/lustre/ldlm/ldlm_lockd.c b/fs/lustre/ldlm/ldlm_lockd.c index 83f5a22..2c21e7d 100644 --- a/fs/lustre/ldlm/ldlm_lockd.c +++ b/fs/lustre/ldlm/ldlm_lockd.c @@ -160,7 +160,7 @@ static void ldlm_handle_cp_callback(struct ptlrpc_request *req, LDLM_DEBUG(lock, "client completion callback handler START"); if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_CANCEL_BL_CB_RACE)) { - int to = HZ; + long to = HZ; while (to > 0) { schedule_timeout_interruptible(to); diff --git a/fs/lustre/ldlm/ldlm_request.c b/fs/lustre/ldlm/ldlm_request.c index 6089ac4..13b323a 100644 --- a/fs/lustre/ldlm/ldlm_request.c +++ b/fs/lustre/ldlm/ldlm_request.c @@ -110,16 +110,16 @@ static void ldlm_expired_completion_wait(struct ldlm_lock *lock, u32 conn_cnt) struct obd_device *obd; if (!lock->l_conn_export) { - static unsigned long next_dump, last_dump; + static time64_t next_dump, last_dump; LDLM_ERROR(lock, "lock timed out (enqueued at %lld, %llds ago); not entering recovery in server code, just going back to sleep", (s64)lock->l_last_activity, (s64)(ktime_get_real_seconds() - lock->l_last_activity)); - if (time_after(jiffies, next_dump)) { + if (ktime_get_seconds() > next_dump) { last_dump = next_dump; - next_dump = jiffies + 300 * HZ; + next_dump = ktime_get_seconds() + 300; ldlm_namespace_dump(D_DLMTRACE, ldlm_lock_to_ns(lock)); if (last_dump == 0) @@ -150,9 +150,9 @@ static void ldlm_expired_completion_wait(struct ldlm_lock *lock, u32 conn_cnt) /* We use the same basis for both server side and client side functions * from a single node. */ -static unsigned int ldlm_cp_timeout(struct ldlm_lock *lock) +static time64_t ldlm_cp_timeout(struct ldlm_lock *lock) { - unsigned int timeout; + time64_t timeout; if (AT_OFF) return obd_timeout; @@ -163,7 +163,7 @@ static unsigned int ldlm_cp_timeout(struct ldlm_lock *lock) * doesn't respond reasonably, and then give us the lock. */ timeout = at_get(ldlm_lock_to_ns_at(lock)); - return max(3 * timeout, ldlm_enqueue_min); + return max(3 * timeout, (time64_t) ldlm_enqueue_min); } /** @@ -218,7 +218,7 @@ int ldlm_completion_ast(struct ldlm_lock *lock, u64 flags, void *data) /* XXX ALLOCATE - 160 bytes */ struct obd_device *obd; struct obd_import *imp = NULL; - u32 timeout; + time64_t timeout; u32 conn_cnt = 0; int rc = 0; diff --git a/fs/lustre/ldlm/ldlm_resource.c b/fs/lustre/ldlm/ldlm_resource.c index bf8abd4..de0ae1a 100644 --- a/fs/lustre/ldlm/ldlm_resource.c +++ b/fs/lustre/ldlm/ldlm_resource.c @@ -274,7 +274,6 @@ static ssize_t lru_max_age_store(struct kobject *kobj, struct attribute *attr, int scale = NSEC_PER_MSEC; unsigned long long tmp; char *buf; - int err; /* Did the user ask in seconds or milliseconds. Default is in ms */ buf = strstr(buffer, "ms"); @@ -287,8 +286,7 @@ static ssize_t lru_max_age_store(struct kobject *kobj, struct attribute *attr, if (buf) *buf = '\0'; - err = kstrtoull(buffer, 10, &tmp); - if (err != 0) + if (kstrtoull(buffer, 10, &tmp)) return -EINVAL; ns->ns_max_age = ktime_set(0, tmp * scale); @@ -1283,14 +1281,14 @@ void ldlm_namespace_dump(int level, struct ldlm_namespace *ns) CDEBUG(level, "--- Namespace: %s (rc: %d, side: client)\n", ldlm_ns_name(ns), atomic_read(&ns->ns_bref)); - if (time_before(jiffies, ns->ns_next_dump)) + if (ktime_get_seconds() < ns->ns_next_dump) return; cfs_hash_for_each_nolock(ns->ns_rs_hash, ldlm_res_hash_dump, (void *)(unsigned long)level, 0); spin_lock(&ns->ns_lock); - ns->ns_next_dump = jiffies + 10 * HZ; + ns->ns_next_dump = ktime_get_seconds() + 10; spin_unlock(&ns->ns_lock); } diff --git a/fs/lustre/ptlrpc/client.c b/fs/lustre/ptlrpc/client.c index e0b2b91..da7077a 100644 --- a/fs/lustre/ptlrpc/client.c +++ b/fs/lustre/ptlrpc/client.c @@ -1084,7 +1084,7 @@ void ptlrpc_set_add_req(struct ptlrpc_request_set *set, list_add_tail(&req->rq_set_chain, &set->set_requests); req->rq_set = set; atomic_inc(&set->set_remaining); - req->rq_queued_time = jiffies; + req->rq_queued_time = ktime_get_seconds(); if (req->rq_reqmsg) lustre_msg_set_jobid(req->rq_reqmsg, NULL); @@ -1115,7 +1115,7 @@ void ptlrpc_set_add_new_req(struct ptlrpcd_ctl *pc, spin_lock(&set->set_new_req_lock); /* The set takes over the caller's request reference. */ req->rq_set = set; - req->rq_queued_time = jiffies; + req->rq_queued_time = ktime_get_seconds(); list_add_tail(&req->rq_set_chain, &set->set_new_requests); count = atomic_inc_return(&set->set_new_count); spin_unlock(&set->set_new_req_lock); diff --git a/fs/lustre/ptlrpc/import.c b/fs/lustre/ptlrpc/import.c index e1fc2ef..32aceb9 100644 --- a/fs/lustre/ptlrpc/import.c +++ b/fs/lustre/ptlrpc/import.c @@ -717,7 +717,7 @@ int ptlrpc_connect_import(struct obd_import *imp) lustre_msg_add_op_flags(request->rq_reqmsg, MSG_CONNECT_TRANSNO); - DEBUG_REQ(D_RPCTRACE, request, "(re)connect request (timeout %d)", + DEBUG_REQ(D_RPCTRACE, request, "(re)connect request (timeout %ld)", request->rq_timeout); ptlrpcd_add_req(request); rc = 0; From patchwork Mon Sep 30 18:55:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167173 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5536D16B1 for ; Mon, 30 Sep 2019 19:01:30 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3D937224EF for ; Mon, 30 Sep 2019 19:01:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3D937224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 838855C4150; Mon, 30 Sep 2019 11:59:04 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 11E0C5C3AC4 for ; Mon, 30 Sep 2019 11:57:17 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id DF9B91005861; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id DD1BEB5; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:18 -0400 Message-Id: <1569869810-23848-60-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 059/151] lustre: llite: sync bdi sysfs name with lustre sysfs tree X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: James Simmons , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" When the linux kernel introduced super_setup_bdi_name() the port for the lustre client was done which ends up creating sysfs files of the format /sys/devices/virtual/bdi/lustre-1 but their is not way to correlate this with the naming of lustre sysfs objects. Update the bdi handle to use the naming method applied to the llite sysfs tree. WC-bug-id: https://jira.whamcloud.com/browse/LU-9558 Lustre-commit: 1defa62a19e7 ("LU-9558 llite: port lustre to unified handling of bdi") Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/28511 Reviewed-by: Dmitry Eremin Reviewed-by: Andreas Dilger Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/llite_lib.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index 07d1568..b48b23e 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -945,7 +945,6 @@ int ll_fill_super(struct super_block *sb) char *ptr; int len; int err; - static atomic_t ll_bdi_num = ATOMIC_INIT(0); CDEBUG(D_VFSTRACE, "VFS Op: sb %p\n", sb); @@ -970,8 +969,7 @@ int ll_fill_super(struct super_block *sb) if (err) goto out_free; - err = super_setup_bdi_name(sb, "lustre-%d", - atomic_inc_return(&ll_bdi_num)); + err = super_setup_bdi_name(sb, "lustre-%p", sb); if (err) goto out_free; From patchwork Mon Sep 30 18:55:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167177 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 18FB0912 for ; Mon, 30 Sep 2019 19:01:42 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0159A224EF for ; Mon, 30 Sep 2019 19:01:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0159A224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 0306A21FAC5; Mon, 30 Sep 2019 11:59:07 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 530065C3B7D for ; Mon, 30 Sep 2019 11:57:17 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id E2E501005863; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id E0174BD; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:19 -0400 Message-Id: <1569869810-23848-61-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 060/151] lustre: lov: allow lov.*.stripe{size, count}=-1 param X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Andreas Dilger Both the lov stripecount and stripeoffset sysfs files allows the value of "-1". Setting it to "-1" to indicate that the filesystem-wide default value should be used. Update the ld_default_* setting to the proper signed type. WC-bug-id: https://jira.whamcloud.com/browse/LU-9611 Lustre-commit: f93276d9a5b5 ("LU-9611 lov: allow lov.*.stripe{size,count}=-1 param") Signed-off-by: Andreas Dilger Reviewed-on: https://review.whamcloud.com/27946 Reviewed-by: James Simmons Reviewed-by: Jian Yu Reviewed-by: John L. Hammond Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/lov/lproc_lov.c | 2 +- fs/lustre/obdclass/obd_config.c | 6 +++--- include/uapi/linux/lustre/lustre_idl.h | 6 +++--- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/fs/lustre/lov/lproc_lov.c b/fs/lustre/lov/lproc_lov.c index cfb2744..4fecd4e 100644 --- a/fs/lustre/lov/lproc_lov.c +++ b/fs/lustre/lov/lproc_lov.c @@ -75,7 +75,7 @@ static ssize_t stripeoffset_show(struct kobject *kobj, struct attribute *attr, obd_kset.kobj); struct lov_desc *desc = &dev->u.lov.desc; - return sprintf(buf, "%llu\n", desc->ld_default_stripe_offset); + return sprintf(buf, "%lld\n", desc->ld_default_stripe_offset); } static ssize_t stripeoffset_store(struct kobject *kobj, struct attribute *attr, diff --git a/fs/lustre/obdclass/obd_config.c b/fs/lustre/obdclass/obd_config.c index 26b3e01..1485cc7 100644 --- a/fs/lustre/obdclass/obd_config.c +++ b/fs/lustre/obdclass/obd_config.c @@ -1226,14 +1226,14 @@ int class_process_proc_param(char *prefix, struct lprocfs_vars *lvars, /* rc = -EINVAL; continue parsing other params */ skip++; } else if (rc < 0) { - CERROR("%s: error writing proc entry '%s': rc = %d\n", + CERROR("%s: error writing parameter '%s': rc = %d\n", prefix, var->name, rc); rc = 0; } else { - CDEBUG(D_CONFIG, "%s.%.*s: Set parameter %.*s=%s\n", + CDEBUG(D_CONFIG, "%s.%.*s: set parameter %.*s\n", lustre_cfg_string(lcfg, 0), (int)strlen(prefix) - 1, prefix, - (int)(sval - key - 1), key, sval); + (int)(sval - key - 1), key); } } diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index 2b585b6..2181386 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -2035,10 +2035,10 @@ enum fld_op { struct lov_desc { __u32 ld_tgt_count; /* how many OBD's */ __u32 ld_active_tgt_count; /* how many active */ - __u32 ld_default_stripe_count; /* how many objects are used */ + __s32 ld_default_stripe_count; /* how many objects are used */ __u32 ld_pattern; /* default PATTERN_RAID0 */ - __u64 ld_default_stripe_size; /* in bytes */ - __u64 ld_default_stripe_offset; /* in bytes */ + __u64 ld_default_stripe_size; /* in bytes */ + __s64 ld_default_stripe_offset; /* starting OST index */ __u32 ld_padding_0; /* unused */ __u32 ld_qos_maxage; /* in second */ __u32 ld_padding_1; /* also fix lustre_swab_lov_desc */ From patchwork Mon Sep 30 18:55:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167095 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 90B3316C1 for ; Mon, 30 Sep 2019 18:58:34 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 790A7224D5 for ; Mon, 30 Sep 2019 18:58:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 790A7224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 7EA8F5C3D33; Mon, 30 Sep 2019 11:57:50 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id A98755C3B7D for ; Mon, 30 Sep 2019 11:57:17 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id E44481005864; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id E32F7A9; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:20 -0400 Message-Id: <1569869810-23848-62-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 061/151] lustre: brw: add short io osc/ost transfer. X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Patrick Farrell There's no need to do target bulk io for small amount of data, and it requires extra network operations. For this case we add short i/o. When the i/o size is less than or equal to some number of pages (default 3), we encapsulate the data in the ptlrpc request. With this patch, 4k direct i/o read latency on a Cray Aries network (data is on flash on another node on the Aries) drops from ~280 microseconds to ~200 microseconds. Write latency drops from ~370 microseconds to ~350 microseconds (much more of write latency is waiting for write commit). This translates to about a 25-30% performance improvement on 4k direct i/o reads and 4k random reads. (Write performance improvement was small to non-existent.) Improvement was similar with 8k i/o. Buffered sequential i/o sees no improvement, because it does not perform small i/os. Performance data: access = file-per-process pattern = segmented (1 segment) ordering in a file = random offsets ordering inter file= no tasks offsets xfersize = 4096 bytes blocksize = 100 MiB nprocs xfsize shortio dio random Read (MB/s) 1 4k no yes no 15.0 8 4k no yes no 73.4 16 4k no yes no 81.1 1 4k yes yes no 16.5 8 4k yes yes no 95.2 16 4k yes yes no 107.3 1 4k no no yes 15.5 8 4k no no yes 73.4 16 4k no no yes 81.2 1 4k yes no yes 16.8 8 4k yes no yes 95.0 16 4k yes no yes 106.5 Note even when individual i/o performance is not improved, this change reduces the # of network operations required for small i/o, which can help on large systems. WC-bug-id: https://jira.whamcloud.com/browse/LU-1757 Cray-bug-id: LUS-187 Lustre-commit: 70f092a05878 ("LU-1757 LU-1757 brw: add short io osc/ost transfer") Signed-off-by: Patrick Farrell Reviewed-on: https://review.whamcloud.com/27767 Lustre-commit: 3483e195314b ("LU-1757 brw: Fix short i/o and enable for mdc") Signed-off-by: Patrick Farrell Reviewed-on: https://review.whamcloud.com/30435 WC-bug-id: https://jira.whamcloud.com/browse/LU-8066 Lustre-commit: 32fb31f3bf3d ("LU-8066 osc: move suitable values from procfs to sysfs") Reviewed-on: https://review.whamcloud.com/30962 Reviewed-by: Alexey Lyashkov Reviewed-by: Alexandr Boyko Reviewed-by: Dmitry Eremin Reviewed-by: Andreas Dilger Reviewed-by: Mike Pershin Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lprocfs_status.h | 4 ++ fs/lustre/include/lustre_export.h | 7 ++ fs/lustre/include/lustre_net.h | 37 +++++++++- fs/lustre/include/lustre_osc.h | 12 ++++ fs/lustre/include/lustre_req_layout.h | 1 + fs/lustre/include/obd.h | 1 + fs/lustre/llite/llite_lib.c | 6 +- fs/lustre/obdclass/lprocfs_status.c | 55 +++++++++++++++ fs/lustre/osc/lproc_osc.c | 3 + fs/lustre/osc/osc_page.c | 48 ++++++++++--- fs/lustre/osc/osc_request.c | 129 ++++++++++++++++++++++++++++------ fs/lustre/ptlrpc/layout.c | 10 ++- 12 files changed, 272 insertions(+), 41 deletions(-) diff --git a/fs/lustre/include/lprocfs_status.h b/fs/lustre/include/lprocfs_status.h index 815ca37..e923673 100644 --- a/fs/lustre/include/lprocfs_status.h +++ b/fs/lustre/include/lprocfs_status.h @@ -590,6 +590,10 @@ ssize_t max_pages_per_rpc_show(struct kobject *kobj, struct attribute *attr, char *buf); ssize_t max_pages_per_rpc_store(struct kobject *kobj, struct attribute *attr, const char *buffer, size_t count); +ssize_t short_io_bytes_show(struct kobject *kobj, struct attribute *attr, + char *buf); +ssize_t short_io_bytes_store(struct kobject *kobj, struct attribute *attr, + const char *buffer, size_t count); struct root_squash_info; int lprocfs_wr_root_squash(const char __user *buffer, unsigned long count, diff --git a/fs/lustre/include/lustre_export.h b/fs/lustre/include/lustre_export.h index ed0664b..4ac996b 100644 --- a/fs/lustre/include/lustre_export.h +++ b/fs/lustre/include/lustre_export.h @@ -245,6 +245,13 @@ static inline bool imp_connect_disp_stripe(struct obd_import *imp) return ocd->ocd_connect_flags & OBD_CONNECT_DISP_STRIPE; } +static inline bool imp_connect_shortio(struct obd_import *imp) +{ + struct obd_connect_data *ocd = &imp->imp_connect_data; + + return ocd->ocd_connect_flags & OBD_CONNECT_SHORTIO; +} + static inline int exp_connect_lockahead_old(struct obd_export *exp) { return !!(exp_connect_flags(exp) & OBD_CONNECT_LOCKAHEAD_OLD); diff --git a/fs/lustre/include/lustre_net.h b/fs/lustre/include/lustre_net.h index 8500db5..200422f 100644 --- a/fs/lustre/include/lustre_net.h +++ b/fs/lustre/include/lustre_net.h @@ -273,9 +273,41 @@ #define MDS_MAXREQSIZE (5 * 1024) /* >= 4736 */ /** + * OST_IO_MAXREQSIZE ~= + * lustre_msg + ptlrpc_body + obdo + obd_ioobj + + * DT_MAX_BRW_PAGES * niobuf_remote + * + * - single object with 16 pages is 512 bytes + * - OST_IO_MAXREQSIZE must be at least 1 page of cookies plus some spillover + * - Must be a multiple of 1024 + */ +#define _OST_MAXREQSIZE_BASE (sizeof(struct lustre_msg) + \ + sizeof(struct ptlrpc_body) + \ + sizeof(struct obdo) + \ + sizeof(struct obd_ioobj) + \ + sizeof(struct niobuf_remote)) +#define _OST_MAXREQSIZE_SUM (_OST_MAXREQSIZE_BASE + \ + sizeof(struct niobuf_remote) * \ + (DT_MAX_BRW_PAGES - 1)) + +/** * FIEMAP request can be 4K+ for now */ #define OST_MAXREQSIZE (16 * 1024) +#define OST_IO_MAXREQSIZE max_t(int, OST_MAXREQSIZE, \ + (((_OST_MAXREQSIZE_SUM - 1) | (1024 - 1)) + 1)) + +/* Safe estimate of free space in standard RPC, provides upper limit for # of + * bytes of i/o to pack in RPC (skipping bulk transfer). + */ +#define OST_SHORT_IO_SPACE (OST_IO_MAXREQSIZE - _OST_MAXREQSIZE_BASE) + +/* Actual size used for short i/o buffer. Calculation means this: + * At least one page (for large PAGE_SIZE), or 16 KiB, but not more + * than the available space aligned to a page boundary. + */ +#define OBD_MAX_SHORT_IO_BYTES (min(max(PAGE_SIZE, 16UL * 1024UL), \ + OST_SHORT_IO_SPACE & PAGE_MASK)) /* Macro to hide a typecast. */ #define ptlrpc_req_async_args(req) ((void *)&req->rq_async_args) @@ -1758,13 +1790,12 @@ static inline int ptlrpc_client_bulk_active(struct ptlrpc_request *req) int rc; desc = req->rq_bulk; + if (!desc) + return 0; if (req->rq_bulk_deadline > ktime_get_real_seconds()) return 1; - if (!desc) - return 0; - spin_lock(&desc->bd_lock); rc = desc->bd_md_count; spin_unlock(&desc->bd_lock); diff --git a/fs/lustre/include/lustre_osc.h b/fs/lustre/include/lustre_osc.h index c3b8849..895c1cb 100644 --- a/fs/lustre/include/lustre_osc.h +++ b/fs/lustre/include/lustre_osc.h @@ -535,6 +535,18 @@ struct osc_page { unsigned long ops_submit_time; }; +struct osc_brw_async_args { + struct obdo *aa_oa; + int aa_requested_nob; + int aa_nio_count; + u32 aa_page_count; + int aa_resends; + struct brw_page **aa_ppga; + struct client_obd *aa_cli; + struct list_head aa_oaps; + struct list_head aa_exts; +}; + extern struct kmem_cache *osc_lock_kmem; extern struct kmem_cache *osc_object_kmem; extern struct kmem_cache *osc_thread_kmem; diff --git a/fs/lustre/include/lustre_req_layout.h b/fs/lustre/include/lustre_req_layout.h index 57ac618..c255648 100644 --- a/fs/lustre/include/lustre_req_layout.h +++ b/fs/lustre/include/lustre_req_layout.h @@ -297,6 +297,7 @@ void req_capsule_shrink(struct req_capsule *pill, extern struct req_msg_field RMF_FIEMAP_KEY; extern struct req_msg_field RMF_FIEMAP_VAL; extern struct req_msg_field RMF_OST_ID; +extern struct req_msg_field RMF_SHORT_IO; /* MGS config read message format */ extern struct req_msg_field RMF_MGS_CONFIG_BODY; diff --git a/fs/lustre/include/obd.h b/fs/lustre/include/obd.h index 62a8ea9..2f586cb 100644 --- a/fs/lustre/include/obd.h +++ b/fs/lustre/include/obd.h @@ -248,6 +248,7 @@ struct client_obd { atomic_t cl_pending_r_pages; u32 cl_max_pages_per_rpc; u32 cl_max_rpcs_in_flight; + u32 cl_short_io_bytes; struct obd_histogram cl_read_rpc_hist; struct obd_histogram cl_write_rpc_hist; struct obd_histogram cl_read_page_hist; diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index b48b23e..12a68873 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -205,7 +205,8 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt) OBD_CONNECT_BULK_MBITS | OBD_CONNECT_CKSUM | OBD_CONNECT_SUBTREE | OBD_CONNECT_MULTIMODRPCS | - OBD_CONNECT_GRANT_PARAM | OBD_CONNECT_FLAGS2; + OBD_CONNECT_GRANT_PARAM | + OBD_CONNECT_SHORTIO | OBD_CONNECT_FLAGS2; data->ocd_connect_flags2 = 0; @@ -396,7 +397,8 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt) OBD_CONNECT_JOBSTATS | OBD_CONNECT_LVB_TYPE | OBD_CONNECT_LAYOUTLOCK | OBD_CONNECT_PINGLESS | OBD_CONNECT_LFSCK | - OBD_CONNECT_BULK_MBITS | OBD_CONNECT_FLAGS2; + OBD_CONNECT_BULK_MBITS | OBD_CONNECT_SHORTIO | + OBD_CONNECT_FLAGS2; /* The client currently advertises support for OBD_CONNECT_LOCKAHEAD_OLD * so it can interoperate with an older version of lockahead which was diff --git a/fs/lustre/obdclass/lprocfs_status.c b/fs/lustre/obdclass/lprocfs_status.c index 1ff12d55..a9617e5 100644 --- a/fs/lustre/obdclass/lprocfs_status.c +++ b/fs/lustre/obdclass/lprocfs_status.c @@ -1868,3 +1868,58 @@ ssize_t max_pages_per_rpc_store(struct kobject *kobj, struct attribute *attr, return count; } EXPORT_SYMBOL(max_pages_per_rpc_store); + +ssize_t short_io_bytes_show(struct kobject *kobj, struct attribute *attr, + char *buf) +{ + struct obd_device *dev = container_of(kobj, struct obd_device, + obd_kset.kobj); + struct client_obd *cli = &dev->u.cli; + int rc; + + spin_lock(&cli->cl_loi_list_lock); + rc = sprintf(buf, "%d\n", cli->cl_short_io_bytes); + spin_unlock(&cli->cl_loi_list_lock); + return rc; +} +EXPORT_SYMBOL(short_io_bytes_show); + +/* Used to catch people who think they're specifying pages. */ +#define MIN_SHORT_IO_BYTES 64 + +ssize_t short_io_bytes_store(struct kobject *kobj, struct attribute *attr, + const char *buffer, size_t count) +{ + struct obd_device *dev = container_of(kobj, struct obd_device, + obd_kset.kobj); + struct client_obd *cli = &dev->u.cli; + u32 val; + int rc; + + rc = lprocfs_climp_check(dev); + if (rc) + return rc; + + rc = kstrtouint(buffer, 0, &val); + if (rc) + goto out; + + if (val > OBD_MAX_SHORT_IO_BYTES || val < MIN_SHORT_IO_BYTES) { + rc = -ERANGE; + goto out; + } + + rc = count; + + spin_lock(&cli->cl_loi_list_lock); + if (val > (cli->cl_max_pages_per_rpc << PAGE_SHIFT)) + rc = -ERANGE; + else + cli->cl_short_io_bytes = val; + spin_unlock(&cli->cl_loi_list_lock); + +out: + up_read(&dev->u.cli.cl_sem); + return rc; +} +EXPORT_SYMBOL(short_io_bytes_store); diff --git a/fs/lustre/osc/lproc_osc.c b/fs/lustre/osc/lproc_osc.c index 2a57982..2f122a2 100644 --- a/fs/lustre/osc/lproc_osc.c +++ b/fs/lustre/osc/lproc_osc.c @@ -573,7 +573,9 @@ static ssize_t destroys_in_flight_show(struct kobject *kobj, atomic_read(&obd->u.cli.cl_destroy_in_flight)); } LUSTRE_RO_ATTR(destroys_in_flight); + LUSTRE_RW_ATTR(max_pages_per_rpc); +LUSTRE_RW_ATTR(short_io_bytes); static int osc_unstable_stats_seq_show(struct seq_file *m, void *v) { @@ -807,6 +809,7 @@ void lproc_osc_attach_seqstat(struct obd_device *dev) &lustre_attr_max_dirty_mb.attr, &lustre_attr_max_pages_per_rpc.attr, &lustre_attr_max_rpcs_in_flight.attr, + &lustre_attr_short_io_bytes.attr, &lustre_attr_resend_count.attr, &lustre_attr_ost_conn_uuid.attr, NULL, diff --git a/fs/lustre/osc/osc_page.c b/fs/lustre/osc/osc_page.c index 4e41b26..836cb14 100644 --- a/fs/lustre/osc/osc_page.c +++ b/fs/lustre/osc/osc_page.c @@ -858,17 +858,28 @@ void osc_lru_unreserve(struct client_obd *cli, unsigned long npages) * are likely from the same page zone. */ static inline void unstable_page_accounting(struct ptlrpc_bulk_desc *desc, + struct osc_brw_async_args *aa, int factor) { - int page_count = desc->bd_iov_count; + int page_count; pg_data_t *last = NULL; int count = 0; int i; - LASSERT(ptlrpc_is_bulk_desc_kiov(desc->bd_type)); + if (desc) { + LASSERT(ptlrpc_is_bulk_desc_kiov(desc->bd_type)); + page_count = desc->bd_iov_count; + } else { + page_count = aa->aa_page_count; + } for (i = 0; i < page_count; i++) { - pg_data_t *pgdat = page_pgdat(BD_GET_KIOV(desc, i).bv_page); + pg_data_t *pgdat; + + if (desc) + pgdat = page_pgdat(BD_GET_KIOV(desc, i).bv_page); + else + pgdat = page_pgdat(aa->aa_ppga[i]->pg); if (likely(pgdat == last)) { ++count; @@ -887,14 +898,16 @@ static inline void unstable_page_accounting(struct ptlrpc_bulk_desc *desc, mod_node_page_state(last, NR_UNSTABLE_NFS, factor * count); } -static inline void add_unstable_page_accounting(struct ptlrpc_bulk_desc *desc) +static inline void add_unstable_page_accounting(struct ptlrpc_bulk_desc *desc, + struct osc_brw_async_args *aa) { - unstable_page_accounting(desc, 1); + unstable_page_accounting(desc, aa, 1); } -static inline void dec_unstable_page_accounting(struct ptlrpc_bulk_desc *desc) +static inline void dec_unstable_page_accounting(struct ptlrpc_bulk_desc *desc, + struct osc_brw_async_args *aa) { - unstable_page_accounting(desc, -1); + unstable_page_accounting(desc, aa, -1); } /** @@ -910,13 +923,20 @@ static inline void dec_unstable_page_accounting(struct ptlrpc_bulk_desc *desc) */ void osc_dec_unstable_pages(struct ptlrpc_request *req) { + struct osc_brw_async_args *aa = (void *)&req->rq_async_args; struct client_obd *cli = &req->rq_import->imp_obd->u.cli; struct ptlrpc_bulk_desc *desc = req->rq_bulk; - int page_count = desc->bd_iov_count; + int page_count; long unstable_count; + if (desc) + page_count = desc->bd_iov_count; + else + page_count = aa->aa_page_count; + LASSERT(page_count >= 0); - dec_unstable_page_accounting(desc); + + dec_unstable_page_accounting(desc, aa); unstable_count = atomic_long_sub_return(page_count, &cli->cl_unstable_count); @@ -937,15 +957,21 @@ void osc_dec_unstable_pages(struct ptlrpc_request *req) */ void osc_inc_unstable_pages(struct ptlrpc_request *req) { + struct osc_brw_async_args *aa = (void *)&req->rq_async_args; struct client_obd *cli = &req->rq_import->imp_obd->u.cli; struct ptlrpc_bulk_desc *desc = req->rq_bulk; - long page_count = desc->bd_iov_count; + long page_count; /* No unstable page tracking */ if (!cli->cl_cache || !cli->cl_cache->ccc_unstable_check) return; - add_unstable_page_accounting(desc); + if (desc) + page_count = desc->bd_iov_count; + else + page_count = aa->aa_page_count; + + add_unstable_page_accounting(desc, aa); atomic_long_add(page_count, &cli->cl_unstable_count); atomic_long_add(page_count, &cli->cl_cache->ccc_unstable_nr); diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index 33555ed..e164d6a 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -62,18 +62,6 @@ static unsigned int osc_reqpool_mem_max = 5; module_param(osc_reqpool_mem_max, uint, 0444); -struct osc_brw_async_args { - struct obdo *aa_oa; - int aa_requested_nob; - int aa_nio_count; - u32 aa_page_count; - int aa_resends; - struct brw_page **aa_ppga; - struct client_obd *aa_cli; - struct list_head aa_oaps; - struct list_head aa_exts; -}; - struct osc_async_args { struct obd_info *aa_oi; }; @@ -1010,7 +998,8 @@ static int check_write_rcs(struct ptlrpc_request *req, } } - if (req->rq_bulk->bd_nob_transferred != requested_nob) { + if (req->rq_bulk && + req->rq_bulk->bd_nob_transferred != requested_nob) { CERROR("Unexpected # bytes transferred: %d (requested %d)\n", req->rq_bulk->bd_nob_transferred, requested_nob); return -EPROTO; @@ -1111,10 +1100,11 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli, struct ost_body *body; struct obd_ioobj *ioobj; struct niobuf_remote *niobuf; - int niocount, i, requested_nob, opc, rc; + int niocount, i, requested_nob, opc, rc, short_io_size = 0; struct osc_brw_async_args *aa; struct req_capsule *pill; struct brw_page *pg_prev; + void *short_io_buf; if (OBD_FAIL_CHECK(OBD_FAIL_OSC_BRW_PREP_REQ)) return -ENOMEM; /* Recoverable */ @@ -1144,6 +1134,20 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli, req_capsule_set_size(pill, &RMF_NIOBUF_REMOTE, RCL_CLIENT, niocount * sizeof(*niobuf)); + for (i = 0; i < page_count; i++) + short_io_size += pga[i]->count; + + /* Check if we can do a short io. */ + if (!(short_io_size <= cli->cl_short_io_bytes && niocount == 1 && + imp_connect_shortio(cli->cl_import))) + short_io_size = 0; + + req_capsule_set_size(pill, &RMF_SHORT_IO, RCL_CLIENT, + opc == OST_READ ? 0 : short_io_size); + if (opc == OST_READ) + req_capsule_set_size(pill, &RMF_SHORT_IO, RCL_SERVER, + short_io_size); + rc = ptlrpc_request_pack(req, LUSTRE_OST_VERSION, opc); if (rc) { ptlrpc_request_free(req); @@ -1152,11 +1156,18 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli, osc_set_io_portal(req); ptlrpc_at_set_req_timeout(req); + /* ask ptlrpc not to resend on EINPROGRESS since BRWs have their own * retry logic */ req->rq_no_retry_einprogress = 1; + if (short_io_size != 0) { + desc = NULL; + short_io_buf = NULL; + goto no_bulk; + } + desc = ptlrpc_prep_bulk_imp( req, page_count, cli->cl_import->imp_connect_data.ocd_brw_size >> LNET_MTU_BITS, @@ -1169,7 +1180,7 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli, goto out; } /* NB request now owns desc and will free it when it gets freed */ - +no_bulk: body = req_capsule_client_get(pill, &RMF_OST_BODY); ioobj = req_capsule_client_get(pill, &RMF_OBD_IOOBJ); niobuf = req_capsule_client_get(pill, &RMF_NIOBUF_REMOTE); @@ -1185,7 +1196,26 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli, * "max - 1" for old client compatibility sending "0", and also so the * the actual maximum is a power-of-two number, not one less. LU-1431 */ - ioobj_max_brw_set(ioobj, desc->bd_md_max_brw); + if (desc) + ioobj_max_brw_set(ioobj, desc->bd_md_max_brw); + else /* short i/o */ + ioobj_max_brw_set(ioobj, 0); + + if (short_io_size != 0) { + if ((body->oa.o_valid & OBD_MD_FLFLAGS) == 0) { + body->oa.o_valid |= OBD_MD_FLFLAGS; + body->oa.o_flags = 0; + } + body->oa.o_flags |= OBD_FL_SHORT_IO; + CDEBUG(D_CACHE, "Using short io for data transfer, size = %d\n", + short_io_size); + if (opc == OST_WRITE) { + short_io_buf = req_capsule_client_get(pill, + &RMF_SHORT_IO); + LASSERT(short_io_buf); + } + } + LASSERT(page_count > 0); pg_prev = pga[0]; for (requested_nob = i = 0; i < page_count; i++, niobuf++) { @@ -1210,7 +1240,17 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli, LASSERT((pga[0]->flag & OBD_BRW_SRVLOCK) == (pg->flag & OBD_BRW_SRVLOCK)); - desc->bd_frag_ops->add_kiov_frag(desc, pg->pg, poff, pg->count); + if (short_io_size != 0 && opc == OST_WRITE) { + unsigned char *ptr = kmap_atomic(pg->pg); + + LASSERT(short_io_size >= requested_nob + pg->count); + memcpy(short_io_buf + requested_nob, ptr + poff, + pg->count); + kunmap_atomic(ptr); + } else if (short_io_size == 0) { + desc->bd_frag_ops->add_kiov_frag(desc, pg->pg, poff, + pg->count); + } requested_nob += pg->count; if (i > 0 && can_merge_pages(pg_prev, pg)) { @@ -1477,7 +1517,8 @@ static int osc_brw_fini_request(struct ptlrpc_request *req, int rc) } LASSERT(req->rq_bulk->bd_nob == aa->aa_requested_nob); - if (sptlrpc_cli_unwrap_bulk_write(req, req->rq_bulk)) + if (req->rq_bulk && + sptlrpc_cli_unwrap_bulk_write(req, req->rq_bulk)) return -EAGAIN; if ((aa->aa_oa->o_valid & OBD_MD_FLCKSUM) && client_cksum && @@ -1493,8 +1534,14 @@ static int osc_brw_fini_request(struct ptlrpc_request *req, int rc) /* The rest of this function executes only for OST_READs */ - /* if unwrap_bulk failed, return -EAGAIN to retry */ - rc = sptlrpc_cli_unwrap_bulk_read(req, req->rq_bulk, rc); + if (!req->rq_bulk) { + rc = req_capsule_get_size(&req->rq_pill, &RMF_SHORT_IO, + RCL_SERVER); + LASSERT(rc == req->rq_status); + } else { + /* if unwrap_bulk failed, return -EAGAIN to retry */ + rc = sptlrpc_cli_unwrap_bulk_read(req, req->rq_bulk, rc); + } if (rc < 0) { rc = -EAGAIN; goto out; @@ -1506,12 +1553,42 @@ static int osc_brw_fini_request(struct ptlrpc_request *req, int rc) return -EPROTO; } - if (rc != req->rq_bulk->bd_nob_transferred) { + if (req->rq_bulk && rc != req->rq_bulk->bd_nob_transferred) { CERROR("Unexpected rc %d (%d transferred)\n", rc, req->rq_bulk->bd_nob_transferred); return -EPROTO; } + if (!req->rq_bulk) { + /* short io */ + int nob, pg_count, i = 0; + unsigned char *buf; + + CDEBUG(D_CACHE, "Using short io read, size %d\n", rc); + pg_count = aa->aa_page_count; + buf = req_capsule_server_sized_get(&req->rq_pill, &RMF_SHORT_IO, + rc); + nob = rc; + + while (nob > 0 && pg_count > 0) { + int count = aa->aa_ppga[i]->count > nob ? + nob : aa->aa_ppga[i]->count; + unsigned char *ptr; + + CDEBUG(D_CACHE, "page %p count %d\n", + aa->aa_ppga[i]->pg, count); + ptr = kmap_atomic(aa->aa_ppga[i]->pg); + memcpy(ptr + (aa->aa_ppga[i]->off & ~PAGE_MASK), buf, + count); + kunmap_atomic((void *) ptr); + + buf += count; + nob -= count; + i++; + pg_count--; + } + } + if (rc < aa->aa_requested_nob) handle_short_read(rc, aa->aa_page_count, aa->aa_ppga); @@ -1529,7 +1606,8 @@ static int osc_brw_fini_request(struct ptlrpc_request *req, int rc) aa->aa_ppga, OST_READ, cksum_type); - if (peer->nid != req->rq_bulk->bd_sender) { + if (req->rq_bulk && + peer->nid != req->rq_bulk->bd_sender) { via = " via "; router = libcfs_nid2str(req->rq_bulk->bd_sender); } @@ -1705,6 +1783,7 @@ static int brw_interpret(const struct lu_env *env, struct osc_extent *ext; struct osc_extent *tmp; struct client_obd *cli = aa->aa_cli; + unsigned long transferred = 0; rc = osc_brw_fini_request(req, rc); CDEBUG(D_INODE, "request %p aa %p rc %d\n", req, aa, rc); @@ -1798,8 +1877,12 @@ static int brw_interpret(const struct lu_env *env, LASSERT(list_empty(&aa->aa_exts)); LASSERT(list_empty(&aa->aa_oaps)); + transferred = (!req->rq_bulk ? /* short io */ + aa->aa_requested_nob : + req->rq_bulk->bd_nob_transferred); + osc_release_ppga(aa->aa_ppga, aa->aa_page_count); - ptlrpc_lprocfs_brw(req, req->rq_bulk->bd_nob_transferred); + ptlrpc_lprocfs_brw(req, transferred); spin_lock(&cli->cl_loi_list_lock); /* We need to decrement before osc_ap_completion->osc_wake_cache_waiters diff --git a/fs/lustre/ptlrpc/layout.c b/fs/lustre/ptlrpc/layout.c index da315f7..b6476bc 100644 --- a/fs/lustre/ptlrpc/layout.c +++ b/fs/lustre/ptlrpc/layout.c @@ -574,12 +574,14 @@ &RMF_OST_BODY, &RMF_OBD_IOOBJ, &RMF_NIOBUF_REMOTE, - &RMF_CAPA1 + &RMF_CAPA1, + &RMF_SHORT_IO }; static const struct req_msg_field *ost_brw_read_server[] = { &RMF_PTLRPC_BODY, - &RMF_OST_BODY + &RMF_OST_BODY, + &RMF_SHORT_IO }; static const struct req_msg_field *ost_brw_write_server[] = { @@ -1102,6 +1104,10 @@ struct req_msg_field RMF_FIEMAP_VAL = DEFINE_MSGF("fiemap", 0, -1, lustre_swab_fiemap, NULL); EXPORT_SYMBOL(RMF_FIEMAP_VAL); +struct req_msg_field RMF_SHORT_IO = + DEFINE_MSGF("short_io", 0, -1, NULL, NULL); +EXPORT_SYMBOL(RMF_SHORT_IO); + struct req_msg_field RMF_HSM_USER_STATE = DEFINE_MSGF("hsm_user_state", 0, sizeof(struct hsm_user_state), lustre_swab_hsm_user_state, NULL); From patchwork Mon Sep 30 18:55:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167167 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 96ED016B1 for ; Mon, 30 Sep 2019 19:01:18 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7F575224EF for ; Mon, 30 Sep 2019 19:01:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7F575224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 0B5C35C4110; Mon, 30 Sep 2019 11:59:00 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 0BECE5C36DD for ; Mon, 30 Sep 2019 11:57:18 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id E8956100587C; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id E6466B4; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:21 -0400 Message-Id: <1569869810-23848-63-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 062/151] lustre: lov: take lov layout lock for I/O with ignore_layout X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Jinshan Xiong A rule of thumb for taking lov layout configuration lock is if I/O is initiated from LLITE layer, it should grab the lock. If an I/O starts from the OSC layer, it won't be necessary because if the OSC object exists, layout reconfiguration will move forward. Right now CIT_MISC + ci_ignore_layout can identify the I/O from the OSC layer, I just use this in lov_io_init() for this purpose. In the future, an explicit bit may be defined for this. WC-bug-id: https://jira.whamcloud.com/browse/LU-9785 Lustre-commit: e43b0e5c0ccb ("LU-9785 lov: take lov layout lock for I/O with ignore_layout") Signed-off-by: Jinshan Xiong Reviewed-on: https://review.whamcloud.com/29638 Reviewed-by: Andreas Dilger Reviewed-by: Bobi Jam Reviewed-by: Fan Yong Reviewed-by: Jinshan Xiong Signed-off-by: James Simmons --- fs/lustre/lov/lov_object.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/fs/lustre/lov/lov_object.c b/fs/lustre/lov/lov_object.c index 3a7b38d..cb9b108 100644 --- a/fs/lustre/lov/lov_object.c +++ b/fs/lustre/lov/lov_object.c @@ -1224,8 +1224,14 @@ int lov_io_init(const struct lu_env *env, struct cl_object *obj, PFID(lu_object_fid(&obj->co_lu)), io, io->ci_type, io->ci_ignore_layout, io->ci_verify_layout); + /* IO type CIT_MISC with ci_ignore_layout set are usually invoked from + * the OSC layer. It shouldn't take lov layout conf lock in that case, + * because as long as the OSC object exists, the layout can't be + * reconfigured. + */ return LOV_2DISPATCH_MAYLOCK(cl2lov(obj), llo_io_init, - !io->ci_ignore_layout, env, obj, io); + !(io->ci_ignore_layout && io->ci_type == CIT_MISC), + env, obj, io); } /** From patchwork Mon Sep 30 18:55:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167171 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ABE4D16B1 for ; Mon, 30 Sep 2019 19:01:28 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 93B0F224EF for ; Mon, 30 Sep 2019 19:01:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 93B0F224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id A3B545C4146; Mon, 30 Sep 2019 11:59:03 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 4DA095C36DD for ; Mon, 30 Sep 2019 11:57:18 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id EAD25100587D; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id E9306BB; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:22 -0400 Message-Id: <1569869810-23848-64-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 063/151] lustre: lov: pack lsm_flags from layout X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Jinshan Xiong Pack lsm_flags from compount layout. WC-bug-id: https://jira.whamcloud.com/browse/LU-9771 Lustre-commit: 9b74a75ecbdc ("LU-9771 lov: pack lsm_flags from layout") Signed-off-by: Jinshan Xiong Reviewed-on: https://review.whamcloud.com/29081 Reviewed-by: Bobi Jam Reviewed-by: Lai Siyao Signed-off-by: James Simmons --- fs/lustre/lov/lov_ea.c | 1 + fs/lustre/lov/lov_internal.h | 3 ++- fs/lustre/lov/lov_pack.c | 1 + 3 files changed, 4 insertions(+), 1 deletion(-) diff --git a/fs/lustre/lov/lov_ea.c b/fs/lustre/lov/lov_ea.c index e1630f6..8ecf3eb 100644 --- a/fs/lustre/lov/lov_ea.c +++ b/fs/lustre/lov/lov_ea.c @@ -446,6 +446,7 @@ static int lsm_verify_comp_md_v1(struct lov_comp_md_v1 *lcm, lsm->lsm_magic = le32_to_cpu(lcm->lcm_magic); lsm->lsm_layout_gen = le32_to_cpu(lcm->lcm_layout_gen); lsm->lsm_entry_count = entry_count; + lsm->lsm_flags = le16_to_cpu(lcm->lcm_flags); lsm->lsm_is_released = true; lsm->lsm_maxbytes = LLONG_MIN; diff --git a/fs/lustre/lov/lov_internal.h b/fs/lustre/lov/lov_internal.h index e18ea8e..18fe92f 100644 --- a/fs/lustre/lov/lov_internal.h +++ b/fs/lustre/lov/lov_internal.h @@ -86,8 +86,9 @@ struct lov_stripe_md { struct ost_id lsm_oi; u32 lsm_magic; u32 lsm_layout_gen; - u32 lsm_entry_count; + u16 lsm_flags; bool lsm_is_released; + u32 lsm_entry_count; struct lov_stripe_md_entry *lsm_entries[]; }; diff --git a/fs/lustre/lov/lov_pack.c b/fs/lustre/lov/lov_pack.c index 269e61c..c8b55eb 100644 --- a/fs/lustre/lov/lov_pack.c +++ b/fs/lustre/lov/lov_pack.c @@ -185,6 +185,7 @@ ssize_t lov_lsm_pack(const struct lov_stripe_md *lsm, void *buf, lcmv1->lcm_magic = cpu_to_le32(lsm->lsm_magic); lcmv1->lcm_size = cpu_to_le32(lmm_size); lcmv1->lcm_layout_gen = cpu_to_le32(lsm->lsm_layout_gen); + lcmv1->lcm_flags = cpu_to_le16(lsm->lsm_flags); lcmv1->lcm_entry_count = cpu_to_le16(lsm->lsm_entry_count); offset = sizeof(*lcmv1) + sizeof(*lcme) * lsm->lsm_entry_count; From patchwork Mon Sep 30 18:55:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167179 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C01F216B1 for ; Mon, 30 Sep 2019 19:01:54 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A88ED224EF for ; Mon, 30 Sep 2019 19:01:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A88ED224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 65ABC5E4285; Mon, 30 Sep 2019 11:59:11 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8ED975C3ACB for ; Mon, 30 Sep 2019 11:57:18 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id EE0BF100587E; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id EC5B8B5; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:23 -0400 Message-Id: <1569869810-23848-65-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 064/151] lustre: clio: introduce CIT_GLIMPSE for glimpse X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Jinshan Xiong It turned out that it's better to have a standaline I/O context for glimpse request. In that case, it's easier for glimpse to try a new mirror if the current one is out of access. Another problem in this patch is to zero cl_io data structure if the I/O starts from OSC layer. WC-bug-id: https://jira.whamcloud.com/browse/LU-9771 Lustre-commit: d8638fd870ec ("LU-9771 clio: introduce CIT_GLIMPSE for glimpse") Signed-off-by: Jinshan Xiong Reviewed-on: https://review.whamcloud.com/29082 Reviewed-by: Bobi Jam Reviewed-by: Fan Yong Signed-off-by: James Simmons --- fs/lustre/include/cl_object.h | 6 ++++-- fs/lustre/llite/glimpse.c | 16 +++++++++------- fs/lustre/llite/vvp_io.c | 4 ++++ fs/lustre/lov/lov_io.c | 12 ++++++++++++ fs/lustre/obdclass/cl_io.c | 2 ++ fs/lustre/osc/osc_cache.c | 4 ++-- fs/lustre/osc/osc_internal.h | 8 ++++++++ fs/lustre/osc/osc_lock.c | 2 +- fs/lustre/osc/osc_page.c | 2 +- 9 files changed, 43 insertions(+), 13 deletions(-) diff --git a/fs/lustre/include/cl_object.h b/fs/lustre/include/cl_object.h index c0d6452..81190ba 100644 --- a/fs/lustre/include/cl_object.h +++ b/fs/lustre/include/cl_object.h @@ -1367,6 +1367,10 @@ enum cl_io_type { */ CIT_FSYNC, /** + * glimpse. An io context to acquire glimpse lock. + */ + CIT_GLIMPSE, + /** * Miscellaneous io. This is used for occasional io activity that * doesn't fit into other types. Currently this is used for: * @@ -1377,8 +1381,6 @@ enum cl_io_type { * - VM induced page write-out. An io context for writing page out * for memory cleansing; * - * - glimpse. An io context to acquire glimpse lock. - * * - grouplock. An io context to acquire group lock. * * CIT_MISC io is used simply as a context in which locks and pages diff --git a/fs/lustre/llite/glimpse.c b/fs/lustre/llite/glimpse.c index ed5164a..f2f4e4e 100644 --- a/fs/lustre/llite/glimpse.c +++ b/fs/lustre/llite/glimpse.c @@ -189,10 +189,13 @@ int __cl_glimpse_size(struct inode *inode, int agl) u16 refcheck; result = cl_io_get(inode, &env, &io, &refcheck); - if (result > 0) { -again: + if (result <= 0) + return result; + + do { + io->ci_need_restart = 0; io->ci_verify_layout = 1; - result = cl_io_init(env, io, CIT_MISC, io->ci_obj); + result = cl_io_init(env, io, CIT_GLIMPSE, io->ci_obj); if (result > 0) /* * nothing to do for this io. This currently happens @@ -205,9 +208,8 @@ int __cl_glimpse_size(struct inode *inode, int agl) OBD_FAIL_TIMEOUT(OBD_FAIL_GLIMPSE_DELAY, 2); cl_io_fini(env, io); - if (unlikely(io->ci_need_restart)) - goto again; - cl_env_put(env, &refcheck); - } + } while (unlikely(io->ci_need_restart)); + + cl_env_put(env, &refcheck); return result; } diff --git a/fs/lustre/llite/vvp_io.c b/fs/lustre/llite/vvp_io.c index dea820e..3479f7a 100644 --- a/fs/lustre/llite/vvp_io.c +++ b/fs/lustre/llite/vvp_io.c @@ -1352,6 +1352,9 @@ static int vvp_io_read_ahead(const struct lu_env *env, .cio_start = vvp_io_fsync_start, .cio_fini = vvp_io_fini }, + [CIT_GLIMPSE] = { + .cio_fini = vvp_io_fini + }, [CIT_MISC] = { .cio_fini = vvp_io_fini }, @@ -1425,5 +1428,6 @@ int vvp_io_init(const struct lu_env *env, struct cl_object *obj, PFID(lu_object_fid(&obj->co_lu)), result); } + io->ci_result = result < 0 ? result : 0; return result; } diff --git a/fs/lustre/lov/lov_io.c b/fs/lustre/lov/lov_io.c index c7fe4a2..1415702 100644 --- a/fs/lustre/lov/lov_io.c +++ b/fs/lustre/lov/lov_io.c @@ -256,6 +256,7 @@ static int lov_io_slice_init(struct lov_io *lio, struct lov_object *obj, break; } + case CIT_GLIMPSE: case CIT_MISC: lio->lis_pos = 0; lio->lis_endpos = OBD_OBJECT_EOF; @@ -362,6 +363,8 @@ static void lov_io_sub_inherit(struct lov_io_sub *sub, struct lov_io *lio, io->u.ci_ladvise.li_flags = parent->u.ci_ladvise.li_flags; break; } + case CIT_GLIMPSE: + case CIT_MISC: default: break; } @@ -972,6 +975,9 @@ static void lov_io_fsync_end(const struct lu_env *env, .cio_start = lov_io_start, .cio_end = lov_io_end }, + [CIT_GLIMPSE] = { + .cio_fini = lov_io_fini, + }, [CIT_MISC] = { .cio_fini = lov_io_fini } @@ -1046,6 +1052,9 @@ static void lov_empty_impossible(const struct lu_env *env, [CIT_LADVISE] = { .cio_fini = lov_empty_io_fini }, + [CIT_GLIMPSE] = { + .cio_fini = lov_empty_io_fini + }, [CIT_MISC] = { .cio_fini = lov_empty_io_fini } @@ -1084,7 +1093,9 @@ int lov_io_init_empty(const struct lu_env *env, struct cl_object *obj, switch (io->ci_type) { default: LBUG(); + break; case CIT_MISC: + case CIT_GLIMPSE: case CIT_READ: result = 0; break; @@ -1128,6 +1139,7 @@ int lov_io_init_released(const struct lu_env *env, struct cl_object *obj, result = -EOPNOTSUPP; break; case CIT_MISC: + case CIT_GLIMPSE: case CIT_FSYNC: case CIT_LADVISE: case CIT_DATA_VERSION: diff --git a/fs/lustre/obdclass/cl_io.c b/fs/lustre/obdclass/cl_io.c index 1f1579b..3dc5c87 100644 --- a/fs/lustre/obdclass/cl_io.c +++ b/fs/lustre/obdclass/cl_io.c @@ -124,6 +124,8 @@ void cl_io_fini(const struct lu_env *env, struct cl_io *io) LASSERT(ergo(io->ci_ignore_layout || !io->ci_verify_layout, !io->ci_need_restart)); break; + case CIT_GLIMPSE: + break; case CIT_LADVISE: break; default: diff --git a/fs/lustre/osc/osc_cache.c b/fs/lustre/osc/osc_cache.c index 32785e4..7a707a1 100644 --- a/fs/lustre/osc/osc_cache.c +++ b/fs/lustre/osc/osc_cache.c @@ -1005,7 +1005,7 @@ static int osc_extent_truncate(struct osc_extent *ext, pgoff_t trunc_index, if (IS_ERR(env)) return PTR_ERR(env); - io = &osc_env_info(env)->oti_io; + io = osc_env_thread_io(env); io->ci_obj = cl_object_top(osc2cl(obj)); io->ci_ignore_layout = 1; rc = cl_io_init(env, io, CIT_MISC, io->ci_obj); @@ -3180,7 +3180,7 @@ int osc_lock_discard_pages(const struct lu_env *env, struct osc_object *osc, pgoff_t start, pgoff_t end, bool discard) { struct osc_thread_info *info = osc_env_info(env); - struct cl_io *io = &info->oti_io; + struct cl_io *io = osc_env_thread_io(env); osc_page_gang_cbt cb; int result; diff --git a/fs/lustre/osc/osc_internal.h b/fs/lustre/osc/osc_internal.h index 65f52f9..2e0c521 100644 --- a/fs/lustre/osc/osc_internal.h +++ b/fs/lustre/osc/osc_internal.h @@ -89,6 +89,14 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, extern struct lu_device_type osc_device_type; +static inline struct cl_io *osc_env_thread_io(const struct lu_env *env) +{ + struct cl_io *io = &osc_env_info(env)->oti_io; + + memset(io, 0, sizeof(*io)); + return io; +} + static inline int osc_is_object(const struct lu_object *obj) { return obj->lo_dev->ld_type == &osc_device_type; diff --git a/fs/lustre/osc/osc_lock.c b/fs/lustre/osc/osc_lock.c index 42dd654..84dda93 100644 --- a/fs/lustre/osc/osc_lock.c +++ b/fs/lustre/osc/osc_lock.c @@ -618,7 +618,7 @@ static unsigned long osc_lock_weight(const struct lu_env *env, struct osc_object *oscobj, struct ldlm_extent *extent) { - struct cl_io *io = &osc_env_info(env)->oti_io; + struct cl_io *io = osc_env_thread_io(env); struct cl_object *obj = cl_object_top(&oscobj->oo_cl); pgoff_t page_index; int result; diff --git a/fs/lustre/osc/osc_page.c b/fs/lustre/osc/osc_page.c index 836cb14..6e49cf4 100644 --- a/fs/lustre/osc/osc_page.c +++ b/fs/lustre/osc/osc_page.c @@ -560,7 +560,7 @@ long osc_lru_shrink(const struct lu_env *env, struct client_obd *cli, } pvec = (struct cl_page **)osc_env_info(env)->oti_pvec; - io = &osc_env_info(env)->oti_io; + io = osc_env_thread_io(env); spin_lock(&cli->cl_lru_list_lock); if (force) From patchwork Mon Sep 30 18:55:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167099 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C7D7D16C1 for ; Mon, 30 Sep 2019 18:58:41 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B04A9224D5 for ; Mon, 30 Sep 2019 18:58:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B04A9224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 6A2365C3D5C; Mon, 30 Sep 2019 11:57:54 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id E5B385C3ACB for ; Mon, 30 Sep 2019 11:57:18 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id F216110058DA; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id EF6A0A9; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:24 -0400 Message-Id: <1569869810-23848-66-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 065/151] lustre: flr: add infrastructure to create a new mirror X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Jinshan Xiong This patch adds the initial code to support file level redundancy phase 1. Details about the implementation can be read at http://wiki.lustre.org/File_Level_Replication_High_Level_Design. Add the definitions for FLR support to the UAPI headers. These values are also placed in RPC packets to send to the servers. We expand both enum mds_op_bias and enum lu_xattr_flags with the 'MERGE' values that cover the FLR support. WC-bug-id: https://jira.whamcloud.com/browse/LU-9771 Lustre-commit: b879bbc27db5 ("LU-9771 flr: add infrastructure to create a new mirror") Signed-off-by: Jinshan Xiong Reviewed-on: https://review.whamcloud.com/29083 Reviewed-by: Bobi Jam Reviewed-by: Fan Yong Signed-off-by: James Simmons --- fs/lustre/include/lu_object.h | 3 ++- fs/lustre/llite/file.c | 38 +++++++++++++++++---------- fs/lustre/lov/lov_ea.c | 1 + fs/lustre/lov/lov_internal.h | 3 ++- fs/lustre/lov/lov_pack.c | 1 + fs/lustre/mdc/mdc_lib.c | 3 +-- fs/lustre/mdc/mdc_request.c | 3 ++- fs/lustre/ptlrpc/pack_generic.c | 1 + fs/lustre/ptlrpc/wiretest.c | 18 +++++++++++-- include/uapi/linux/lustre/lustre_idl.h | 4 +++ include/uapi/linux/lustre/lustre_user.h | 46 ++++++++++++++++++++++++++++++++- 11 files changed, 99 insertions(+), 22 deletions(-) diff --git a/fs/lustre/include/lu_object.h b/fs/lustre/include/lu_object.h index 05f8d81..9427ad4 100644 --- a/fs/lustre/include/lu_object.h +++ b/fs/lustre/include/lu_object.h @@ -864,7 +864,8 @@ struct lu_rdpg { enum lu_xattr_flags { LU_XATTR_REPLACE = (1 << 0), - LU_XATTR_CREATE = (1 << 1) + LU_XATTR_CREATE = BIT(1), + LU_XATTR_MERGE = BIT(2), }; /** @} helpers */ diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 0852e37..e1f264d 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -147,9 +147,10 @@ static int ll_close_inode_openhandle(struct inode *inode, ll_prepare_close(inode, op_data, och); switch (bias) { + case MDS_CLOSE_LAYOUT_MERGE: case MDS_CLOSE_LAYOUT_SWAP: LASSERT(data); - op_data->op_bias |= MDS_CLOSE_LAYOUT_SWAP; + op_data->op_bias |= bias; op_data->op_data_version = 0; op_data->op_lease_handle = och->och_lease_handle; op_data->op_fid2 = *ll_inode2fid(data); @@ -175,8 +176,7 @@ static int ll_close_inode_openhandle(struct inode *inode, md_exp->exp_obd->obd_name, PFID(&lli->lli_fid), rc); } - if (op_data->op_bias & (MDS_HSM_RELEASE | MDS_CLOSE_LAYOUT_SWAP) && - !rc) { + if (rc == 0 && (op_data->op_bias & bias)) { struct mdt_body *body; body = req_capsule_server_get(&req->rq_pill, &RMF_MDT_BODY); @@ -948,10 +948,12 @@ static int ll_check_swap_layouts_validity(struct inode *inode1, } static int ll_swap_layouts_close(struct obd_client_handle *och, - struct inode *inode, struct inode *inode2) + struct inode *inode, struct inode *inode2, + int intent) { const struct lu_fid *fid1 = ll_inode2fid(inode); const struct lu_fid *fid2; + enum mds_op_bias bias; int rc; CDEBUG(D_INODE, "%s: biased close of file " DFID "\n", @@ -970,13 +972,24 @@ static int ll_swap_layouts_close(struct obd_client_handle *och, goto out_free_och; } + switch (intent) { + case SWAP_LAYOUTS_CLOSE: + bias = MDS_CLOSE_LAYOUT_SWAP; + break; + case MERGE_LAYOUTS_CLOSE: + bias = MDS_CLOSE_LAYOUT_MERGE; + break; + default: + rc = -EOPNOTSUPP; + goto out_free_och; + } + /* - * Close the file and swap layouts between inode & inode2. + * Close the file and {swap,merge} layouts between inode & inode2. * NB: lease lock handle is released in mdc_close_layout_swap_pack() * because we still need it to pack l_remote_handle to MDT. */ - rc = ll_close_inode_openhandle(inode, och, MDS_CLOSE_LAYOUT_SWAP, - inode2); + rc = ll_close_inode_openhandle(inode, och, bias, inode2); och = NULL; /* freed in ll_close_inode_openhandle() */ @@ -2557,6 +2570,7 @@ int ll_ioctl_fssetxattr(struct inode *inode, unsigned int cmd, case LL_IOC_LOV_SWAP_LAYOUTS: { struct file *file2; struct lustre_swap_layouts lsl; + u64 intent; if (copy_from_user(&lsl, (char __user *)arg, sizeof(struct lustre_swap_layouts))) @@ -2575,16 +2589,12 @@ int ll_ioctl_fssetxattr(struct inode *inode, unsigned int cmd, goto out; } - if (lsl.sl_flags & SWAP_LAYOUTS_CLOSE) { + intent = lsl.sl_flags & INTENT_LAYOUTS_CLOSE; + if (intent) { struct obd_client_handle *och = NULL; struct ll_inode_info *lli; struct inode *inode2; - if (lsl.sl_flags != SWAP_LAYOUTS_CLOSE) { - rc = -EINVAL; - goto out; - } - lli = ll_i2info(inode); mutex_lock(&lli->lli_och_mutex); if (fd->fd_lease_och) { @@ -2597,7 +2607,7 @@ int ll_ioctl_fssetxattr(struct inode *inode, unsigned int cmd, goto out; } inode2 = file_inode(file2); - rc = ll_swap_layouts_close(och, inode, inode2); + rc = ll_swap_layouts_close(och, inode, inode2, intent); } else { rc = ll_swap_layouts(file, file2, &lsl); } diff --git a/fs/lustre/lov/lov_ea.c b/fs/lustre/lov/lov_ea.c index 8ecf3eb..081a04f 100644 --- a/fs/lustre/lov/lov_ea.c +++ b/fs/lustre/lov/lov_ea.c @@ -446,6 +446,7 @@ static int lsm_verify_comp_md_v1(struct lov_comp_md_v1 *lcm, lsm->lsm_magic = le32_to_cpu(lcm->lcm_magic); lsm->lsm_layout_gen = le32_to_cpu(lcm->lcm_layout_gen); lsm->lsm_entry_count = entry_count; + lsm->lsm_mirror_count = le16_to_cpu(lcm->lcm_mirror_count); lsm->lsm_flags = le16_to_cpu(lcm->lcm_flags); lsm->lsm_is_released = true; lsm->lsm_maxbytes = LLONG_MIN; diff --git a/fs/lustre/lov/lov_internal.h b/fs/lustre/lov/lov_internal.h index 18fe92f..b3712d2 100644 --- a/fs/lustre/lov/lov_internal.h +++ b/fs/lustre/lov/lov_internal.h @@ -88,7 +88,8 @@ struct lov_stripe_md { u32 lsm_layout_gen; u16 lsm_flags; bool lsm_is_released; - u32 lsm_entry_count; + u16 lsm_mirror_count; + u16 lsm_entry_count; struct lov_stripe_md_entry *lsm_entries[]; }; diff --git a/fs/lustre/lov/lov_pack.c b/fs/lustre/lov/lov_pack.c index c8b55eb..aac5048 100644 --- a/fs/lustre/lov/lov_pack.c +++ b/fs/lustre/lov/lov_pack.c @@ -186,6 +186,7 @@ ssize_t lov_lsm_pack(const struct lov_stripe_md *lsm, void *buf, lcmv1->lcm_size = cpu_to_le32(lmm_size); lcmv1->lcm_layout_gen = cpu_to_le32(lsm->lsm_layout_gen); lcmv1->lcm_flags = cpu_to_le16(lsm->lsm_flags); + lcmv1->lcm_mirror_count = cpu_to_le16(lsm->lsm_mirror_count); lcmv1->lcm_entry_count = cpu_to_le16(lsm->lsm_entry_count); offset = sizeof(*lcmv1) + sizeof(*lcme) * lsm->lsm_entry_count; diff --git a/fs/lustre/mdc/mdc_lib.c b/fs/lustre/mdc/mdc_lib.c index 6aa9dc2..3601e17 100644 --- a/fs/lustre/mdc/mdc_lib.c +++ b/fs/lustre/mdc/mdc_lib.c @@ -431,8 +431,7 @@ static void mdc_intent_close_pack(struct ptlrpc_request *req, struct close_data *data; struct ldlm_lock *lock; - if (!(bias & (MDS_HSM_RELEASE | MDS_CLOSE_LAYOUT_SWAP | - MDS_RENAME_MIGRATE))) + if (!(bias & (MDS_CLOSE_INTENT | MDS_RENAME_MIGRATE))) return; data = req_capsule_client_get(&req->rq_pill, &RMF_CLOSE_DATA); diff --git a/fs/lustre/mdc/mdc_request.c b/fs/lustre/mdc/mdc_request.c index 655f1c2..a1ed9bf 100644 --- a/fs/lustre/mdc/mdc_request.c +++ b/fs/lustre/mdc/mdc_request.c @@ -776,7 +776,8 @@ static int mdc_close(struct obd_export *exp, struct md_op_data *op_data, /* save the errcode and proceed to close */ saved_rc = rc; } - } else if (op_data->op_bias & MDS_CLOSE_LAYOUT_SWAP) { + } else if (op_data->op_bias & (MDS_CLOSE_LAYOUT_SWAP | + MDS_CLOSE_LAYOUT_MERGE)) { req_fmt = &RQF_MDS_INTENT_CLOSE; } else { req_fmt = &RQF_MDS_CLOSE; diff --git a/fs/lustre/ptlrpc/pack_generic.c b/fs/lustre/ptlrpc/pack_generic.c index a631dde..eb82eaa 100644 --- a/fs/lustre/ptlrpc/pack_generic.c +++ b/fs/lustre/ptlrpc/pack_generic.c @@ -2018,6 +2018,7 @@ void lustre_swab_lov_comp_md_v1(struct lov_comp_md_v1 *lum) __swab32s(&lum->lcm_layout_gen); __swab16s(&lum->lcm_flags); __swab16s(&lum->lcm_entry_count); + __swab16s(&lum->lcm_mirror_count); BUILD_BUG_ON(offsetof(typeof(*lum), lcm_padding1) == 0); BUILD_BUG_ON(offsetof(typeof(*lum), lcm_padding2) == 0); diff --git a/fs/lustre/ptlrpc/wiretest.c b/fs/lustre/ptlrpc/wiretest.c index 1092cc2..749effb 100644 --- a/fs/lustre/ptlrpc/wiretest.c +++ b/fs/lustre/ptlrpc/wiretest.c @@ -1513,6 +1513,8 @@ void lustre_assert_wire_constants(void) (long long)(int)sizeof(((struct lov_comp_md_entry_v1 *)0)->lcme_padding)); LASSERTF(LCME_FL_INIT == 0x00000010UL, "found 0x%.8xUL\n", (unsigned int)LCME_FL_INIT); + LASSERTF(LCME_FL_NEG == 0x80000000UL, "found 0x%.8xUL\n", + (unsigned int)LCME_FL_NEG); /* Checks for struct lov_comp_md_v1 */ LASSERTF((int)sizeof(struct lov_comp_md_v1) == 32, "found %lld\n", @@ -1537,9 +1539,13 @@ void lustre_assert_wire_constants(void) (long long)(int)offsetof(struct lov_comp_md_v1, lcm_entry_count)); LASSERTF((int)sizeof(((struct lov_comp_md_v1 *)0)->lcm_entry_count) == 2, "found %lld\n", (long long)(int)sizeof(((struct lov_comp_md_v1 *)0)->lcm_entry_count)); - LASSERTF((int)offsetof(struct lov_comp_md_v1, lcm_padding1) == 16, "found %lld\n", + LASSERTF((int)offsetof(struct lov_comp_md_v1, lcm_mirror_count) == 16, "found %lld\n", + (long long)(int)offsetof(struct lov_comp_md_v1, lcm_mirror_count)); + LASSERTF((int)sizeof(((struct lov_comp_md_v1 *)0)->lcm_mirror_count) == 2, "found %lld\n", + (long long)(int)sizeof(((struct lov_comp_md_v1 *)0)->lcm_mirror_count)); + LASSERTF((int)offsetof(struct lov_comp_md_v1, lcm_padding1) == 18, "found %lld\n", (long long)(int)offsetof(struct lov_comp_md_v1, lcm_padding1)); - LASSERTF((int)sizeof(((struct lov_comp_md_v1 *)0)->lcm_padding1) == 8, "found %lld\n", + LASSERTF((int)sizeof(((struct lov_comp_md_v1 *)0)->lcm_padding1) == 6, "found %lld\n", (long long)(int)sizeof(((struct lov_comp_md_v1 *)0)->lcm_padding1)); LASSERTF((int)offsetof(struct lov_comp_md_v1, lcm_padding2) == 24, "found %lld\n", (long long)(int)offsetof(struct lov_comp_md_v1, lcm_padding2)); @@ -1550,6 +1556,14 @@ void lustre_assert_wire_constants(void) LASSERTF((int)sizeof(((struct lov_comp_md_v1 *)0)->lcm_entries[0]) == 48, "found %lld\n", (long long)(int)sizeof(((struct lov_comp_md_v1 *)0)->lcm_entries[0])); BUILD_BUG_ON(LOV_MAGIC_COMP_V1 != (0x0BD60000 | 0x0BD0)); + LASSERTF(LCM_FL_NOT_FLR == 0, "found %lld\n", + (long long)LCM_FL_NOT_FLR); + LASSERTF(LCM_FL_RDONLY == 1, "found %lld\n", + (long long)LCM_FL_RDONLY); + LASSERTF(LCM_FL_WRITE_PENDING == 2, "found %lld\n", + (long long)LCM_FL_WRITE_PENDING); + LASSERTF(LCM_FL_SYNC_PENDING == 3, "found %lld\n", + (long long)LCM_FL_SYNC_PENDING); /* Checks for struct lmv_mds_md_v1 */ LASSERTF((int)sizeof(struct lmv_mds_md_v1) == 56, "found %lld\n", diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index 2181386..131f1c9 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -1686,8 +1686,12 @@ enum mds_op_bias { MDS_HSM_RELEASE = 1 << 12, MDS_RENAME_MIGRATE = 1 << 13, MDS_CLOSE_LAYOUT_SWAP = 1 << 14, + MDS_CLOSE_LAYOUT_MERGE = 1 << 15, }; +#define MDS_CLOSE_INTENT (MDS_HSM_RELEASE | MDS_CLOSE_LAYOUT_SWAP | \ + MDS_CLOSE_LAYOUT_MERGE) + /* instance of mdt_reint_rec */ struct mdt_rec_create { __u32 cr_opcode; diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index 4a6ed5e..34c86df 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -424,6 +424,11 @@ static inline bool lu_extent_is_overlapped(struct lu_extent *e1, return e1->e_start < e2->e_end && e2->e_start < e1->e_end; } +static inline bool lu_extent_is_whole(struct lu_extent *e) +{ + return e->e_start == 0 && e->e_end == LUSTRE_EOF; +} + enum lov_comp_md_entry_flags { LCME_FL_PRIMARY = 0x00000001, /* Not used */ LCME_FL_STALE = 0x00000002, /* Not used */ @@ -460,17 +465,54 @@ struct lov_comp_md_entry_v1 { __u64 lcme_padding[2]; } __packed; +#define SEQ_ID_MAX 0x0000FFFF +#define SEQ_ID_MASK SEQ_ID_MAX +/* bit 30:16 of lcme_id is used to store mirror id */ +#define MIRROR_ID_MASK 0x7FFF0000 +#define MIRROR_ID_SHIFT 16 + +static inline __u32 pflr_id(__u16 mirror_id, __u16 seqid) +{ + return ((mirror_id << MIRROR_ID_SHIFT) & MIRROR_ID_MASK) | seqid; +} + +static inline __u16 mirror_id_of(__u32 id) +{ + return (id & MIRROR_ID_MASK) >> MIRROR_ID_SHIFT; +} + +/** + * on-disk data for lcm_flags. Valid if lcm_magic is LOV_MAGIC_COMP_V1. + */ +enum lov_comp_md_flags { + /* the least 2 bits are used by FLR to record file state */ + LCM_FL_NOT_FLR = 0, + LCM_FL_RDONLY = 1, + LCM_FL_WRITE_PENDING = 2, + LCM_FL_SYNC_PENDING = 3, + LCM_FL_FLR_MASK = 0x3, +}; + struct lov_comp_md_v1 { __u32 lcm_magic; /* LOV_USER_MAGIC_COMP_V1 */ __u32 lcm_size; /* overall size including this struct */ __u32 lcm_layout_gen; __u16 lcm_flags; __u16 lcm_entry_count; - __u64 lcm_padding1; + /* lcm_mirror_count stores the number of actual mirrors minus 1, + * so that non-flr files will have value 0 meaning 1 mirror. + */ + __u16 lcm_mirror_count; + __u16 lcm_padding1[3]; __u64 lcm_padding2; struct lov_comp_md_entry_v1 lcm_entries[0]; } __packed; +/* + * Maximum number of mirrors Lustre can support. + */ +#define LUSTRE_MIRROR_COUNT_MAX 16 + static inline __u32 lov_user_md_size(__u16 stripes, __u32 lmm_magic) { if (stripes == (__u16)-1) @@ -745,6 +787,8 @@ struct if_quotactl { #define SWAP_LAYOUTS_KEEP_MTIME (1 << 2) #define SWAP_LAYOUTS_KEEP_ATIME (1 << 3) #define SWAP_LAYOUTS_CLOSE (1 << 4) +#define MERGE_LAYOUTS_CLOSE (1 << 5) +#define INTENT_LAYOUTS_CLOSE (SWAP_LAYOUTS_CLOSE | MERGE_LAYOUTS_CLOSE) /* Swap XATTR_NAME_HSM as well, only on the MDT so far */ #define SWAP_LAYOUTS_MDS_HSM (1 << 31) From patchwork Mon Sep 30 18:55:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167089 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 388BF16B1 for ; Mon, 30 Sep 2019 18:58:27 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 20932224D5 for ; Mon, 30 Sep 2019 18:58:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 20932224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 1CBF95C3CFD; Mon, 30 Sep 2019 11:57:47 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 46C0F5C3B00 for ; Mon, 30 Sep 2019 11:57:19 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 00C5C10058DB; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id F26D4BD; Mon, 30 Sep 2019 14:56:56 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:25 -0400 Message-Id: <1569869810-23848-67-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 066/151] lustre: clio: no glimpse for data immutable file X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Jinshan Xiong When merging a layout to an existing file as a mirror, client will report the size and blocks to the MDT, which will be set to the MDT object. In that case, if the MDT discovers a mirrored file in READ_ONLY state, it will report the size and blocks. Clients should take this advantage and skip glimpse. WC-bug-id: https://jira.whamcloud.com/browse/LU-9771 Lustre-commit: ce61bcac8661 ("LU-9771 clio: no glimpse for data immutable file") Signed-off-by: Jinshan Xiong Reviewed-on: https://review.whamcloud.com/29084 Reviewed-by: Fan Yong Reviewed-by: Bobi Jam Signed-off-by: James Simmons --- fs/lustre/llite/file.c | 5 +++++ fs/lustre/lov/lov_io.c | 22 ++++++++++++++++------ include/uapi/linux/lustre/lustre_idl.h | 1 + include/uapi/linux/lustre/lustre_user.h | 11 +++++++++++ 4 files changed, 33 insertions(+), 6 deletions(-) diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index e1f264d..6f991ed 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -148,6 +148,11 @@ static int ll_close_inode_openhandle(struct inode *inode, ll_prepare_close(inode, op_data, och); switch (bias) { case MDS_CLOSE_LAYOUT_MERGE: + /* merge blocks from the victim inode */ + op_data->op_attr_blocks += ((struct inode *)data)->i_blocks; + op_data->op_attr.ia_valid |= ATTR_SIZE; + op_data->op_xvalid |= OP_XVALID_BLOCKS; + /* fallthrough */ case MDS_CLOSE_LAYOUT_SWAP: LASSERT(data); op_data->op_bias |= bias; diff --git a/fs/lustre/lov/lov_io.c b/fs/lustre/lov/lov_io.c index 1415702..c3fd86a 100644 --- a/fs/lustre/lov/lov_io.c +++ b/fs/lustre/lov/lov_io.c @@ -257,6 +257,13 @@ static int lov_io_slice_init(struct lov_io *lio, struct lov_object *obj, } case CIT_GLIMPSE: + lio->lis_pos = 0; + lio->lis_endpos = OBD_OBJECT_EOF; + + if ((obj->lo_lsm->lsm_flags & LCM_FL_FLR_MASK) == LCM_FL_RDONLY) + return 1; /* SoM is accurate, no need glimpse */ + break; + case CIT_MISC: lio->lis_pos = 0; lio->lis_endpos = OBD_OBJECT_EOF; @@ -1068,18 +1075,21 @@ int lov_io_init_composite(const struct lu_env *env, struct cl_object *obj, { struct lov_io *lio = lov_env_io(env); struct lov_object *lov = cl2lov(obj); + int result; INIT_LIST_HEAD(&lio->lis_active); - io->ci_result = lov_io_slice_init(lio, lov, io); - if (io->ci_result) - return io->ci_result; + result = lov_io_slice_init(lio, lov, io); + if (result) + goto out; - io->ci_result = lov_io_subio_init(env, lio, io); - if (io->ci_result == 0) { + result = lov_io_subio_init(env, lio, io); + if (!result) { cl_io_slice_add(io, &lio->lis_cl, obj, &lov_io_ops); atomic_inc(&lov->lo_active_ios); } - return io->ci_result; +out: + io->ci_result = result < 0 ? result : 0; + return result; } int lov_io_init_empty(const struct lu_env *env, struct cl_object *obj, diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index 131f1c9..951d501 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -1015,6 +1015,7 @@ struct lov_mds_md_v1 { /* LOV EA mds/wire data (little-endian) */ #define XATTR_SECURITY_PREFIX "security." #define XATTR_LUSTRE_PREFIX "lustre." +#define XATTR_NAME_SOM "trusted.som" #define XATTR_NAME_LOV "trusted.lov" #define XATTR_NAME_LMA "trusted.lma" #define XATTR_NAME_LMV "trusted.lmv" diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index 34c86df..7d9f0d2 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -201,6 +201,17 @@ struct lustre_mdt_attrs { */ #define LMA_OLD_SIZE (sizeof(struct lustre_mdt_attrs) + 5 * sizeof(__u64)) +enum { + LSOM_FL_VALID = 1 << 0, +}; + +struct lustre_som_attrs { + __u16 lsa_valid; + __u16 lsa_reserved[3]; + __u64 lsa_size; + __u64 lsa_blocks; +}; + /** * OST object IDentifier. */ From patchwork Mon Sep 30 18:55:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167101 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E9A0316B1 for ; Mon, 30 Sep 2019 18:58:48 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D14F6224D5 for ; Mon, 30 Sep 2019 18:58:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D14F6224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 0B9F05C3D9D; Mon, 30 Sep 2019 11:57:58 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 9BE765C3A37 for ; Mon, 30 Sep 2019 11:57:19 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 02A1C10058DC; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 01824B4; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:26 -0400 Message-Id: <1569869810-23848-68-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 067/151] lustre: flr: read support for flr X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Jinshan Xiong Avoid stale mirrors to read; If preferred mirror is inaccessible, try next one, ndelay RPC is implemented to make the error-out quick. ndelay RPC has rq_no_delay bit set that can be applied to brw and lock RPC. WC-bug-id: https://jira.whamcloud.com/browse/LU-9771 Lustre-commit: 526dbd3d8723 ("LU-9971 flr: read support for flr") Signed-off-by: Jinshan Xiong Reviewed-on: https://review.whamcloud.com/29085 Reviewed-by: Bobi Jam Reviewed-by: Fan Yong Signed-off-by: James Simmons --- fs/lustre/include/cl_object.h | 22 +++- fs/lustre/include/lustre_dlm_flags.h | 7 ++ fs/lustre/include/lustre_osc.h | 6 +- fs/lustre/include/obd_support.h | 3 + fs/lustre/ldlm/ldlm_request.c | 9 ++ fs/lustre/llite/file.c | 14 +++ fs/lustre/llite/glimpse.c | 12 ++- fs/lustre/llite/rw.c | 28 ++++- fs/lustre/llite/vvp_internal.h | 1 + fs/lustre/llite/vvp_io.c | 55 +++++++--- fs/lustre/llite/vvp_page.c | 9 +- fs/lustre/lov/lov_cl_internal.h | 186 +++++++++++++++++++++++++++------ fs/lustre/lov/lov_io.c | 183 ++++++++++++++++++++++++++++---- fs/lustre/lov/lov_lock.c | 15 +-- fs/lustre/lov/lov_object.c | 147 +++++++++++++++++++++----- fs/lustre/lov/lov_page.c | 2 +- fs/lustre/obdclass/cl_io.c | 14 ++- fs/lustre/osc/osc_cache.c | 8 +- fs/lustre/osc/osc_io.c | 9 +- fs/lustre/osc/osc_lock.c | 4 + fs/lustre/osc/osc_request.c | 15 ++- include/uapi/linux/lustre/lustre_idl.h | 4 + 22 files changed, 623 insertions(+), 130 deletions(-) diff --git a/fs/lustre/include/cl_object.h b/fs/lustre/include/cl_object.h index 81190ba..f573c6f 100644 --- a/fs/lustre/include/cl_object.h +++ b/fs/lustre/include/cl_object.h @@ -1871,7 +1871,20 @@ struct cl_io { */ ci_noatime:1, /* Tell sublayers not to expand LDLM locks requested for this IO */ - ci_lock_no_expand:1; + ci_lock_no_expand:1, + /** + * Set if non-delay RPC should be used for this IO. + * + * If this file has multiple mirrors, and if the OSTs of the current + * mirror is inaccessible, non-delay RPC would error out quickly so + * that the upper layer can try to access the next mirror. + */ + ci_ndelay:1; + /** + * How many times the read has retried before this one. + * Set by the top level and consumed by the LOV. + */ + unsigned int ci_ndelay_tried; /** * Number of pages owned by this IO. For invariant checking. */ @@ -2336,9 +2349,8 @@ static inline int cl_io_is_trunc(const struct cl_io *io) do { \ typeof(foo_io) __foo_io = (foo_io); \ \ - BUILD_BUG_ON(offsetof(typeof(*__foo_io), base) != 0); \ - memset(&__foo_io->base + 1, 0, \ - sizeof(*__foo_io) - sizeof(__foo_io->base)); \ + memset(&__foo_io->base, 0, \ + sizeof(*__foo_io) - offsetof(typeof(*__foo_io), base)); \ } while (0) /** @} cl_io */ @@ -2385,6 +2397,8 @@ void cl_page_list_del(const struct lu_env *env, struct cl_page_list *plist, struct cl_page *page); void cl_page_list_disown(const struct lu_env *env, struct cl_io *io, struct cl_page_list *plist); +void cl_page_list_discard(const struct lu_env *env, + struct cl_io *io, struct cl_page_list *plist); void cl_page_list_fini(const struct lu_env *env, struct cl_page_list *plist); void cl_2queue_init(struct cl_2queue *queue); diff --git a/fs/lustre/include/lustre_dlm_flags.h b/fs/lustre/include/lustre_dlm_flags.h index f1f454f..22fb595 100644 --- a/fs/lustre/include/lustre_dlm_flags.h +++ b/fs/lustre/include/lustre_dlm_flags.h @@ -391,6 +391,13 @@ #define ldlm_set_excl(_l) LDLM_SET_FLAG((_l), 1ULL << 55) #define ldlm_clear_excl(_l) LDLM_CLEAR_FLAG((_l), 1ULL << 55) +/** + * This flags means to use non-delay RPC to send dlm request RPC. + */ +#define LDLM_FL_NDELAY 0x0400000000000000ULL /* bit 58 */ +#define ldlm_is_ndelay(_l) LDLM_TEST_FLAG((_l), 1ULL << 58) +#define ldlm_set_ndelay(_l) LDLM_SET_FLAG((_l), 1ULL << 58) + /** l_flags bits marked as "ast" bits */ #define LDLM_FL_AST_MASK (LDLM_FL_FLOCK_DEADLOCK |\ LDLM_FL_DISCARD_DATA) diff --git a/fs/lustre/include/lustre_osc.h b/fs/lustre/include/lustre_osc.h index 895c1cb..5efceef 100644 --- a/fs/lustre/include/lustre_osc.h +++ b/fs/lustre/include/lustre_osc.h @@ -587,7 +587,7 @@ int osc_teardown_async_page(const struct lu_env *env, struct osc_object *obj, int osc_flush_async_page(const struct lu_env *env, struct cl_io *io, struct osc_page *ops); int osc_queue_sync_pages(const struct lu_env *env, struct osc_object *obj, - struct list_head *list, int cmd, int brw_flags); + struct list_head *list, int brw_flags); int osc_cache_truncate_start(const struct lu_env *env, struct osc_object *obj, u64 size, struct osc_extent **extp); void osc_cache_truncate_end(const struct lu_env *env, struct osc_extent *ext); @@ -927,7 +927,9 @@ struct osc_extent { /* this extent should be written back asap. set if one of pages is * called by page WB daemon, or sync write or reading requests. */ - oe_urgent:1; + oe_urgent:1, + /* Non-delay RPC should be used for this extent. */ + oe_ndelay:1; /* how many grants allocated for this extent. * Grant allocated for this extent. There is no grant allocated * for reading extents and sync write extents. diff --git a/fs/lustre/include/obd_support.h b/fs/lustre/include/obd_support.h index 7999ac6..e6dff44b 100644 --- a/fs/lustre/include/obd_support.h +++ b/fs/lustre/include/obd_support.h @@ -474,6 +474,9 @@ /* LMV */ #define OBD_FAIL_UNKNOWN_LMV_STRIPE 0x1901 +/* FLR */ +#define OBD_FAIL_FLR_GLIMPSE_IMMUTABLE 0x1A00 + /* Assign references to moved code to reduce code changes */ #define OBD_FAIL_PRECHECK(id) CFS_FAIL_PRECHECK(id) #define OBD_FAIL_CHECK(id) CFS_FAIL_CHECK(id) diff --git a/fs/lustre/ldlm/ldlm_request.c b/fs/lustre/ldlm/ldlm_request.c index 13b323a..65c3558 100644 --- a/fs/lustre/ldlm/ldlm_request.c +++ b/fs/lustre/ldlm/ldlm_request.c @@ -748,6 +748,15 @@ int ldlm_cli_enqueue(struct obd_export *exp, struct ptlrpc_request **reqp, DLM_LOCKREQ_OFF, len, (int)sizeof(*body)); } + if (*flags & LDLM_FL_NDELAY) { + DEBUG_REQ(D_DLMTRACE, req, "enque lock with no delay\n"); + req->rq_no_resend = req->rq_no_delay = 1; + /* probably set a shorter timeout value and handle ETIMEDOUT + * in osc_lock_upcall() correctly + */ + /* lustre_msg_set_timeout(req, req->rq_timeout / 2); */ + } + /* Dump lock data into the request buffer */ body = req_capsule_client_get(&req->rq_pill, &RMF_DLM_REQ); ldlm_lock2desc(lock, &body->lock_desc); diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 6f991ed..1856aa6 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -1155,6 +1155,11 @@ static void ll_io_init(struct cl_io *io, const struct file *file, int write) } io->ci_noatime = file_is_noatime(file); + + /* FLR: only use non-delay I/O for read as there is only one + * available mirror for write. + */ + io->ci_ndelay = !write; } static ssize_t @@ -1169,6 +1174,7 @@ static void ll_io_init(struct cl_io *io, const struct file *file, int write) struct cl_io *io; ssize_t result = 0; int rc = 0; + unsigned int retried = 0; CDEBUG(D_VFSTRACE, "file: %pD, type: %d ppos: %llu, count: %zu\n", file, iot, *ppos, count); @@ -1176,6 +1182,7 @@ static void ll_io_init(struct cl_io *io, const struct file *file, int write) restart: io = vvp_env_thread_io(env); ll_io_init(io, file, iot == CIT_WRITE); + io->ci_ndelay_tried = retried; if (cl_io_rw_init(env, io, iot, *ppos, count) == 0) { struct vvp_io *vio = vvp_env_io(env); @@ -1232,12 +1239,19 @@ static void ll_io_init(struct cl_io *io, const struct file *file, int write) out: cl_io_fini(env, io); + CDEBUG(D_VFSTRACE, + "%s: %d io complete with rc: %d, result: %zd, restart: %d\n", + file->f_path.dentry->d_name.name, + iot, rc, result, io->ci_need_restart); + if ((!rc || rc == -ENODATA) && count > 0 && io->ci_need_restart) { CDEBUG(D_VFSTRACE, "%s: restart %s from %lld, count:%zu, result: %zd\n", file_dentry(file)->d_name.name, iot == CIT_READ ? "read" : "write", *ppos, count, result); + /* preserve the tried count for FLR */ + retried = io->ci_ndelay_tried; goto restart; } diff --git a/fs/lustre/llite/glimpse.c b/fs/lustre/llite/glimpse.c index f2f4e4e..13a2621 100644 --- a/fs/lustre/llite/glimpse.c +++ b/fs/lustre/llite/glimpse.c @@ -187,24 +187,28 @@ int __cl_glimpse_size(struct inode *inode, int agl) struct cl_io *io = NULL; int result; u16 refcheck; + int retried = 0; result = cl_io_get(inode, &env, &io, &refcheck); if (result <= 0) return result; do { - io->ci_need_restart = 0; - io->ci_verify_layout = 1; + io->ci_ndelay_tried = retried++; + io->ci_ndelay = io->ci_verify_layout = 1; result = cl_io_init(env, io, CIT_GLIMPSE, io->ci_obj); - if (result > 0) + if (result > 0) { /* * nothing to do for this io. This currently happens * when stripe sub-object's are not yet created. */ result = io->ci_result; - else if (result == 0) + } else if (result == 0) { result = cl_glimpse_lock(env, io, inode, io->ci_obj, agl); + if (!agl && result == -EWOULDBLOCK) + io->ci_need_restart = 1; + } OBD_FAIL_TIMEOUT(OBD_FAIL_GLIMPSE_DELAY, 2); cl_io_fini(env, io); diff --git a/fs/lustre/llite/rw.c b/fs/lustre/llite/rw.c index ca0b357..e2e47174 100644 --- a/fs/lustre/llite/rw.c +++ b/fs/lustre/llite/rw.c @@ -1100,8 +1100,9 @@ static int ll_io_read_page(const struct lu_env *env, struct cl_io *io, struct inode *inode = vvp_object_inode(page->cp_obj); struct ll_file_data *fd = LUSTRE_FPRIVATE(file); struct ll_readahead_state *ras = &fd->fd_ras; - struct cl_2queue *queue = &io->ci_queue; + struct cl_2queue *queue = &io->ci_queue; struct ll_sb_info *sbi = ll_i2sbi(inode); + struct cl_sync_io *anchor = NULL; struct vvp_page *vpg; bool uptodate; int rc = 0; @@ -1128,6 +1129,10 @@ static int ll_io_read_page(const struct lu_env *env, struct cl_io *io, cl_page_export(env, page, 1); cl_page_disown(env, io, page); } else { + anchor = &vvp_env_info(env)->vti_anchor; + cl_sync_io_init(anchor, 1); + page->cp_sync_io = anchor; + cl_page_list_add(&queue->c2_qin, page); } @@ -1148,6 +1153,27 @@ static int ll_io_read_page(const struct lu_env *env, struct cl_io *io, if (!rc) task_io_account_read(PAGE_SIZE * count); } + if (anchor && !cl_page_is_owned(page, io)) { /* have sent */ + rc = cl_sync_io_wait(env, anchor, 0); + + cl_page_assume(env, io, page); + cl_page_list_del(env, &queue->c2_qout, page); + + if (!PageUptodate(cl_page_vmpage(page))) { + /* Failed to read a mirror, discard this page so that + * new page can be created with new mirror. + * + * TODO: this is not needed after page reinit + * route is implemented + */ + cl_page_discard(env, io, page); + } + cl_page_disown(env, io, page); + } + + /* TODO: discard all pages until page reinit route is implemented */ + cl_page_list_discard(env, io, &queue->c2_qin); + /* * Unlock unsent pages in case of error. */ diff --git a/fs/lustre/llite/vvp_internal.h b/fs/lustre/llite/vvp_internal.h index 0f7c027..f9a4552 100644 --- a/fs/lustre/llite/vvp_internal.h +++ b/fs/lustre/llite/vvp_internal.h @@ -127,6 +127,7 @@ struct vvp_thread_info { struct cl_lock_descr vti_descr; struct cl_io vti_io; struct cl_attr vti_attr; + struct cl_sync_io vti_anchor; }; static inline struct vvp_thread_info *vvp_env_info(const struct lu_env *env) diff --git a/fs/lustre/llite/vvp_io.c b/fs/lustre/llite/vvp_io.c index 3479f7a..8518423 100644 --- a/fs/lustre/llite/vvp_io.c +++ b/fs/lustre/llite/vvp_io.c @@ -281,6 +281,7 @@ static void vvp_io_fini(const struct lu_env *env, const struct cl_io_slice *ios) struct cl_object *obj = io->ci_obj; struct vvp_io *vio = cl2vvp_io(env, ios); struct inode *inode = vvp_object_inode(obj); + u32 gen = 0; int rc; CLOBINVRNT(env, obj, vvp_object_invariant(obj)); @@ -304,18 +305,41 @@ static void vvp_io_fini(const struct lu_env *env, const struct cl_io_slice *ios) * block on layout lock hold by the MDT * as MDT will not send new layout in lvb (see LU-3124) * we have to explicitly fetch it, all this will be done - * by ll_layout_refresh() + * by ll_layout_refresh(). + * Even if ll_layout_restore() returns zero, it doesn't mean + * that restore has been successful. Therefore it sets + * ci_verify_layout so that it will check layout at the end + * of this function. */ - if (rc == 0) { - io->ci_restore_needed = 0; - io->ci_need_restart = 1; - io->ci_verify_layout = 1; - } else { + if (rc) { io->ci_restore_needed = 1; io->ci_need_restart = 0; io->ci_verify_layout = 0; io->ci_result = rc; + goto out; + } + + io->ci_restore_needed = 0; + + /* Even if ll_layout_restore() returns zero, it doesn't mean + * that restore has been successful. Therefore it should verify + * if there was layout change and restart I/O correspondingly. + */ + ll_layout_refresh(inode, &gen); + io->ci_need_restart = vio->vui_layout_gen != gen; + if (io->ci_need_restart) { + CDEBUG(D_VFSTRACE, + DFID" layout changed from %d to %d.\n", + PFID(lu_object_fid(&obj->co_lu)), + vio->vui_layout_gen, gen); + /* today successful restore is the only possible + * case + */ + /* restore was done, clear restoring state */ + clear_bit(LLIF_FILE_RESTORING, + &ll_i2info(vvp_object_inode(obj))->lli_flags); } + goto out; } /** @@ -352,11 +376,11 @@ static void vvp_io_fini(const struct lu_env *env, const struct cl_io_slice *ios) io->ci_result = rc; if (!rc) io->ci_need_restart = 1; + goto out; } - if (!io->ci_ignore_layout && io->ci_verify_layout) { - u32 gen = 0; - + if (!io->ci_need_restart && + !io->ci_ignore_layout && io->ci_verify_layout) { /* check layout version */ ll_layout_refresh(inode, &gen); io->ci_need_restart = vio->vui_layout_gen != gen; @@ -365,12 +389,11 @@ static void vvp_io_fini(const struct lu_env *env, const struct cl_io_slice *ios) DFID " layout changed from %d to %d.\n", PFID(lu_object_fid(&obj->co_lu)), vio->vui_layout_gen, gen); - /* today successful restore is the only possible case */ - /* restore was done, clear restoring state */ - clear_bit(LLIF_FILE_RESTORING, - &ll_i2info(inode)->lli_flags); } + goto out; } +out: + ; } static void vvp_io_fault_fini(const struct lu_env *env, @@ -740,7 +763,10 @@ static int vvp_io_read_start(const struct lu_env *env, if (!can_populate_pages(env, io, inode)) return 0; - result = vvp_prep_size(env, obj, io, pos, tot, &exceed); + /* Unless this is reading a sparse file, otherwise the lock has already + * been acquired so vvp_prep_size() is an empty op. + */ + result = vvp_prep_size(env, obj, io, pos, cnt, &exceed); if (result != 0) return result; if (exceed != 0) @@ -765,6 +791,7 @@ static int vvp_io_read_start(const struct lu_env *env, file_accessed(file); LASSERT(vio->vui_iocb->ki_pos == pos); result = generic_file_read_iter(vio->vui_iocb, vio->vui_iter); + goto out; out: if (result >= 0) { diff --git a/fs/lustre/llite/vvp_page.c b/fs/lustre/llite/vvp_page.c index 590e5f5..7b07eae 100644 --- a/fs/lustre/llite/vvp_page.c +++ b/fs/lustre/llite/vvp_page.c @@ -267,8 +267,15 @@ static void vvp_page_completion_read(const struct lu_env *env, if (ioret == 0) { if (!vpg->vpg_defer_uptodate) cl_page_export(env, page, 1); - } else { + } else if (vpg->vpg_defer_uptodate) { vpg->vpg_defer_uptodate = 0; + if (ioret == -EWOULDBLOCK) { + /* mirror read failed, it needs to destroy the page + * because subpage would be from wrong osc when trying + * to read from a new mirror + */ + generic_error_remove_page(vmpage->mapping, vmpage); + } } if (!page->cp_sync_io) diff --git a/fs/lustre/lov/lov_cl_internal.h b/fs/lustre/lov/lov_cl_internal.h index 069b30e..94a5638 100644 --- a/fs/lustre/lov/lov_cl_internal.h +++ b/fs/lustre/lov/lov_cl_internal.h @@ -178,7 +178,7 @@ struct lov_layout_raid0 { * object. This field is reset to 0 when attributes of * any sub-object change. */ - int lo_attr_valid; + bool lo_attr_valid; /** * Array of sub-objects. Allocated when top-object is * created (lov_init_raid0()). @@ -217,7 +217,9 @@ struct lov_layout_dom { struct lov_layout_entry { u32 lle_type; - struct lu_extent lle_extent; + unsigned int lle_valid:1; + struct lu_extent *lle_extent; + struct lov_stripe_md_entry *lle_lsme; struct lov_comp_layout_entry_ops *lle_comp_ops; union { struct lov_layout_raid0 lle_raid0; @@ -225,6 +227,18 @@ struct lov_layout_entry { }; }; +struct lov_mirror_entry { + unsigned short lre_mirror_id; + unsigned short lre_preferred:1, + lre_valid:1; /* set if at least one of components + * in this mirror is valid + */ + unsigned short lre_start; /* index to lo_entries, start index of + * this mirror + */ + unsigned short lre_end; /* end index of this mirror */ +}; + /** * lov-specific file state. * @@ -280,9 +294,36 @@ struct lov_object { } released; struct lov_layout_composite { /** - * Current valid entry count of entries. + * flags of lov_comp_md_v1::lcm_flags. Mainly used + * by FLR. + */ + u32 lo_flags; + /** + * For FLR: index of preferred mirror to read. + * Preferred mirror is initialized by the preferred + * bit of lsme. It can be changed when the preferred + * is inaccessible. + * In order to make lov_lsm_entry() return the same + * mirror in the same IO context, it's only possible + * to change the preferred mirror when the + * lo_active_ios reaches zero. + */ + int lo_preferred_mirror; + /** + * For FLR: the lock to protect access to + * lo_preferred_mirror. */ - unsigned int lo_entry_count; + spinlock_t lo_write_lock; + /** + * For FLR: Number of (valid) mirrors. + */ + unsigned int lo_mirror_count; + struct lov_mirror_entry *lo_mirrors; + /** + * Current entry count of lo_entries, include + * invalid entries. + */ + unsigned int lo_entry_count; struct lov_layout_entry *lo_entries; } composite; } u; @@ -293,10 +334,80 @@ struct lov_object { struct task_struct *lo_owner; }; +static inline struct lov_layout_raid0 *lov_r0(struct lov_object *lov, int i) +{ + LASSERT(lov->lo_type == LLT_COMP); + LASSERTF(i < lov->u.composite.lo_entry_count, + "entry %d entry_count %d", i, lov->u.composite.lo_entry_count); + + return &lov->u.composite.lo_entries[i].lle_raid0; +} + +static inline struct lov_stripe_md_entry *lov_lse(struct lov_object *lov, int i) +{ + LASSERT(lov->lo_lsm); + LASSERT(i < lov->lo_lsm->lsm_entry_count); + + return lov->lo_lsm->lsm_entries[i]; +} + +static inline unsigned int lov_flr_state(const struct lov_object *lov) +{ + if (lov->lo_type != LLT_COMP) + return LCM_FL_NOT_FLR; + + return lov->u.composite.lo_flags & LCM_FL_FLR_MASK; +} + +static inline bool lov_is_flr(const struct lov_object *lov) +{ + return lov_flr_state(lov) != LCM_FL_NOT_FLR; +} + +static inline struct lov_layout_entry *lov_entry(struct lov_object *lov, int i) +{ + LASSERT(lov->lo_type == LLT_COMP); + LASSERTF(i < lov->u.composite.lo_entry_count, + "entry %d entry_count %d", i, lov->u.composite.lo_entry_count); + + return &lov->u.composite.lo_entries[i]; +} + +#define lov_for_layout_entry(lov, entry, start, end) \ + for (entry = lov_entry(lov, start); \ + entry <= lov_entry(lov, end); entry++) + #define lov_foreach_layout_entry(lov, entry) \ - for (entry = &lov->u.composite.lo_entries[0]; \ - entry < &lov->u.composite.lo_entries[lov->u.composite.lo_entry_count];\ - entry++) + lov_for_layout_entry(lov, entry, 0, \ + (lov)->u.composite.lo_entry_count - 1) + +#define lov_foreach_mirror_layout_entry(lov, entry, lre) \ + lov_for_layout_entry(lov, entry, (lre)->lre_start, (lre)->lre_end) + +static inline struct lov_mirror_entry * +lov_mirror_entry(struct lov_object *lov, int i) +{ + LASSERT(i < lov->u.composite.lo_mirror_count); + return &lov->u.composite.lo_mirrors[i]; +} + +#define lov_foreach_mirror_entry(lov, lre) \ + for (lre = lov_mirror_entry(lov, 0); \ + lre <= lov_mirror_entry(lov, \ + lov->u.composite.lo_mirror_count - 1);\ + lre++) + +static inline unsigned +lov_layout_entry_index(struct lov_object *lov, struct lov_layout_entry *entry) +{ + struct lov_layout_entry *first = &lov->u.composite.lo_entries[0]; + unsigned int index = (unsigned int)(entry - first); + + LASSERT(entry >= first); + LASSERT(index < lov->u.composite.lo_entry_count); + + return index; +} /** * State lov_lock keeps for each sub-lock. @@ -412,6 +523,26 @@ struct lov_io_sub { struct lov_io { /** super-class */ struct cl_io_slice lis_cl; + + /** + * FLR: index to lo_mirrors. Valid only if lov_is_flr() returns true. + * + * The mirror index of this io. Preserved over cl_io_init() + * if io->ci_ndelay_tried is greater than zero. + */ + int lis_mirror_index; + /** + * FLR: the layout gen when lis_mirror_index was cached. The + * mirror index makes sense only when the layout gen doesn't + * change. + */ + int lis_mirror_layout_gen; + + /** + * fields below this will be initialized in lov_io_init(). + */ + unsigned int lis_preserved; + /** * Pointer to the object slice. This is a duplicate of * lov_io::lis_cl::cis_object. @@ -518,10 +649,25 @@ struct lu_object *lovsub_object_alloc(const struct lu_env *env, struct lov_stripe_md *lov_lsm_addref(struct lov_object *lov); int lov_lsm_entry(const struct lov_stripe_md *lsm, u64 offset); +int lov_io_layout_at(struct lov_io *lio, u64 offset); #define lov_foreach_target(lov, var) \ for (var = 0; var < lov_targets_nr(lov); ++var) +static inline struct lu_extent *lov_io_extent(struct lov_io *io, int i) +{ + return &lov_lse(io->lis_object, i)->lsme_extent; +} + +/** + * For layout entries within @ext. + */ +#define lov_foreach_io_layout(ind, lio, ext) \ + for (ind = lov_io_layout_at(lio, (ext)->e_start); \ + ind >= 0 && \ + lu_extent_is_overlapped(lov_io_extent(lio, ind), ext); \ + ind = lov_io_layout_at(lio, lov_io_extent(lio, ind)->e_end)) + /***************************************************************************** * * Type conversions. @@ -690,32 +836,6 @@ static inline struct lov_thread_info *lov_env_info(const struct lu_env *env) return info; } -static inline struct lov_layout_entry *lov_entry(struct lov_object *lov, int i) -{ - LASSERT(lov->lo_type == LLT_COMP); - LASSERTF(i < lov->u.composite.lo_entry_count, - "entry %d entry_count %d", i, lov->u.composite.lo_entry_count); - - return &lov->u.composite.lo_entries[i]; -} - -static inline struct lov_layout_raid0 *lov_r0(struct lov_object *lov, int i) -{ - LASSERT(lov->lo_type == LLT_COMP); - LASSERTF(i < lov->u.composite.lo_entry_count, - "entry %d entry_count %d", i, lov->u.composite.lo_entry_count); - - return &lov->u.composite.lo_entries[i].lle_raid0; -} - -static inline struct lov_stripe_md_entry *lov_lse(struct lov_object *lov, int i) -{ - LASSERT(lov->lo_lsm); - LASSERT(i < lov->lo_lsm->lsm_entry_count); - - return lov->lo_lsm->lsm_entries[i]; -} - /* lov_pack.c */ int lov_getstripe(const struct lu_env *env, struct lov_object *obj, struct lov_stripe_md *lsm, struct lov_user_md __user *lump, diff --git a/fs/lustre/lov/lov_io.c b/fs/lustre/lov/lov_io.c index c3fd86a..12ea614 100644 --- a/fs/lustre/lov/lov_io.c +++ b/fs/lustre/lov/lov_io.c @@ -87,6 +87,15 @@ static void lov_io_sub_fini(const struct lu_env *env, struct lov_io *lio, } } +static inline bool +is_index_within_mirror(struct lov_object *lov, int index, int mirror_index) +{ + struct lov_layout_composite *comp = &lov->u.composite; + struct lov_mirror_entry *lre = &comp->lo_mirrors[mirror_index]; + + return (index >= lre->lre_start && index <= lre->lre_end); +} + static int lov_io_sub_init(const struct lu_env *env, struct lov_io *lio, struct lov_io_sub *sub) { @@ -104,6 +113,11 @@ static int lov_io_sub_init(const struct lu_env *env, struct lov_io *lio, !lov_r0(lov, index)->lo_sub[stripe])) return -EIO; + LASSERTF(is_index_within_mirror(lov, index, lio->lis_mirror_index), + DFID "iot = %d, index = %d, mirror = %d\n", + PFID(lu_object_fid(lov2lu(lov))), io->ci_type, index, + lio->lis_mirror_index); + /* obtain new environment */ sub->sub_env = cl_env_get(&sub->sub_refcheck); if (IS_ERR(sub->sub_env)) @@ -121,6 +135,7 @@ static int lov_io_sub_init(const struct lu_env *env, struct lov_io *lio, sub_io->ci_no_srvlock = io->ci_no_srvlock; sub_io->ci_noatime = io->ci_noatime; sub_io->ci_lock_no_expand = io->ci_lock_no_expand; + sub_io->ci_ndelay = io->ci_ndelay; rc = cl_io_sub_init(sub->sub_env, sub_io, io->ci_type, sub_obj); if (rc < 0) @@ -193,9 +208,102 @@ static int lov_io_subio_init(const struct lu_env *env, struct lov_io *lio, return 0; } +static int lov_io_mirror_init(struct lov_io *lio, struct lov_object *obj, + struct cl_io *io) +{ + struct lov_layout_composite *comp = &obj->u.composite; + int index; + int i; + + if (!lov_is_flr(obj)) { + LASSERT(comp->lo_preferred_mirror == 0); + lio->lis_mirror_index = comp->lo_preferred_mirror; + io->ci_ndelay = 0; + return 0; + } + + if (io->ci_ndelay_tried == 0 || /* first time to try */ + /* reset the mirror index if layout has changed */ + lio->lis_mirror_layout_gen != obj->lo_lsm->lsm_layout_gen) { + lio->lis_mirror_layout_gen = obj->lo_lsm->lsm_layout_gen; + index = lio->lis_mirror_index = comp->lo_preferred_mirror; + } else { + index = lio->lis_mirror_index; + LASSERT(index >= 0); + + /* move mirror index to the next one */ + index = (index + 1) % comp->lo_mirror_count; + } + + for (i = 0; i < comp->lo_mirror_count; i++) { + struct lu_extent ext = { .e_start = lio->lis_pos, + .e_end = lio->lis_pos + 1 }; + struct lov_mirror_entry *lre; + struct lov_layout_entry *lle; + bool found = false; + + lre = &comp->lo_mirrors[(index + i) % comp->lo_mirror_count]; + if (!lre->lre_valid) + continue; + + lov_foreach_mirror_layout_entry(obj, lle, lre) { + if (!lle->lle_valid) + continue; + + if (lu_extent_is_overlapped(&ext, lle->lle_extent)) { + found = true; + break; + } + } + + if (found) { + index = (index + i) % comp->lo_mirror_count; + break; + } + } + if (i == comp->lo_mirror_count) { + CERROR(DFID ": failed to find a component covering I/O region at %llu\n", + PFID(lu_object_fid(lov2lu(obj))), lio->lis_pos); + + dump_lsm(D_ERROR, obj->lo_lsm); + + return -EIO; + } + + CDEBUG(D_VFSTRACE, + DFID ": flr state: %d, move mirror from %d to %d, have retried: %d, mirror count: %d\n", + PFID(lu_object_fid(lov2lu(obj))), lov_flr_state(obj), + lio->lis_mirror_index, index, io->ci_ndelay_tried, + comp->lo_mirror_count); + + lio->lis_mirror_index = index; + + /* FLR: if all mirrors have been tried once, most likely the network + * of this client has been partitioned. We should relinquish CPU for + * a while before trying again. + */ + ++io->ci_ndelay_tried; + if (io->ci_ndelay && io->ci_ndelay_tried >= comp->lo_mirror_count) { + set_current_state(TASK_INTERRUPTIBLE); + schedule_timeout(msecs_to_jiffies(MSEC_PER_SEC)); /* 10ms */ + if (signal_pending(current)) + return -EINTR; + + /* reset retry counter */ + io->ci_ndelay_tried = 1; + } + + CDEBUG(D_VFSTRACE, "use %sdelayed RPC state for this IO\n", + io->ci_ndelay ? "non-" : ""); + + return 0; +} + static int lov_io_slice_init(struct lov_io *lio, struct lov_object *obj, struct cl_io *io) { + int result = 0; + io->ci_result = 0; lio->lis_object = obj; @@ -260,7 +368,8 @@ static int lov_io_slice_init(struct lov_io *lio, struct lov_object *obj, lio->lis_pos = 0; lio->lis_endpos = OBD_OBJECT_EOF; - if ((obj->lo_lsm->lsm_flags & LCM_FL_FLR_MASK) == LCM_FL_RDONLY) + if (lov_flr_state(obj) == LCM_FL_RDONLY && + !OBD_FAIL_CHECK(OBD_FAIL_FLR_GLIMPSE_IMMUTABLE)) return 1; /* SoM is accurate, no need glimpse */ break; @@ -272,7 +381,8 @@ static int lov_io_slice_init(struct lov_io *lio, struct lov_object *obj, default: LBUG(); } - return 0; + result = lov_io_mirror_init(lio, obj, io); + return result; } static void lov_io_fini(const struct lu_env *env, const struct cl_io_slice *ios) @@ -390,7 +500,6 @@ static int lov_io_iter_init(const struct lu_env *env, struct lov_io *lio = cl2lov_io(env, ios); struct lov_stripe_md *lsm = lio->lis_object->lo_lsm; struct cl_io *io = ios->cis_io; - struct lov_layout_entry *le; struct lov_io_sub *sub; struct lu_extent ext; int rc = 0; @@ -399,20 +508,15 @@ static int lov_io_iter_init(const struct lu_env *env, ext.e_start = lio->lis_pos; ext.e_end = lio->lis_endpos; - index = 0; - lov_foreach_layout_entry(lio->lis_object, le) { - struct lov_layout_raid0 *r0 = &le->lle_raid0; + lov_foreach_io_layout(index, lio, &ext) { + struct lov_layout_raid0 *r0 = lov_r0(lio->lis_object, index); int stripe; u64 start; u64 end; - index++; - if (!lu_extent_is_overlapped(&ext, &le->lle_extent)) - continue; - CDEBUG(D_VFSTRACE, "component[%d] flags %#x\n", - index - 1, lsm->lsm_entries[index - 1]->lsme_flags); - if (!lsm_entry_inited(lsm, index - 1)) { + index, lsm->lsm_entries[index]->lsme_flags); + if (!lsm_entry_inited(lsm, index)) { /* truncate IO will trigger write intent as well, and * it's handled in lov_io_setattr_iter_init() */ @@ -429,7 +533,7 @@ static int lov_io_iter_init(const struct lu_env *env, } for (stripe = 0; stripe < r0->lo_nr; stripe++) { - if (!lov_stripe_intersects(lsm, index - 1, stripe, + if (!lov_stripe_intersects(lsm, index, stripe, &ext, &start, &end)) continue; @@ -444,7 +548,7 @@ static int lov_io_iter_init(const struct lu_env *env, end = lov_offset_mod(end, 1); sub = lov_sub_get(env, lio, - lov_comp_index(index - 1, stripe)); + lov_comp_index(index, stripe)); if (IS_ERR(sub)) { rc = PTR_ERR(sub); break; @@ -472,7 +576,6 @@ static int lov_io_rw_iter_init(const struct lu_env *env, const struct cl_io_slice *ios) { struct lov_io *lio = cl2lov_io(env, ios); - struct lov_stripe_md *lsm = lio->lis_object->lo_lsm; struct cl_io *io = ios->cis_io; u64 start = io->u.ci_rw.crw_pos; struct lov_stripe_md_entry *lse; @@ -484,7 +587,7 @@ static int lov_io_rw_iter_init(const struct lu_env *env, if (cl_io_is_append(io)) return lov_io_iter_init(env, ios); - index = lov_lsm_entry(lsm, io->u.ci_rw.crw_pos); + index = lov_io_layout_at(lio, io->u.ci_rw.crw_pos); if (index < 0) { /* non-existing layout component */ if (io->ci_type == CIT_READ) { /* TODO: it needs to detect the next component and @@ -542,7 +645,7 @@ static int lov_io_setattr_iter_init(const struct lu_env *env, int index; if (cl_io_is_trunc(io) && lio->lis_pos > 0) { - index = lov_lsm_entry(lsm, lio->lis_pos - 1); + index = lov_io_layout_at(lio, lio->lis_pos - 1); /* no entry found for such offset */ if (index < 0) { io->ci_result = -ENODATA; @@ -676,7 +779,7 @@ static int lov_io_read_ahead(const struct lu_env *env, int rc; offset = cl_offset(obj, start); - index = lov_lsm_entry(loo->lo_lsm, offset); + index = lov_io_layout_at(lio, offset); if (index < 0 || !lsm_entry_inited(loo->lo_lsm, index)) return -ENODATA; @@ -715,7 +818,7 @@ static int lov_io_read_ahead(const struct lu_env *env, ra_end, stripe); /* boundary of current component */ - ra_end = cl_index(obj, (loff_t)lov_lse(loo, index)->lsme_extent.e_end); + ra_end = cl_index(obj, (loff_t)lov_io_extent(lio, index)->e_end); if (ra_end != CL_PAGE_EOF && ra->cra_end >= ra_end) ra->cra_end = ra_end - 1; @@ -1148,8 +1251,8 @@ int lov_io_init_released(const struct lu_env *env, struct cl_object *obj, LASSERTF(0, "invalid type %d\n", io->ci_type); result = -EOPNOTSUPP; break; - case CIT_MISC: case CIT_GLIMPSE: + case CIT_MISC: case CIT_FSYNC: case CIT_LADVISE: case CIT_DATA_VERSION: @@ -1184,4 +1287,42 @@ int lov_io_init_released(const struct lu_env *env, struct cl_object *obj, return result; } -/** @} lov */ +/** + * Return the index in composite:lo_entries by the file offset + */ +int lov_io_layout_at(struct lov_io *lio, u64 offset) +{ + struct lov_object *lov = lio->lis_object; + struct lov_layout_composite *comp = &lov->u.composite; + int start_index = 0; + int end_index = comp->lo_entry_count - 1; + int i; + + LASSERT(lov->lo_type == LLT_COMP); + + /* This is actual file offset so nothing can cover eof. */ + if (offset == LUSTRE_EOF) + return -1; + + if (lov_is_flr(lov)) { + struct lov_mirror_entry *lre; + + LASSERT(lio->lis_mirror_index >= 0); + + lre = &comp->lo_mirrors[lio->lis_mirror_index]; + start_index = lre->lre_start; + end_index = lre->lre_end; + } + + for (i = start_index; i <= end_index; i++) { + struct lov_layout_entry *lle = lov_entry(lov, i); + + if ((offset >= lle->lle_extent->e_start && + offset < lle->lle_extent->e_end) || + (offset == OBD_OBJECT_EOF && + lle->lle_extent->e_end == OBD_OBJECT_EOF)) + return i; + } + + return -1; +} diff --git a/fs/lustre/lov/lov_lock.c b/fs/lustre/lov/lov_lock.c index 5520199..0c10c3a 100644 --- a/fs/lustre/lov/lov_lock.c +++ b/fs/lustre/lov/lov_lock.c @@ -131,15 +131,9 @@ static struct lov_lock *lov_lock_sub_init(const struct lu_env *env, ext.e_end = cl_offset(obj, lock->cll_descr.cld_end + 1); nr = 0; - for (index = lov_lsm_entry(lov->lo_lsm, ext.e_start); - index >= 0 && index < lov->lo_lsm->lsm_entry_count; index++) { + lov_foreach_io_layout(index, lov_env_io(env), &ext) { struct lov_layout_raid0 *r0 = lov_r0(lov, index); - /* assume lsm entries are sorted. */ - if (!lu_extent_is_overlapped(&ext, - &lov_lse(lov, index)->lsme_extent)) - break; - for (i = 0; i < r0->lo_nr; i++) { if (likely(r0->lo_sub[i]) && /* spare layout */ lov_stripe_intersects(lov->lo_lsm, index, i, @@ -160,14 +154,9 @@ static struct lov_lock *lov_lock_sub_init(const struct lu_env *env, lovlck->lls_nr = nr; nr = 0; - for (index = lov_lsm_entry(lov->lo_lsm, ext.e_start); - index >= 0 && index < lov->lo_lsm->lsm_entry_count; index++) { + lov_foreach_io_layout(index, lov_env_io(env), &ext) { struct lov_layout_raid0 *r0 = lov_r0(lov, index); - /* assume lsm entries are sorted. */ - if (!lu_extent_is_overlapped(&ext, - &lov_lse(lov, index)->lsme_extent)) - break; for (i = 0; i < r0->lo_nr; ++i) { struct lov_lock_sub *lls = &lovlck->lls_sub[nr]; struct cl_lock_descr *descr = &lls->sub_lock.cll_descr; diff --git a/fs/lustre/lov/lov_object.c b/fs/lustre/lov/lov_object.c index cb9b108..42acc78 100644 --- a/fs/lustre/lov/lov_object.c +++ b/fs/lustre/lov/lov_object.c @@ -437,8 +437,8 @@ static int lov_attr_get_dom(const struct lu_env *env, struct lov_object *lov, * component end. Alternatively, check that limit on server * and do not allow size overflow there. */ - if (attr->cat_size > lle->lle_extent.e_end) - attr->cat_size = lle->lle_extent.e_end; + if (attr->cat_size > lle->lle_extent->e_end) + attr->cat_size = lle->lle_extent->e_end; attr->cat_kms = attr->cat_size; @@ -604,19 +604,38 @@ static int lov_init_composite(const struct lu_env *env, struct lov_device *dev, union lov_layout_state *state) { struct lov_layout_composite *comp = &state->composite; + int flr_state = lsm->lsm_flags & LCM_FL_FLR_MASK; struct lov_layout_entry *lle; + struct lov_mirror_entry *lre; unsigned int entry_count; unsigned int psz = 0; + unsigned int mirror_count; int result = 0; - int i; + int i, j; LASSERT(lsm->lsm_entry_count > 0); LASSERT(!lov->lo_lsm); lov->lo_lsm = lsm_addref(lsm); lov->lo_layout_invalid = true; + dump_lsm(D_INODE, lsm); + entry_count = lsm->lsm_entry_count; - comp->lo_entry_count = entry_count; + + spin_lock_init(&comp->lo_write_lock); + comp->lo_flags = lsm->lsm_flags; + comp->lo_mirror_count = lsm->lsm_mirror_count + 1; + comp->lo_entry_count = lsm->lsm_entry_count; + comp->lo_preferred_mirror = -1; + + if (equi(flr_state == LCM_FL_NOT_FLR, comp->lo_mirror_count > 1)) + return -EINVAL; + + comp->lo_mirrors = kcalloc(comp->lo_mirror_count, + sizeof(*comp->lo_mirrors), + GFP_NOFS); + if (!comp->lo_mirrors) + return -ENOMEM; comp->lo_entries = kcalloc(entry_count, sizeof(*comp->lo_entries), GFP_NOFS); @@ -624,10 +643,13 @@ static int lov_init_composite(const struct lu_env *env, struct lov_device *dev, return -ENOMEM; /* Initiate all entry types and extents data at first */ - for (i = 0; i < entry_count; i++) { + for (i = 0, j = 0, mirror_count = 1; i < entry_count; i++) { + int mirror_id = 0; + lle = &comp->lo_entries[i]; - lle->lle_type = lov_entry_type(lsm->lsm_entries[i]); + lle->lle_lsme = lsm->lsm_entries[i]; + lle->lle_type = lov_entry_type(lle->lle_lsme); switch (lle->lle_type) { case LOV_PATTERN_RAID0: lle->lle_comp_ops = &raid0_ops; @@ -642,30 +664,99 @@ static int lov_init_composite(const struct lu_env *env, struct lov_device *dev, dump_lsm(D_ERROR, lsm); return -EIO; } - lle->lle_extent = lsm->lsm_entries[i]->lsme_extent; + + lle->lle_extent = &lle->lle_lsme->lsme_extent; + lle->lle_valid = !(lle->lle_lsme->lsme_flags & LCME_FL_STALE); + + if (flr_state != LCM_FL_NOT_FLR) + mirror_id = mirror_id_of(lle->lle_lsme->lsme_id); + + lre = &comp->lo_mirrors[j]; + if (i > 0) { + if (mirror_id == lre->lre_mirror_id) { + lre->lre_valid |= lle->lle_valid; + lre->lre_end = i; + continue; + } + + /* new mirror detected, assume that the mirrors + * are shorted in layout + */ + ++mirror_count; + ++j; + if (j >= comp->lo_mirror_count) + break; + + lre = &comp->lo_mirrors[j]; + } + + /* entries must be sorted by mirrors */ + lre->lre_mirror_id = mirror_id; + lre->lre_start = lre->lre_end = i; + lre->lre_preferred = (lle->lle_lsme->lsme_flags & + LCME_FL_PREFERRED); + lre->lre_valid = lle->lle_valid; + } + + /* sanity check for FLR */ + if (mirror_count != comp->lo_mirror_count) { + CDEBUG(D_INODE, DFID + " doesn't have the # of mirrors it claims, %u/%u\n", + PFID(lu_object_fid(lov2lu(lov))), mirror_count, + comp->lo_mirror_count + 1); + + result = -EINVAL; + goto out; } - i = 0; lov_foreach_layout_entry(lov, lle) { + int index = lov_layout_entry_index(lov, lle); + /** * If the component has not been init-ed on MDS side, for * PFL layout, we'd know that the components beyond this one * will be dynamically init-ed later on file write/trunc ops. */ - if (lsm_entry_inited(lsm, i)) { - result = lle->lle_comp_ops->lco_init(env, dev, lov, i, - conf, lle); - if (result < 0) - break; + if (!lsme_inited(lle->lle_lsme)) + continue; - LASSERT(ergo(psz > 0, psz == result)); - psz = result; - } - i++; + result = lle->lle_comp_ops->lco_init(env, dev, lov, index, + conf, lle); + if (result < 0) + break; + + LASSERT(ergo(psz > 0, psz == result)); + psz = result; } + if (psz > 0) cl_object_header(&lov->lo_cl)->coh_page_bufsize += psz; + /* decide the preferred mirror */ + mirror_count = 0; + i = 0; + lov_foreach_mirror_entry(lov, lre) { + i++; + if (!lre->lre_valid) + continue; + + mirror_count++; /* valid mirror */ + + if (lre->lre_preferred || comp->lo_preferred_mirror < 0) + comp->lo_preferred_mirror = i - 1; + } + if (mirror_count == 0) { + CDEBUG(D_INODE, DFID + " doesn't have any valid mirrors\n", + PFID(lu_object_fid(lov2lu(lov)))); + + result = -EINVAL; + goto out; + } + + LASSERT(comp->lo_preferred_mirror >= 0); + +out: return result > 0 ? 0 : result; } @@ -739,6 +830,10 @@ static void lov_fini_composite(const struct lu_env *env, comp->lo_entries = NULL; } + kfree(comp->lo_mirrors); + + memset(comp, 0, sizeof(*comp)); + dump_lsm(D_INODE, lov->lo_lsm); lov_free_memmd(&lov->lo_lsm); } @@ -821,24 +916,25 @@ static int lov_attr_get_composite(const struct lu_env *env, struct lov_object *lov = cl2lov(obj); struct lov_layout_entry *entry; int result = 0; - int index = 0; attr->cat_size = 0; attr->cat_blocks = 0; lov_foreach_layout_entry(lov, entry) { + int index = lov_layout_entry_index(lov, entry); struct cl_attr *lov_attr = NULL; + if (!entry->lle_valid) + continue; + /* PFL: This component has not been init-ed. */ if (!lsm_entry_inited(lov->lo_lsm, index)) - break; + continue; result = entry->lle_comp_ops->lco_getattr(env, lov, index, entry, &lov_attr); if (result < 0) return result; - index++; - if (!lov_attr) continue; @@ -861,6 +957,7 @@ static int lov_attr_get_composite(const struct lu_env *env, if (attr->cat_mtime < lov_attr->cat_mtime) attr->cat_mtime = lov_attr->cat_mtime; } + return 0; } @@ -1051,12 +1148,11 @@ static int lov_layout_change(const struct lu_env *unused, CDEBUG(D_INODE, DFID "Apply new layout lov %p, type %d\n", PFID(lu_object_fid(lov2lu(lov))), lov, llt); - lov->lo_type = LLT_EMPTY; - /* page bufsize fixup */ cl_object_header(&lov->lo_cl)->coh_page_bufsize -= lov_page_slice_fixup(lov, NULL); + lov->lo_type = llt; rc = new_ops->llo_init(env, lov_dev, lov, lsm, conf, state); if (rc) { struct obd_device *obd = lov2obd(lov_dev->ld_lov); @@ -1066,10 +1162,10 @@ static int lov_layout_change(const struct lu_env *unused, new_ops->llo_delete(env, lov, state); new_ops->llo_fini(env, lov, state); /* this file becomes an EMPTY file. */ + lov->lo_type = LLT_EMPTY; goto out; } - lov->lo_type = llt; out: cl_env_put(env, &refcheck); return rc; @@ -1218,7 +1314,7 @@ int lov_page_init(const struct lu_env *env, struct cl_object *obj, int lov_io_init(const struct lu_env *env, struct cl_object *obj, struct cl_io *io) { - CL_IO_SLICE_CLEAN(lov_env_io(env), lis_cl); + CL_IO_SLICE_CLEAN(lov_env_io(env), lis_preserved); CDEBUG(D_INODE, DFID "io %p type %d ignore/verify layout %d/%d\n", PFID(lu_object_fid(&obj->co_lu)), io, io->ci_type, @@ -1767,6 +1863,7 @@ static int lov_object_fiemap(const struct lu_env *env, struct cl_object *obj, goto out_fm_local; } + /* TODO: rewrite it with lov_foreach_io_layout() */ for (entry = start_entry; entry <= end_entry; entry++) { lsme = lsm->lsm_entries[entry]; diff --git a/fs/lustre/lov/lov_page.c b/fs/lustre/lov/lov_page.c index e64b350..ad2a4e7 100644 --- a/fs/lustre/lov/lov_page.c +++ b/fs/lustre/lov/lov_page.c @@ -82,7 +82,7 @@ int lov_page_init_composite(const struct lu_env *env, struct cl_object *obj, int rc; offset = cl_offset(obj, index); - entry = lov_lsm_entry(loo->lo_lsm, offset); + entry = lov_io_layout_at(lio, offset); if (entry < 0 || !lsm_entry_inited(loo->lo_lsm, entry)) { /* non-existing layout component */ lov_page_init_empty(env, obj, page, index); diff --git a/fs/lustre/obdclass/cl_io.c b/fs/lustre/obdclass/cl_io.c index 3dc5c87..ca94eb3 100644 --- a/fs/lustre/obdclass/cl_io.c +++ b/fs/lustre/obdclass/cl_io.c @@ -191,6 +191,9 @@ int cl_io_init(const struct lu_env *env, struct cl_io *io, { LASSERT(obj == cl_object_top(obj)); + /* clear I/O restart from previous instance */ + io->ci_need_restart = 0; + return __cl_io_init(env, io, iot, obj); } EXPORT_SYMBOL(cl_io_init); @@ -722,6 +725,12 @@ int cl_io_loop(const struct lu_env *env, struct cl_io *io) } cl_io_iter_fini(env, io); } while (result == 0 && io->ci_continue); + + if (result == -EWOULDBLOCK && io->ci_ndelay) { + io->ci_need_restart = 1; + result = 0; + } + if (result == 0) result = io->ci_result; return result < 0 ? result : 0; @@ -917,8 +926,8 @@ static void cl_page_list_assume(const struct lu_env *env, /** * Discards all pages in a queue. */ -static void cl_page_list_discard(const struct lu_env *env, struct cl_io *io, - struct cl_page_list *plist) +void cl_page_list_discard(const struct lu_env *env, struct cl_io *io, + struct cl_page_list *plist) { struct cl_page *page; @@ -926,6 +935,7 @@ static void cl_page_list_discard(const struct lu_env *env, struct cl_io *io, cl_page_list_for_each(page, plist) cl_page_discard(env, io, page); } +EXPORT_SYMBOL(cl_page_list_discard); /** * Initialize dual page queue. diff --git a/fs/lustre/osc/osc_cache.c b/fs/lustre/osc/osc_cache.c index 7a707a1..4ddca32 100644 --- a/fs/lustre/osc/osc_cache.c +++ b/fs/lustre/osc/osc_cache.c @@ -1916,6 +1916,7 @@ static int try_to_add_extent_for_io(struct client_obd *cli, if (tmp->oe_srvlock != ext->oe_srvlock || !tmp->oe_grants != !ext->oe_grants || + tmp->oe_ndelay != ext->oe_ndelay || tmp->oe_no_merge || ext->oe_no_merge) return 0; @@ -2604,7 +2605,7 @@ int osc_flush_async_page(const struct lu_env *env, struct cl_io *io, } int osc_queue_sync_pages(const struct lu_env *env, struct osc_object *obj, - struct list_head *list, int cmd, int brw_flags) + struct list_head *list, int brw_flags) { struct client_obd *cli = osc_cli(obj); struct osc_extent *ext; @@ -2642,7 +2643,7 @@ int osc_queue_sync_pages(const struct lu_env *env, struct osc_object *obj, return -ENOMEM; } - ext->oe_rw = !!(cmd & OBD_BRW_READ); + ext->oe_rw = !!(brw_flags & OBD_BRW_READ); ext->oe_sync = 1; ext->oe_no_merge = !can_merge; ext->oe_urgent = 1; @@ -2651,6 +2652,7 @@ int osc_queue_sync_pages(const struct lu_env *env, struct osc_object *obj, ext->oe_max_end = end; ext->oe_obj = obj; ext->oe_srvlock = !!(brw_flags & OBD_BRW_SRVLOCK); + ext->oe_ndelay = !!(brw_flags & OBD_BRW_NDELAY); ext->oe_nr_pages = page_count; ext->oe_mppr = mppr; list_splice_init(list, &ext->oe_pages); @@ -2658,7 +2660,7 @@ int osc_queue_sync_pages(const struct lu_env *env, struct osc_object *obj, osc_object_lock(obj); /* Reuse the initial refcount for RPC, don't drop it */ osc_extent_state_set(ext, OES_LOCK_DONE); - if (cmd & OBD_BRW_WRITE) { + if (!ext->oe_rw) { /* write */ list_add_tail(&ext->oe_link, &obj->oo_urgent_exts); osc_update_pending(obj, OBD_BRW_WRITE, page_count); } else { diff --git a/fs/lustre/osc/osc_io.c b/fs/lustre/osc/osc_io.c index d2e2f7f..b26d513 100644 --- a/fs/lustre/osc/osc_io.c +++ b/fs/lustre/osc/osc_io.c @@ -120,7 +120,6 @@ int osc_io_submit(const struct lu_env *env, const struct cl_io_slice *ios, struct cl_page_list *qout = &queue->c2_qout; unsigned int queued = 0; int result = 0; - int cmd; int brw_flags; unsigned int max_pages; @@ -132,8 +131,10 @@ int osc_io_submit(const struct lu_env *env, const struct cl_io_slice *ios, cli = osc_cli(osc); max_pages = cli->cl_max_pages_per_rpc; - cmd = crt == CRT_WRITE ? OBD_BRW_WRITE : OBD_BRW_READ; brw_flags = osc_io_srvlock(cl2osc_io(env, ios)) ? OBD_BRW_SRVLOCK : 0; + brw_flags |= crt == CRT_WRITE ? OBD_BRW_WRITE : OBD_BRW_READ; + if (crt == CRT_READ && ios->cis_io->ci_ndelay) + brw_flags |= OBD_BRW_NDELAY; /* * NOTE: here @page is a top-level page. This is done to avoid @@ -187,7 +188,7 @@ int osc_io_submit(const struct lu_env *env, const struct cl_io_slice *ios, if (++queued == max_pages) { queued = 0; - result = osc_queue_sync_pages(env, osc, &list, cmd, + result = osc_queue_sync_pages(env, osc, &list, brw_flags); if (result < 0) break; @@ -195,7 +196,7 @@ int osc_io_submit(const struct lu_env *env, const struct cl_io_slice *ios, } if (queued > 0) - result = osc_queue_sync_pages(env, osc, &list, cmd, brw_flags); + result = osc_queue_sync_pages(env, osc, &list, brw_flags); /* Update c/mtime for sync write. LU-7310 */ if (crt == CRT_WRITE && qout->pl_nr > 0 && !result) { diff --git a/fs/lustre/osc/osc_lock.c b/fs/lustre/osc/osc_lock.c index 84dda93..1ede75c 100644 --- a/fs/lustre/osc/osc_lock.c +++ b/fs/lustre/osc/osc_lock.c @@ -301,6 +301,8 @@ static int osc_lock_upcall(void *cookie, struct lustre_handle *lockh, NULL, &oscl->ols_lvb); /* Hide the error. */ rc = 0; + } else if (rc < 0 && oscl->ols_flags & LDLM_FL_NDELAY) { + rc = -EWOULDBLOCK; } if (oscl->ols_owner) @@ -1167,6 +1169,8 @@ int osc_lock_init(const struct lu_env *env, oscl->ols_flags |= LDLM_FL_BLOCK_GRANTED; oscl->ols_glimpse = 1; } + if (io->ci_ndelay && cl_object_same(io->ci_obj, obj)) + oscl->ols_flags |= LDLM_FL_NDELAY; osc_lock_build_einfo(env, lock, cl2osc(obj), &oscl->ols_einfo); cl_lock_slice_add(lock, &oscl->ols_cl, obj, &osc_lock_ops); diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index e164d6a..06ecd20 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -1790,7 +1790,7 @@ static int brw_interpret(const struct lu_env *env, /* When server return -EINPROGRESS, client should always retry * regardless of the number of times the bulk was resent already. */ - if (osc_recoverable_error(rc)) { + if (osc_recoverable_error(rc) && !req->rq_no_delay) { if (req->rq_import_generation != req->rq_import->imp_generation) { CDEBUG(D_HA, @@ -1872,7 +1872,8 @@ static int brw_interpret(const struct lu_env *env, list_for_each_entry_safe(ext, tmp, &aa->aa_exts, oe_link) { list_del_init(&ext->oe_link); - osc_extent_finish(env, ext, 1, rc); + osc_extent_finish(env, ext, 1, + rc && req->rq_no_delay ? -EWOULDBLOCK : rc); } LASSERT(list_empty(&aa->aa_exts)); LASSERT(list_empty(&aa->aa_oaps)); @@ -1942,6 +1943,7 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, int page_count = 0; bool soft_sync = false; int grant = 0; + bool ndelay = false; int i; int rc; struct ost_body *body; @@ -1999,6 +2001,8 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, LASSERT(oap->oap_page_off + oap->oap_count == PAGE_SIZE); } + if (ext->oe_ndelay) + ndelay = true; } /* first page in the list */ @@ -2027,6 +2031,13 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, req->rq_memalloc = mem_tight != 0; oap->oap_request = ptlrpc_request_addref(req); + if (ndelay) { + req->rq_no_resend = req->rq_no_delay = 1; + /* probably set a shorter timeout value. + * to handle ETIMEDOUT in brw_interpret() correctly. + */ + /* lustre_msg_set_timeout(req, req->rq_timeout / 2); */ + } /* Need to update the timestamps after the request is built in case * we race with setattr (locally or in queue at OST). If OST gets diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index 951d501..a088f4a 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -1181,6 +1181,10 @@ struct hsm_state_set { #define OBD_BRW_READ 0x01 #define OBD_BRW_WRITE 0x02 #define OBD_BRW_RWMASK (OBD_BRW_READ | OBD_BRW_WRITE) +#define OBD_BRW_NDELAY 0x04 /* Non-delay RPC should be issued for + * this page. Non-delay RPCs have bit + * rq_no_delay set. + */ #define OBD_BRW_SYNC 0x08 /* this page is a part of synchronous * transfer and is not accounted in * the grant. From patchwork Mon Sep 30 18:55:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167185 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 06B0D16B1 for ; Mon, 30 Sep 2019 19:02:13 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E32DC224EF for ; Mon, 30 Sep 2019 19:02:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E32DC224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id EC4E55E4378; Mon, 30 Sep 2019 11:59:14 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id F231B5C3A37 for ; Mon, 30 Sep 2019 11:57:19 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 072CC10058DD; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 04F29B5; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:27 -0400 Message-Id: <1569869810-23848-69-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 068/151] lustre: lov: rework write intent on componect instantiation X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Jinshan Xiong It's revised to do in the I/O initialization. WC-bug-id: https://jira.whamcloud.com/browse/LU-9771 Lustre-commit: 47a07d5e7933 ("LU-9971 lov: rework write intent on componect instantiation") Signed-off-by: Jinshan Xiong Reviewed-on: https://review.whamcloud.com/29086 Reviewed-by: Bobi Jam Reviewed-by: Fan Yong Signed-off-by: James Simmons --- fs/lustre/include/cl_object.h | 4 ++++ fs/lustre/llite/vvp_io.c | 25 +++++------------------ fs/lustre/lov/lov_io.c | 46 +++++++++++++++++++++++++++++-------------- 3 files changed, 40 insertions(+), 35 deletions(-) diff --git a/fs/lustre/include/cl_object.h b/fs/lustre/include/cl_object.h index f573c6f..45068ca 100644 --- a/fs/lustre/include/cl_object.h +++ b/fs/lustre/include/cl_object.h @@ -1889,6 +1889,10 @@ struct cl_io { * Number of pages owned by this IO. For invariant checking. */ unsigned int ci_owned_nr; + /** + * Range of write intent. Valid if ci_need_write_intent is set. + */ + struct lu_extent ci_write_intent; }; /** @} cl_io */ diff --git a/fs/lustre/llite/vvp_io.c b/fs/lustre/llite/vvp_io.c index 8518423..dc9ccef 100644 --- a/fs/lustre/llite/vvp_io.c +++ b/fs/lustre/llite/vvp_io.c @@ -347,32 +347,17 @@ static void vvp_io_fini(const struct lu_env *env, const struct cl_io_slice *ios) * RPC. */ if (io->ci_need_write_intent) { - loff_t end = OBD_OBJECT_EOF; - loff_t start = 0; - io->ci_need_write_intent = 0; LASSERT(io->ci_type == CIT_WRITE || cl_io_is_trunc(io) || cl_io_is_mkwrite(io)); - if (io->ci_type == CIT_WRITE) { - if (!cl_io_is_append(io)) { - start = io->u.ci_rw.crw_pos; - end = start + io->u.ci_rw.crw_count; - } - } else if (cl_io_is_trunc(io)) { - end = io->u.ci_setattr.sa_attr.lvb_size; - } else { /* mkwrite */ - pgoff_t index = io->u.ci_fault.ft_index; - - start = cl_offset(io->ci_obj, index); - end = cl_offset(io->ci_obj, index + 1); - } - - CDEBUG(D_VFSTRACE, DFID" type %d [%llx, %llx)\n", + CDEBUG(D_VFSTRACE, DFID" write layout, type %u " DEXT "\n", PFID(lu_object_fid(&obj->co_lu)), io->ci_type, - start, end); - rc = ll_layout_write_intent(inode, start, end); + PEXT(&io->ci_write_intent)); + + rc = ll_layout_write_intent(inode, io->ci_write_intent.e_start, + io->ci_write_intent.e_end); io->ci_result = rc; if (!rc) io->ci_need_restart = 1; diff --git a/fs/lustre/lov/lov_io.c b/fs/lustre/lov/lov_io.c index 12ea614..22ca77e 100644 --- a/fs/lustre/lov/lov_io.c +++ b/fs/lustre/lov/lov_io.c @@ -302,6 +302,8 @@ static int lov_io_mirror_init(struct lov_io *lio, struct lov_object *obj, static int lov_io_slice_init(struct lov_io *lio, struct lov_object *obj, struct cl_io *io) { + struct lu_extent ext; + int index; int result = 0; io->ci_result = 0; @@ -382,6 +384,35 @@ static int lov_io_slice_init(struct lov_io *lio, struct lov_object *obj, LBUG(); } result = lov_io_mirror_init(lio, obj, io); + if (result) + return result; + + /* check if it needs to instantiate layout */ + if (!(io->ci_type == CIT_WRITE || cl_io_is_mkwrite(io) || + (cl_io_is_trunc(io) && io->u.ci_setattr.sa_attr.lvb_size > 0))) + return 0; + + ext.e_start = lio->lis_pos; + ext.e_end = lio->lis_endpos; + + /* for truncate, it only needs to instantiate the components + * before the truncated size. + */ + if (cl_io_is_trunc(io)) { + ext.e_start = 0; + ext.e_end = io->u.ci_setattr.sa_attr.lvb_size; + } + + index = 0; + lov_foreach_io_layout(index, lio, &ext) { + if (!lsm_entry_inited(obj->lo_lsm, index)) { + io->ci_need_write_intent = 1; + io->ci_write_intent = ext; + result = 1; + break; + } + } + return result; } @@ -499,7 +530,6 @@ static int lov_io_iter_init(const struct lu_env *env, { struct lov_io *lio = cl2lov_io(env, ios); struct lov_stripe_md *lsm = lio->lis_object->lo_lsm; - struct cl_io *io = ios->cis_io; struct lov_io_sub *sub; struct lu_extent ext; int rc = 0; @@ -517,15 +547,6 @@ static int lov_io_iter_init(const struct lu_env *env, CDEBUG(D_VFSTRACE, "component[%d] flags %#x\n", index, lsm->lsm_entries[index]->lsme_flags); if (!lsm_entry_inited(lsm, index)) { - /* truncate IO will trigger write intent as well, and - * it's handled in lov_io_setattr_iter_init() - */ - if (io->ci_type == CIT_WRITE || cl_io_is_mkwrite(io)) { - io->ci_need_write_intent = 1; - rc = -ENODATA; - break; - } - /* Read from uninitialized components should return * zero filled pages. */ @@ -641,7 +662,6 @@ static int lov_io_setattr_iter_init(const struct lu_env *env, { struct lov_io *lio = cl2lov_io(env, ios); struct cl_io *io = ios->cis_io; - struct lov_stripe_md *lsm = lio->lis_object->lo_lsm; int index; if (cl_io_is_trunc(io) && lio->lis_pos > 0) { @@ -650,10 +670,6 @@ static int lov_io_setattr_iter_init(const struct lu_env *env, if (index < 0) { io->ci_result = -ENODATA; return io->ci_result; - } else if (!lsm_entry_inited(lsm, index)) { - io->ci_need_write_intent = 1; - io->ci_result = -ENODATA; - return io->ci_result; } } From patchwork Mon Sep 30 18:55:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167175 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 261B5912 for ; Mon, 30 Sep 2019 19:01:40 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0DEFF224EF for ; Mon, 30 Sep 2019 19:01:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0DEFF224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 5011321FAC3; Mon, 30 Sep 2019 11:59:07 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 54F645C3A37 for ; Mon, 30 Sep 2019 11:57:20 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 0962010058E1; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 07BE2BB; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:28 -0400 Message-Id: <1569869810-23848-70-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 069/151] lustre: ptlrpc: use lu_extent in layout_intent X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Jinshan Xiong This way it will be easier to check and print the extent of layout. WC-bug-id: https://jira.whamcloud.com/browse/LU-9771 Lustre-commit: c1f450270dda ("LU-9971 ptlrpc: use lu_extent in layout_intent") Signed-off-by: Jinshan Xiong Reviewed-on: https://review.whamcloud.com/29087 Reviewed-by: Lai Siyao Reviewed-by: Bobi Jam Signed-off-by: James Simmons --- include/uapi/linux/lustre/lustre_idl.h | 3 +-- include/uapi/linux/lustre/lustre_user.h | 2 +- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index a088f4a..b4f7aec 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -2825,8 +2825,7 @@ enum { struct layout_intent { __u32 li_opc; /* intent operation for enqueue, read, write etc */ __u32 li_flags; - __u64 li_start; - __u64 li_end; + struct lu_extent li_extent; } __packed; /** diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index 7d9f0d2..d4372d9 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -426,7 +426,7 @@ struct lu_extent { __u64 e_end; }; -#define DEXT "[ %#llx , %#llx )" +#define DEXT "[%#llx, %#llx)" #define PEXT(ext) (ext)->e_start, (ext)->e_end static inline bool lu_extent_is_overlapped(struct lu_extent *e1, From patchwork Mon Sep 30 18:55:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167093 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6263313BD for ; Mon, 30 Sep 2019 18:58:34 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4A77D224D5 for ; Mon, 30 Sep 2019 18:58:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4A77D224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 46E3E5C3D2E; Mon, 30 Sep 2019 11:57:50 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 9573F5C3A37 for ; Mon, 30 Sep 2019 11:57:20 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 0C96910058E2; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 0AEDBA9; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:29 -0400 Message-Id: <1569869810-23848-71-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 070/151] lustre: flr: Send write intent RPC to mdt X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Jinshan Xiong When a mirrored file is going to be written, the client needs to send a write intent RPC to the MDT. The MDT will pick a mirror as primary and mark the others as stale. The new md operation moo_layout_change() is introduced for this purpose. The MDT also transfers the latest layout version to the OST objects via do_attr_set(). Once OSTs receive the setattr RPC for layout version change, it will set the update layout version into extended attribute XATTR_NAME_FID. WC-bug-id: https://jira.whamcloud.com/browse/LU-9771 Lustre-commit: 10da8afb2786 ("LU-9771 flr: Send write intent RPC to mdt") Signed-off-by: Jinshan Xiong Reviewed-on: https://review.whamcloud.com/29091 Reviewed-by: Lai Siyao Reviewed-by: Bobi Jam Signed-off-by: James Simmons --- fs/lustre/include/cl_object.h | 8 ++- fs/lustre/include/lustre_osc.h | 7 +- fs/lustre/include/obd.h | 5 +- fs/lustre/include/obd_support.h | 2 + fs/lustre/llite/file.c | 4 +- fs/lustre/llite/vvp_object.c | 7 ++ fs/lustre/lov/lov_cl_internal.h | 3 + fs/lustre/lov/lov_io.c | 121 +++++++++++++++++++++++++++++--- fs/lustre/lov/lov_object.c | 4 +- fs/lustre/lov/lov_page.c | 5 +- fs/lustre/osc/osc_cache.c | 9 ++- fs/lustre/osc/osc_io.c | 10 ++- fs/lustre/osc/osc_request.c | 12 +++- fs/lustre/ptlrpc/pack_generic.c | 11 ++- fs/lustre/ptlrpc/wiretest.c | 20 +++--- include/uapi/linux/lustre/lustre_idl.h | 6 +- include/uapi/linux/lustre/lustre_user.h | 10 ++- 17 files changed, 202 insertions(+), 42 deletions(-) diff --git a/fs/lustre/include/cl_object.h b/fs/lustre/include/cl_object.h index 45068ca..ef7ba76 100644 --- a/fs/lustre/include/cl_object.h +++ b/fs/lustre/include/cl_object.h @@ -1775,6 +1775,8 @@ struct cl_io { struct cl_lockset ci_lockset; /** lock requirements, this is just a help info for sublayers. */ enum cl_io_lock_dmd ci_lockreq; + /** layout version when this IO occurs */ + u32 ci_layout_version; union { struct cl_rd_io { struct cl_io_rw_common rd; @@ -1850,8 +1852,10 @@ struct cl_io { */ ci_ignore_layout:1, /** - * Need MDS intervention to complete a write. This usually means the - * corresponding component is not initialized for the writing extent. + * Need MDS intervention to complete a write. + * Write intent is required for the following cases: + * 1. component being written is not initialized, or + * 2. the mirrored files are NOT in WRITE_PENDING state. */ ci_need_write_intent:1, /** diff --git a/fs/lustre/include/lustre_osc.h b/fs/lustre/include/lustre_osc.h index 5efceef..a81e802 100644 --- a/fs/lustre/include/lustre_osc.h +++ b/fs/lustre/include/lustre_osc.h @@ -586,8 +586,9 @@ int osc_teardown_async_page(const struct lu_env *env, struct osc_object *obj, struct osc_page *ops); int osc_flush_async_page(const struct lu_env *env, struct cl_io *io, struct osc_page *ops); -int osc_queue_sync_pages(const struct lu_env *env, struct osc_object *obj, - struct list_head *list, int brw_flags); +int osc_queue_sync_pages(const struct lu_env *env, const struct cl_io *io, + struct osc_object *obj, struct list_head *list, + int brw_flags); int osc_cache_truncate_start(const struct lu_env *env, struct osc_object *obj, u64 size, struct osc_extent **extp); void osc_cache_truncate_end(const struct lu_env *env, struct osc_extent *ext); @@ -968,6 +969,8 @@ struct osc_extent { int oe_rc; /* max pages per rpc when this extent was created */ unsigned int oe_mppr; + /* FLR: layout version when this osc_extent is publised */ + u32 oe_layout_version; }; /* @} osc */ diff --git a/fs/lustre/include/obd.h b/fs/lustre/include/obd.h index 2f586cb..c377a91 100644 --- a/fs/lustre/include/obd.h +++ b/fs/lustre/include/obd.h @@ -687,9 +687,10 @@ static inline int it_to_lock_mode(struct lookup_intent *it) /* CREAT needs to be tested before open (both could be set) */ if (it->it_op & IT_CREAT) return LCK_CW; - else if (it->it_op & (IT_GETATTR | IT_OPEN | IT_LOOKUP | - IT_LAYOUT)) + else if (it->it_op & (IT_GETATTR | IT_OPEN | IT_LOOKUP)) return LCK_CR; + else if (it->it_op & IT_LAYOUT) + return (it->it_flags & FMODE_WRITE) ? LCK_EX : LCK_CR; else if (it->it_op & IT_READDIR) return LCK_PR; else if (it->it_op & IT_GETXATTR) diff --git a/fs/lustre/include/obd_support.h b/fs/lustre/include/obd_support.h index e6dff44b..9d383f5 100644 --- a/fs/lustre/include/obd_support.h +++ b/fs/lustre/include/obd_support.h @@ -476,6 +476,8 @@ /* FLR */ #define OBD_FAIL_FLR_GLIMPSE_IMMUTABLE 0x1A00 +#define OBD_FAIL_FLR_LV_DELAY 0x1A01 +#define OBD_FAIL_FLR_LV_INC 0x1A02 /* Assign references to moved code to reduce code changes */ #define OBD_FAIL_PRECHECK(id) CFS_FAIL_PRECHECK(id) diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 1856aa6..9b441ba 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -4206,8 +4206,8 @@ int ll_layout_write_intent(struct inode *inode, u64 start, u64 end) { struct layout_intent intent = { .li_opc = LAYOUT_INTENT_WRITE, - .li_start = start, - .li_end = end, + .li_extent.e_start = start, + .li_extent.e_end = end, }; int rc; diff --git a/fs/lustre/llite/vvp_object.c b/fs/lustre/llite/vvp_object.c index 1637972..549837f 100644 --- a/fs/lustre/llite/vvp_object.c +++ b/fs/lustre/llite/vvp_object.c @@ -165,6 +165,13 @@ static int vvp_prune(const struct lu_env *env, struct cl_object *obj) } truncate_inode_pages(inode->i_mapping, 0); + if (inode->i_mapping->nrpages) { + CDEBUG(D_VFSTRACE, DFID ": still has %lu pages remaining\n", + PFID(lu_object_fid(&obj->co_lu)), + inode->i_mapping->nrpages); + return -EIO; + } + return 0; } diff --git a/fs/lustre/lov/lov_cl_internal.h b/fs/lustre/lov/lov_cl_internal.h index 94a5638..ad4a3d3 100644 --- a/fs/lustre/lov/lov_cl_internal.h +++ b/fs/lustre/lov/lov_cl_internal.h @@ -230,6 +230,7 @@ struct lov_layout_entry { struct lov_mirror_entry { unsigned short lre_mirror_id; unsigned short lre_preferred:1, + lre_stale:1, /* set if any components is stale */ lre_valid:1; /* set if at least one of components * in this mirror is valid */ @@ -438,6 +439,8 @@ struct lov_page { struct cl_page_slice lps_cl; /** layout_entry + stripe index, composed using lov_comp_index() */ unsigned int lps_index; + /* the layout gen when this page was created */ + u32 lps_layout_gen; }; /* diff --git a/fs/lustre/lov/lov_io.c b/fs/lustre/lov/lov_io.c index 22ca77e..3d3f07f 100644 --- a/fs/lustre/lov/lov_io.c +++ b/fs/lustre/lov/lov_io.c @@ -136,6 +136,7 @@ static int lov_io_sub_init(const struct lu_env *env, struct lov_io *lio, sub_io->ci_noatime = io->ci_noatime; sub_io->ci_lock_no_expand = io->ci_lock_no_expand; sub_io->ci_ndelay = io->ci_ndelay; + sub_io->ci_layout_version = io->ci_layout_version; rc = cl_io_sub_init(sub->sub_env, sub_io, io->ci_type, sub_obj); if (rc < 0) @@ -208,12 +209,88 @@ static int lov_io_subio_init(const struct lu_env *env, struct lov_io *lio, return 0; } +/** + * Decide if it will need write intent RPC + */ +static int lov_io_mirror_write_intent(struct lov_io *lio, + struct lov_object *obj, struct cl_io *io) +{ + struct lov_layout_composite *comp = &obj->u.composite; + struct lu_extent *ext = &io->ci_write_intent; + struct lov_mirror_entry *lre; + struct lov_mirror_entry *primary; + struct lov_layout_entry *lle; + size_t count = 0; + + *ext = (typeof(*ext)) { lio->lis_pos, lio->lis_endpos }; + io->ci_need_write_intent = 0; + + if (!(io->ci_type == CIT_WRITE || cl_io_is_trunc(io) || + cl_io_is_mkwrite(io))) + return 0; + + if (lov_flr_state(obj) == LCM_FL_RDONLY || + lov_flr_state(obj) == LCM_FL_SYNC_PENDING) { + io->ci_need_write_intent = 1; + return 0; + } + + LASSERT((lov_flr_state(obj) == LCM_FL_WRITE_PENDING)); + LASSERT(comp->lo_preferred_mirror >= 0); + + /* need to iterate all components to see if there are + * multiple components covering the writing component + */ + primary = &comp->lo_mirrors[comp->lo_preferred_mirror]; + LASSERT(!primary->lre_stale); + lov_foreach_mirror_layout_entry(obj, lle, primary) { + LASSERT(lle->lle_valid); + if (!lu_extent_is_overlapped(ext, lle->lle_extent)) + continue; + + ext->e_start = min(ext->e_start, lle->lle_extent->e_start); + ext->e_end = max(ext->e_end, lle->lle_extent->e_end); + ++count; + } + if (count == 0) { + CERROR(DFID ": cannot find any valid components covering file extent " DEXT ", mirror: %d\n", + PFID(lu_object_fid(lov2lu(obj))), PEXT(ext), + primary->lre_mirror_id); + return -EIO; + } + + count = 0; + lov_foreach_mirror_entry(obj, lre) { + if (lre == primary) + continue; + + lov_foreach_mirror_layout_entry(obj, lle, lre) { + if (!lle->lle_valid) + continue; + + if (lu_extent_is_overlapped(ext, lle->lle_extent)) { + ++count; + break; + } + } + } + + CDEBUG(D_VFSTRACE, + DFID "there are %zd components to be staled to modify file extent " DEXT ", iot: %d\n", + PFID(lu_object_fid(lov2lu(obj))), count, PEXT(ext), io->ci_type); + + io->ci_need_write_intent = count > 0; + + return 0; +} + static int lov_io_mirror_init(struct lov_io *lio, struct lov_object *obj, struct cl_io *io) { struct lov_layout_composite *comp = &obj->u.composite; int index; int i; + int result; if (!lov_is_flr(obj)) { LASSERT(comp->lo_preferred_mirror == 0); @@ -222,6 +299,22 @@ static int lov_io_mirror_init(struct lov_io *lio, struct lov_object *obj, return 0; } + result = lov_io_mirror_write_intent(lio, obj, io); + if (result) + return result; + + if (io->ci_need_write_intent) { + CDEBUG(D_VFSTRACE, DFID " need write intent for [%llu, %llu)\n", + PFID(lu_object_fid(lov2lu(obj))), + lio->lis_pos, lio->lis_endpos); + + /* stop cl_io_init() loop */ + return 1; + } + + /* transfer the layout version for verification */ + io->ci_layout_version = obj->lo_lsm->lsm_layout_gen; + if (io->ci_ndelay_tried == 0 || /* first time to try */ /* reset the mirror index if layout has changed */ lio->lis_mirror_layout_gen != obj->lo_lsm->lsm_layout_gen) { @@ -325,8 +418,10 @@ static int lov_io_slice_init(struct lov_io *lio, struct lov_object *obj, * the current file-tail exactly. */ if (unlikely(obj->lo_lsm->lsm_entries[0]->lsme_pattern & - LOV_PATTERN_F_HOLE)) - return -EIO; + LOV_PATTERN_F_HOLE)) { + result = -EIO; + goto out; + } lio->lis_pos = 0; lio->lis_endpos = OBD_OBJECT_EOF; @@ -371,8 +466,11 @@ static int lov_io_slice_init(struct lov_io *lio, struct lov_object *obj, lio->lis_endpos = OBD_OBJECT_EOF; if (lov_flr_state(obj) == LCM_FL_RDONLY && - !OBD_FAIL_CHECK(OBD_FAIL_FLR_GLIMPSE_IMMUTABLE)) - return 1; /* SoM is accurate, no need glimpse */ + !OBD_FAIL_CHECK(OBD_FAIL_FLR_GLIMPSE_IMMUTABLE)) { + /* SoM is accurate, no need glimpse */ + result = 1; + goto out; + } break; case CIT_MISC: @@ -385,12 +483,14 @@ static int lov_io_slice_init(struct lov_io *lio, struct lov_object *obj, } result = lov_io_mirror_init(lio, obj, io); if (result) - return result; + goto out; /* check if it needs to instantiate layout */ if (!(io->ci_type == CIT_WRITE || cl_io_is_mkwrite(io) || - (cl_io_is_trunc(io) && io->u.ci_setattr.sa_attr.lvb_size > 0))) - return 0; + (cl_io_is_trunc(io) && io->u.ci_setattr.sa_attr.lvb_size > 0))) { + result = 0; + goto out; + } ext.e_start = lio->lis_pos; ext.e_end = lio->lis_endpos; @@ -409,10 +509,11 @@ static int lov_io_slice_init(struct lov_io *lio, struct lov_object *obj, io->ci_need_write_intent = 1; io->ci_write_intent = ext; result = 1; - break; + goto out; } } +out: return result; } @@ -799,6 +900,10 @@ static int lov_io_read_ahead(const struct lu_env *env, if (index < 0 || !lsm_entry_inited(loo->lo_lsm, index)) return -ENODATA; + /* avoid readahead to expand to stale components */ + if (!lov_entry(loo, index)->lle_valid) + return -EIO; + stripe = lov_stripe_number(loo->lo_lsm, index, offset); r0 = lov_r0(loo, index); diff --git a/fs/lustre/lov/lov_object.c b/fs/lustre/lov/lov_object.c index 42acc78..f606e04 100644 --- a/fs/lustre/lov/lov_object.c +++ b/fs/lustre/lov/lov_object.c @@ -675,6 +675,7 @@ static int lov_init_composite(const struct lu_env *env, struct lov_device *dev, if (i > 0) { if (mirror_id == lre->lre_mirror_id) { lre->lre_valid |= lle->lle_valid; + lre->lre_stale |= !lle->lle_valid; lre->lre_end = i; continue; } @@ -696,6 +697,7 @@ static int lov_init_composite(const struct lu_env *env, struct lov_device *dev, lre->lre_preferred = (lle->lle_lsme->lsme_flags & LCME_FL_PREFERRED); lre->lre_valid = lle->lle_valid; + lre->lre_stale = !lle->lle_valid; } /* sanity check for FLR */ @@ -737,7 +739,7 @@ static int lov_init_composite(const struct lu_env *env, struct lov_device *dev, i = 0; lov_foreach_mirror_entry(lov, lre) { i++; - if (!lre->lre_valid) + if (lre->lre_stale) continue; mirror_count++; /* valid mirror */ diff --git a/fs/lustre/lov/lov_page.c b/fs/lustre/lov/lov_page.c index ad2a4e7..327bcd3 100644 --- a/fs/lustre/lov/lov_page.c +++ b/fs/lustre/lov/lov_page.c @@ -57,8 +57,8 @@ static int lov_comp_page_print(const struct lu_env *env, struct lov_page *lp = cl2lov_page(slice); return (*printer)(env, cookie, - LUSTRE_LOV_NAME "-page@%p, comp index: %x\n", - lp, lp->lps_index); + LUSTRE_LOV_NAME "-page@%p, comp index: %x, gen: %u\n", + lp, lp->lps_index, lp->lps_layout_gen); } static const struct cl_page_operations lov_comp_page_ops = { @@ -96,6 +96,7 @@ int lov_page_init_composite(const struct lu_env *env, struct cl_object *obj, LASSERT(rc == 0); lpg->lps_index = lov_comp_index(entry, stripe); + lpg->lps_layout_gen = loo->lo_lsm->lsm_layout_gen; cl_page_slice_add(page, &lpg->lps_cl, obj, index, &lov_comp_page_ops); sub = lov_sub_get(env, lio, lpg->lps_index); diff --git a/fs/lustre/osc/osc_cache.c b/fs/lustre/osc/osc_cache.c index 4ddca32..e387b7a 100644 --- a/fs/lustre/osc/osc_cache.c +++ b/fs/lustre/osc/osc_cache.c @@ -2479,6 +2479,9 @@ int osc_queue_async_io(const struct lu_env *env, struct cl_io *io, ++ext->oe_nr_pages; list_add_tail(&oap->oap_pending_item, &ext->oe_pages); osc_object_unlock(osc); + + if (!ext->oe_layout_version) + ext->oe_layout_version = io->ci_layout_version; } return rc; @@ -2604,8 +2607,9 @@ int osc_flush_async_page(const struct lu_env *env, struct cl_io *io, return rc; } -int osc_queue_sync_pages(const struct lu_env *env, struct osc_object *obj, - struct list_head *list, int brw_flags) +int osc_queue_sync_pages(const struct lu_env *env, const struct cl_io *io, + struct osc_object *obj, struct list_head *list, + int brw_flags) { struct client_obd *cli = osc_cli(obj); struct osc_extent *ext; @@ -2656,6 +2660,7 @@ int osc_queue_sync_pages(const struct lu_env *env, struct osc_object *obj, ext->oe_nr_pages = page_count; ext->oe_mppr = mppr; list_splice_init(list, &ext->oe_pages); + ext->oe_layout_version = io->ci_layout_version; osc_object_lock(obj); /* Reuse the initial refcount for RPC, don't drop it */ diff --git a/fs/lustre/osc/osc_io.c b/fs/lustre/osc/osc_io.c index b26d513..8bdfadb 100644 --- a/fs/lustre/osc/osc_io.c +++ b/fs/lustre/osc/osc_io.c @@ -188,7 +188,7 @@ int osc_io_submit(const struct lu_env *env, const struct cl_io_slice *ios, if (++queued == max_pages) { queued = 0; - result = osc_queue_sync_pages(env, osc, &list, + result = osc_queue_sync_pages(env, io, osc, &list, brw_flags); if (result < 0) break; @@ -196,7 +196,7 @@ int osc_io_submit(const struct lu_env *env, const struct cl_io_slice *ios, } if (queued > 0) - result = osc_queue_sync_pages(env, osc, &list, brw_flags); + result = osc_queue_sync_pages(env, io, osc, &list, brw_flags); /* Update c/mtime for sync write. LU-7310 */ if (crt == CRT_WRITE && qout->pl_nr > 0 && !result) { @@ -558,6 +558,12 @@ static int osc_io_setattr_start(const struct lu_env *env, oa->o_flags = OBD_FL_SRVLOCK; oa->o_valid |= OBD_MD_FLFLAGS; } + + if (io->ci_layout_version > 0) { + /* verify layout version */ + oa->o_valid |= OBD_MD_LAYOUT_VERSION; + oa->o_layout_version = io->ci_layout_version; + } } else { LASSERT(oio->oi_lockless == 0); } diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index 06ecd20..5581f42 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -1944,6 +1944,7 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, bool soft_sync = false; int grant = 0; bool ndelay = false; + u32 layout_version = 0; int i; int rc; struct ost_body *body; @@ -1957,6 +1958,7 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, mem_tight |= ext->oe_memalloc; grant += ext->oe_grants; page_count += ext->oe_nr_pages; + layout_version = max(layout_version, ext->oe_layout_version); if (!obj) obj = ext->oe_obj; } @@ -2016,8 +2018,16 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, crattr->cra_oa = oa; cl_req_attr_set(env, osc2cl(obj), crattr); - if (cmd == OBD_BRW_WRITE) + if (cmd == OBD_BRW_WRITE) { oa->o_grant_used = grant; + if (layout_version > 0) { + CDEBUG(D_LAYOUT, DFID": write with layout version %u\n", + PFID(&oa->o_oi.oi_fid), layout_version); + + oa->o_layout_version = layout_version; + oa->o_valid |= OBD_MD_LAYOUT_VERSION; + } + } sort_brw_pages(pga, page_count); rc = osc_brw_prep_request(cmd, cli, oa, page_count, pga, &req, 1, 0); diff --git a/fs/lustre/ptlrpc/pack_generic.c b/fs/lustre/ptlrpc/pack_generic.c index eb82eaa..0c73da6 100644 --- a/fs/lustre/ptlrpc/pack_generic.c +++ b/fs/lustre/ptlrpc/pack_generic.c @@ -1619,7 +1619,7 @@ static void lustre_swab_obdo(struct obdo *o) __swab32s(&o->o_stripe_idx); __swab32s(&o->o_parent_ver); lustre_swab_ost_layout(&o->o_layout); - BUILD_BUG_ON(offsetof(typeof(*o), o_padding_3) == 0); + __swab32s(&o->o_layout_version); __swab32s(&o->o_uid_h); __swab32s(&o->o_gid_h); __swab64s(&o->o_data_version); @@ -2374,12 +2374,17 @@ void lustre_swab_hsm_user_item(struct hsm_user_item *hui) lustre_swab_hsm_extent(&hui->hui_extent); } +void lustre_swab_lu_extent(struct lu_extent *le) +{ + __swab64s(&le->e_start); + __swab64s(&le->e_end); +} + void lustre_swab_layout_intent(struct layout_intent *li) { __swab32s(&li->li_opc); __swab32s(&li->li_flags); - __swab64s(&li->li_start); - __swab64s(&li->li_end); + lustre_swab_lu_extent(&li->li_extent); } void lustre_swab_hsm_progress_kernel(struct hsm_progress_kernel *hpk) diff --git a/fs/lustre/ptlrpc/wiretest.c b/fs/lustre/ptlrpc/wiretest.c index 749effb..0b3c6af 100644 --- a/fs/lustre/ptlrpc/wiretest.c +++ b/fs/lustre/ptlrpc/wiretest.c @@ -1247,10 +1247,10 @@ void lustre_assert_wire_constants(void) (long long)(int)offsetof(struct obdo, o_layout)); LASSERTF((int)sizeof(((struct obdo *)0)->o_layout) == 28, "found %lld\n", (long long)(int)sizeof(((struct obdo *)0)->o_layout)); - LASSERTF((int)offsetof(struct obdo, o_padding_3) == 164, "found %lld\n", - (long long)(int)offsetof(struct obdo, o_padding_3)); - LASSERTF((int)sizeof(((struct obdo *)0)->o_padding_3) == 4, "found %lld\n", - (long long)(int)sizeof(((struct obdo *)0)->o_padding_3)); + LASSERTF((int)offsetof(struct obdo, o_layout_version) == 164, "found %lld\n", + (long long)(int)offsetof(struct obdo, o_layout_version)); + LASSERTF((int)sizeof(((struct obdo *)0)->o_layout_version) == 4, "found %lld\n", + (long long)(int)sizeof(((struct obdo *)0)->o_layout_version)); LASSERTF((int)offsetof(struct obdo, o_uid_h) == 168, "found %lld\n", (long long)(int)offsetof(struct obdo, o_uid_h)); LASSERTF((int)sizeof(((struct obdo *)0)->o_uid_h) == 4, "found %lld\n", @@ -4049,14 +4049,10 @@ void lustre_assert_wire_constants(void) (long long)(int)offsetof(struct layout_intent, li_flags)); LASSERTF((int)sizeof(((struct layout_intent *)0)->li_flags) == 4, "found %lld\n", (long long)(int)sizeof(((struct layout_intent *)0)->li_flags)); - LASSERTF((int)offsetof(struct layout_intent, li_start) == 8, "found %lld\n", - (long long)(int)offsetof(struct layout_intent, li_start)); - LASSERTF((int)sizeof(((struct layout_intent *)0)->li_start) == 8, "found %lld\n", - (long long)(int)sizeof(((struct layout_intent *)0)->li_start)); - LASSERTF((int)offsetof(struct layout_intent, li_end) == 16, "found %lld\n", - (long long)(int)offsetof(struct layout_intent, li_end)); - LASSERTF((int)sizeof(((struct layout_intent *)0)->li_end) == 8, "found %lld\n", - (long long)(int)sizeof(((struct layout_intent *)0)->li_end)); + LASSERTF((int)offsetof(struct layout_intent, li_extent) == 8, "found %lld\n", + (long long)(int)offsetof(struct layout_intent, li_extent)); + LASSERTF((int)sizeof(((struct layout_intent *)0)->li_extent) == 16, "found %lld\n", + (long long)(int)sizeof(((struct layout_intent *)0)->li_extent)); LASSERTF(LAYOUT_INTENT_ACCESS == 0, "found %lld\n", (long long)LAYOUT_INTENT_ACCESS); LASSERTF(LAYOUT_INTENT_READ == 1, "found %lld\n", diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index b4f7aec..27146a6 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -1092,7 +1092,9 @@ static inline __u32 lov_mds_md_size(__u16 stripes, __u32 lmm_magic) #define OBD_MD_DOM_SIZE (0x00001000ULL) /* Data-on-MDT component size */ #define OBD_MD_FLNLINK (0x00002000ULL) /* link count */ #define OBD_MD_FLGENER (0x00004000ULL) /* generation number */ -/*#define OBD_MD_FLINLINE (0x00008000ULL) inline data. used until 1.6.5 */ +#define OBD_MD_LAYOUT_VERSION (0x00008000ULL) /* layout version for + * OST objects + */ #define OBD_MD_FLRDEV (0x00010000ULL) /* device number */ #define OBD_MD_FLEASIZE (0x00020000ULL) /* extended attribute data */ #define OBD_MD_LINKNAME (0x00040000ULL) /* symbolic link target */ @@ -2675,7 +2677,7 @@ struct obdo { * sizeof(ost_layout) + sizeof(__u32) == sizeof(llog_cookie). */ struct ost_layout o_layout; - __u32 o_padding_3; + __u32 o_layout_version; __u32 o_uid_h; __u32 o_gid_h; diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index d4372d9..3b64f734 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -453,6 +453,11 @@ enum lov_comp_md_entry_flags { #define LCME_KNOWN_FLAGS (LCME_FL_NEG | LCME_FL_INIT) +/* the highest bit in obdo::o_layout_version is used to mark if the file is + * being resynced. + */ +#define LU_LAYOUT_RESYNC LCME_FL_NEG + /* lcme_id can be specified as certain flags, and the first * bit of lcme_id is used to indicate that the ID is representing * certain LCME_FL_* but not a real ID. Which implies we can have @@ -834,6 +839,8 @@ enum changelog_rec_type { CL_MTIME = 17, /* Precedence: setattr > mtime > ctime > atime */ CL_CTIME = 18, CL_ATIME = 19, + CL_FLRW = 21, /* FLR: file was firstly written */ + CL_RESYNC = 22, /* FLR: file was resync-ed */ CL_LAST }; @@ -842,7 +849,8 @@ static inline const char *changelog_type2str(int type) static const char *changelog_str[] = { "MARK", "CREAT", "MKDIR", "HLINK", "SLINK", "MKNOD", "UNLNK", "RMDIR", "RENME", "RNMTO", "OPEN", "CLOSE", "LYOUT", "TRUNC", - "SATTR", "XATTR", "HSM", "MTIME", "CTIME", "ATIME", + "SATTR", "XATTR", "HSM", "MTIME", "CTIME", "ATIME", "", + "FLRW", "RESYNC", }; if (type >= 0 && type < CL_LAST) From patchwork Mon Sep 30 18:55:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167097 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9CBE913BD for ; Mon, 30 Sep 2019 18:58:41 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 84E2F224D5 for ; Mon, 30 Sep 2019 18:58:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 84E2F224D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 382E25C3D53; Mon, 30 Sep 2019 11:57:54 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id EAA485C3A37 for ; Mon, 30 Sep 2019 11:57:20 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 1101510058E4; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 0E38DB4; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:30 -0400 Message-Id: <1569869810-23848-72-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 071/151] lustre: flr: extend DATA_VERSION API to read layout version X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Jinshan Xiong The API is extended to fetch layout version from ost objects. WC-bug-id: https://jira.whamcloud.com/browse/LU-9771 Lustre-commit: cdbe35c08c66 ("LU-9771 flr: extend DATA_VERSION API to read layout version") Signed-off-by: Jinshan Xiong Reviewed-on: https://review.whamcloud.com/29092 Reviewed-by: Bobi Jam Reviewed-by: Dmitry Eremin Signed-off-by: James Simmons --- fs/lustre/include/cl_object.h | 1 + fs/lustre/llite/file.c | 52 +++++++++++++++++++++------------ fs/lustre/lov/lov_io.c | 8 +++-- fs/lustre/osc/osc_io.c | 11 +++++-- include/uapi/linux/lustre/lustre_user.h | 11 ++++--- 5 files changed, 56 insertions(+), 27 deletions(-) diff --git a/fs/lustre/include/cl_object.h b/fs/lustre/include/cl_object.h index ef7ba76..d164294 100644 --- a/fs/lustre/include/cl_object.h +++ b/fs/lustre/include/cl_object.h @@ -1798,6 +1798,7 @@ struct cl_io { } ci_setattr; struct cl_data_version_io { u64 dv_data_version; + u32 dv_layout_version; int dv_flags; } ci_data_version; struct cl_fault_io { diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 9b441ba..05a9fea 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -1851,18 +1851,8 @@ int ll_fid2path(struct inode *inode, void __user *arg) return rc; } -/* - * Read the data_version for inode. - * - * This value is computed using stripe object version on OST. - * Version is computed using server side locking. - * - * @flags: if do sync on the OST side; - * 0: no sync - * LL_DV_RD_FLUSH: flush dirty pages, LCK_PR on OSTs - * LL_DV_WR_FLUSH: drop all caching pages, LCK_PW on OSTs - */ -int ll_data_version(struct inode *inode, u64 *data_version, int flags) +static int +ll_ioc_data_version(struct inode *inode, struct ioc_data_version *ioc) { struct cl_object *obj = ll_i2info(inode)->lli_clob; struct lu_env *env; @@ -1870,11 +1860,12 @@ int ll_data_version(struct inode *inode, u64 *data_version, int flags) u16 refcheck; int result; + ioc->idv_version = 0; + ioc->idv_layout_version = UINT_MAX; + /* If no file object initialized, we consider its version is 0. */ - if (!obj) { - *data_version = 0; + if (!obj) return 0; - } env = cl_env_get(&refcheck); if (IS_ERR(env)) @@ -1883,7 +1874,8 @@ int ll_data_version(struct inode *inode, u64 *data_version, int flags) io = vvp_env_thread_io(env); io->ci_obj = obj; io->u.ci_data_version.dv_data_version = 0; - io->u.ci_data_version.dv_flags = flags; + io->u.ci_data_version.dv_layout_version = UINT_MAX; + io->u.ci_data_version.dv_flags = ioc->idv_flags; restart: if (!cl_io_init(env, io, CIT_DATA_VERSION, io->ci_obj)) @@ -1891,7 +1883,8 @@ int ll_data_version(struct inode *inode, u64 *data_version, int flags) else result = io->ci_result; - *data_version = io->u.ci_data_version.dv_data_version; + ioc->idv_version = io->u.ci_data_version.dv_data_version; + ioc->idv_layout_version = io->u.ci_data_version.dv_layout_version; cl_io_fini(env, io); @@ -1904,6 +1897,29 @@ int ll_data_version(struct inode *inode, u64 *data_version, int flags) } /* + * Read the data_version for inode. + * + * This value is computed using stripe object version on OST. + * Version is computed using server side locking. + * + * @param flags if do sync on the OST side; + * 0: no sync + * LL_DV_RD_FLUSH: flush dirty pages, LCK_PR on OSTs + * LL_DV_WR_FLUSH: drop all caching pages, LCK_PW on OSTs + */ +int ll_data_version(struct inode *inode, u64 *data_version, int flags) +{ + struct ioc_data_version ioc = { .idv_flags = flags }; + int rc; + + rc = ll_ioc_data_version(inode, &ioc); + if (!rc) + *data_version = ioc.idv_version; + + return rc; +} + +/* * Trigger a HSM release request for the provided inode. */ int ll_hsm_release(struct inode *inode) @@ -2677,7 +2693,7 @@ int ll_ioctl_fssetxattr(struct inode *inode, unsigned int cmd, return -EFAULT; idv.idv_flags &= LL_DV_RD_FLUSH | LL_DV_WR_FLUSH; - rc = ll_data_version(inode, &idv.idv_version, idv.idv_flags); + rc = ll_ioc_data_version(inode, &idv); if (rc == 0 && copy_to_user((char __user *)arg, &idv, sizeof(idv))) return -EFAULT; diff --git a/fs/lustre/lov/lov_io.c b/fs/lustre/lov/lov_io.c index 3d3f07f..457fa3c 100644 --- a/fs/lustre/lov/lov_io.c +++ b/fs/lustre/lov/lov_io.c @@ -844,13 +844,17 @@ static void lov_io_end(const struct lu_env *env, const struct cl_io_slice *ios) { struct lov_io *lio = cl2lov_io(env, ios); struct cl_io *parent = lio->lis_cl.cis_io; + struct cl_data_version_io *pdv = &parent->u.ci_data_version; struct lov_io_sub *sub; list_for_each_entry(sub, &lio->lis_active, sub_linkage) { + struct cl_data_version_io *sdv = &sub->sub_io.u.ci_data_version; + lov_io_end_wrapper(sub->sub_env, &sub->sub_io); - parent->u.ci_data_version.dv_data_version += - sub->sub_io.u.ci_data_version.dv_data_version; + pdv->dv_data_version += sdv->dv_data_version; + if (pdv->dv_layout_version > sdv->dv_layout_version) + pdv->dv_layout_version = sdv->dv_layout_version; if (!parent->ci_result) parent->ci_result = sub->sub_io.ci_result; diff --git a/fs/lustre/osc/osc_io.c b/fs/lustre/osc/osc_io.c index 8bdfadb..d75f725 100644 --- a/fs/lustre/osc/osc_io.c +++ b/fs/lustre/osc/osc_io.c @@ -714,11 +714,16 @@ static void osc_io_data_version_end(const struct lu_env *env, if (cbargs->opc_rc) { slice->cis_io->ci_result = cbargs->opc_rc; - } else if (!(oio->oi_oa.o_valid & OBD_MD_FLDATAVERSION)) { - slice->cis_io->ci_result = -EOPNOTSUPP; } else { - dv->dv_data_version = oio->oi_oa.o_data_version; slice->cis_io->ci_result = 0; + if (!(oio->oi_oa.o_valid & + (OBD_MD_LAYOUT_VERSION | OBD_MD_FLDATAVERSION))) + slice->cis_io->ci_result = -ENOTSUPP; + + if (oio->oi_oa.o_valid & OBD_MD_LAYOUT_VERSION) + dv->dv_layout_version = oio->oi_oa.o_layout_version; + if (oio->oi_oa.o_valid & OBD_MD_FLDATAVERSION) + dv->dv_data_version = oio->oi_oa.o_data_version; } } diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index 3b64f734..fe60e67 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -1132,12 +1132,15 @@ enum changelog_message_type { /********* Misc **********/ struct ioc_data_version { - __u64 idv_version; - __u64 idv_flags; /* See LL_DV_xxx */ + __u64 idv_version; + __u32 idv_layout_version; /* FLR: layout version for OST objects */ + __u32 idv_flags; /* enum ioc_data_version_flags */ }; -#define LL_DV_RD_FLUSH (1 << 0) /* Flush dirty pages from clients */ -#define LL_DV_WR_FLUSH (1 << 1) /* Flush all caching pages from clients */ +enum ioc_data_version_flags { + LL_DV_RD_FLUSH = (1 << 0), /* Flush dirty pages from clients */ + LL_DV_WR_FLUSH = (1 << 1), /* Flush all caching pages from clients */ +}; #ifndef offsetof # define offsetof(typ, memb) ((unsigned long)((char *)&(((typ *)0)->memb))) From patchwork Mon Sep 30 18:55:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167181 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DB94A16C1 for ; Mon, 30 Sep 2019 19:01:54 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C3D72224EF for ; Mon, 30 Sep 2019 19:01:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C3D72224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 898745E428E; Mon, 30 Sep 2019 11:59:11 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 4C6225C3BC0 for ; Mon, 30 Sep 2019 11:57:21 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 13BDA10058E6; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 11254BD; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:31 -0400 Message-Id: <1569869810-23848-73-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 072/151] lustre: lov: skip empty pages in lov_io_submit() X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Jinshan Xiong empty page doesn't have corresponding bottem half so it should be completed in the LOV layer. WC-bug-id: https://jira.whamcloud.com/browse/LU-9771 Lustre-commit: ec7eb7a932b7 ("LU-9771 lov: skip empty pages in lov_io_submit()") Signed-off-by: Jinshan Xiong Reviewed-on: https://review.whamcloud.com/29094 Reviewed-by: Bobi Jam Reviewed-by: Dmitry Eremin Signed-off-by: James Simmons --- fs/lustre/lov/lov_cl_internal.h | 1 + fs/lustre/lov/lov_io.c | 26 +++++++++++++------------- fs/lustre/lov/lov_page.c | 8 ++++++++ 3 files changed, 22 insertions(+), 13 deletions(-) diff --git a/fs/lustre/lov/lov_cl_internal.h b/fs/lustre/lov/lov_cl_internal.h index ad4a3d3..095bca0 100644 --- a/fs/lustre/lov/lov_cl_internal.h +++ b/fs/lustre/lov/lov_cl_internal.h @@ -651,6 +651,7 @@ struct lu_object *lovsub_object_alloc(const struct lu_env *env, struct lu_device *dev); struct lov_stripe_md *lov_lsm_addref(struct lov_object *lov); +bool lov_page_is_empty(const struct cl_page *page); int lov_lsm_entry(const struct lov_stripe_md *lsm, u64 offset); int lov_io_layout_at(struct lov_io *lio, u64 offset); diff --git a/fs/lustre/lov/lov_io.c b/fs/lustre/lov/lov_io.c index 457fa3c..8e4ff6c 100644 --- a/fs/lustre/lov/lov_io.c +++ b/fs/lustre/lov/lov_io.c @@ -990,24 +990,20 @@ static int lov_io_submit(const struct lu_env *env, int index; int rc = 0; - if (lio->lis_nr_subios == 1) { - int idx = lio->lis_single_subio_index; - - sub = lov_sub_get(env, lio, idx); - LASSERT(!IS_ERR(sub)); - LASSERT(sub == &lio->lis_single_subio); - rc = cl_io_submit_rw(sub->sub_env, &sub->sub_io, - crt, queue); - return rc; - } - cl_page_list_init(plist); while (qin->pl_nr > 0) { struct cl_2queue *cl2q = &lov_env_info(env)->lti_cl2q; - cl_2queue_init(cl2q); - page = cl_page_list_first(qin); + if (lov_page_is_empty(page)) { + cl_page_list_move(&queue->c2_qout, qin, page); + + cl_page_prep(env, ios->cis_io, page, crt); + cl_page_completion(env, page, crt, 0); + continue; + } + + cl_2queue_init(cl2q); cl_page_list_move(&cl2q->c2_qin, qin, page); index = lov_page_index(page); @@ -1055,6 +1051,8 @@ static int lov_io_commit_async(const struct lu_env *env, if (lio->lis_nr_subios == 1) { int idx = lio->lis_single_subio_index; + LASSERT(!lov_page_is_empty(cl_page_list_first(queue))); + sub = lov_sub_get(env, lio, idx); LASSERT(!IS_ERR(sub)); LASSERT(sub == &lio->lis_single_subio); @@ -1070,6 +1068,8 @@ static int lov_io_commit_async(const struct lu_env *env, LASSERT(plist->pl_nr == 0); page = cl_page_list_first(queue); + LASSERT(!lov_page_is_empty(page)); + cl_page_list_move(plist, queue, page); index = lov_page_index(page); diff --git a/fs/lustre/lov/lov_page.c b/fs/lustre/lov/lov_page.c index 327bcd3..22548f4 100644 --- a/fs/lustre/lov/lov_page.c +++ b/fs/lustre/lov/lov_page.c @@ -136,6 +136,7 @@ int lov_page_init_empty(const struct lu_env *env, struct cl_object *obj, struct lov_page *lpg = cl_object_page_slice(obj, page); void *addr; + lpg->lps_index = ~0; cl_page_slice_add(page, &lpg->lps_cl, obj, index, &lov_empty_page_ops); addr = kmap(page->cp_vmpage); memset(addr, 0, cl_page_size(obj)); @@ -144,4 +145,11 @@ int lov_page_init_empty(const struct lu_env *env, struct cl_object *obj, return 0; } +bool lov_page_is_empty(const struct cl_page *page) +{ + const struct cl_page_slice *slice = cl_page_at(page, &lov_device_type); + + LASSERT(slice); + return slice->cpl_ops == &lov_empty_page_ops; +} /** @} lov */ From patchwork Mon Sep 30 18:55:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167207 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9195516B1 for ; Mon, 30 Sep 2019 19:03:31 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 799B2224EF for ; Mon, 30 Sep 2019 19:03:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 799B2224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id A4F1421F63A; Mon, 30 Sep 2019 11:59:42 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id A19F85C3BD3 for ; Mon, 30 Sep 2019 11:57:21 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 158A510058E7; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 145EFB5; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:32 -0400 Message-Id: <1569869810-23848-74-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 073/151] lustre: mdc: don't assert on name pack X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lai Siyao , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Lai Siyao For old version of MDTs which don't support ATTRFID, open will pack dentry name in the request, but this name may change if there is concurrent rename, as a result an assertion will be triggerred. It's inconvenient to copy name in advance or take dentry lock in the call chain, so It's better to remove this assert, which is safe. WC-bug-id: https://jira.whamcloud.com/browse/LU-5024 Lustre-commit: dd9d7cc845df ("LU-5024 mdc: don't assert on name pack") Signed-off-by: Lai Siyao Reviewed-on: https://review.whamcloud.com/29161 Reviewed-by: Andreas Dilger Reviewed-by: John L. Hammond Signed-off-by: James Simmons --- fs/lustre/mdc/mdc_lib.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/fs/lustre/mdc/mdc_lib.c b/fs/lustre/mdc/mdc_lib.c index 3601e17..d0ae6f2 100644 --- a/fs/lustre/mdc/mdc_lib.c +++ b/fs/lustre/mdc/mdc_lib.c @@ -107,7 +107,11 @@ static void mdc_pack_name(struct ptlrpc_request *req, cpy_len = strlcpy(buf, name, buf_size); - LASSERT(cpy_len == name_len && lu_name_is_valid_2(buf, cpy_len)); + LASSERT(lu_name_is_valid_2(buf, cpy_len)); + if (cpy_len != name_len) + CDEBUG(D_DENTRY, "%s: %s len %zd != %zd, concurrent rename?\n", + req->rq_export->exp_obd->obd_name, buf, name_len, + cpy_len); } void mdc_file_secctx_pack(struct ptlrpc_request *req, const char *secctx_name, From patchwork Mon Sep 30 18:55:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167211 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5648F16C1 for ; Mon, 30 Sep 2019 19:03:43 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3E59A224EF for ; Mon, 30 Sep 2019 19:03:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3E59A224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 793A521F7A2; Mon, 30 Sep 2019 11:59:46 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id E2D3C5C3BD3 for ; Mon, 30 Sep 2019 11:57:21 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 1949F10058E8; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 173DFBB; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:33 -0400 Message-Id: <1569869810-23848-75-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 074/151] lustre: flr: mirror read and write X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Jinshan Xiong Support to perform I/O to designated mirror. Create a new LL_IOC_FLR_SET_MIRROR kernel side so user land applications can set the mirror id. WC-bug-id: https://jira.whamcloud.com/browse/LU-9771 Lustre-commit: 5d7c4fa61ce7 ("LU-9771 flr: mirror read and write") Signed-off-by: Jinshan Xiong Reviewed-on: https://review.whamcloud.com/29095 Reviewed-by: Andreas Dilger Reviewed-by: Bobi Jam Reviewed-by: Dmitry Eremin Signed-off-by: James Simmons --- fs/lustre/include/cl_object.h | 4 ++++ fs/lustre/llite/file.c | 34 +++++++++++++++++++++++++++++++++ fs/lustre/llite/lcommon_cl.c | 3 +++ fs/lustre/llite/llite_internal.h | 5 +++++ fs/lustre/lov/lov_io.c | 27 +++++++++++++++++++++++++- include/uapi/linux/lustre/lustre_user.h | 2 +- 6 files changed, 73 insertions(+), 2 deletions(-) diff --git a/fs/lustre/include/cl_object.h b/fs/lustre/include/cl_object.h index d164294..99534f5 100644 --- a/fs/lustre/include/cl_object.h +++ b/fs/lustre/include/cl_object.h @@ -1891,6 +1891,10 @@ struct cl_io { */ unsigned int ci_ndelay_tried; /** + * Designated mirror index for this I/O. + */ + unsigned int ci_designated_mirror; + /** * Number of pages owned by this IO. For invariant checking. */ unsigned int ci_owned_nr; diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 05a9fea..335f9bf 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -1104,6 +1104,28 @@ int ll_merge_attr(const struct lu_env *env, struct inode *inode) return rc; } +/** + * Set designated mirror for I/O. + * + * So far only read, write, and truncated can support to issue I/O to + * designated mirror. + */ +void ll_io_set_mirror(struct cl_io *io, const struct file *file) +{ + struct ll_file_data *fd = LUSTRE_FPRIVATE(file); + + /* FLR: disable non-delay for designated mirror I/O because obviously + * only one mirror is available + */ + if (fd->fd_designated_mirror > 0) { + io->ci_ndelay = 0; + io->ci_designated_mirror = fd->fd_designated_mirror; + } + + CDEBUG(D_VFSTRACE, "%s: desiginated mirror: %d\n", + file->f_path.dentry->d_name.name, io->ci_designated_mirror); +} + static bool file_is_noatime(const struct file *file) { const struct vfsmount *mnt = file->f_path.mnt; @@ -1160,6 +1182,8 @@ static void ll_io_init(struct cl_io *io, const struct file *file, int write) * available mirror for write. */ io->ci_ndelay = !write; + + ll_io_set_mirror(io, file); } static ssize_t @@ -2976,6 +3000,16 @@ int ll_ioctl_fssetxattr(struct inode *inode, unsigned int cmd, kfree(k_ladvise_hdr); return rc; } + case LL_IOC_FLR_SET_MIRROR: { + /* mirror I/O must be direct to avoid polluting page cache + * by stale data. + */ + if (!(file->f_flags & O_DIRECT)) + return -EINVAL; + + fd->fd_designated_mirror = (u32)arg; + return 0; + } case FS_IOC_FSGETXATTR: return ll_ioctl_fsgetxattr(inode, cmd, arg); case FS_IOC_FSSETXATTR: diff --git a/fs/lustre/llite/lcommon_cl.c b/fs/lustre/llite/lcommon_cl.c index e859351..156b1ad 100644 --- a/fs/lustre/llite/lcommon_cl.c +++ b/fs/lustre/llite/lcommon_cl.c @@ -103,6 +103,9 @@ int cl_setattr_ost(struct cl_object *obj, const struct iattr *attr, io->u.ci_setattr.sa_xvalid = xvalid; io->u.ci_setattr.sa_parent_fid = lu_object_fid(&obj->co_lu); + if (attr->ia_valid & ATTR_FILE) + ll_io_set_mirror(io, attr->ia_file); + again: if (cl_io_init(env, io, CIT_SETATTR, io->ci_obj) == 0) { struct vvp_io *vio = vvp_env_io(env); diff --git a/fs/lustre/llite/llite_internal.h b/fs/lustre/llite/llite_internal.h index 392d82c..2d32c5e 100644 --- a/fs/lustre/llite/llite_internal.h +++ b/fs/lustre/llite/llite_internal.h @@ -653,6 +653,10 @@ struct ll_file_data { */ bool fd_write_failed; bool ll_lock_no_expand; + /* Used by mirrored file to lead IOs to a specific mirror, usually + * for mirror resync. 0 means default. + */ + u32 fd_designated_mirror; rwlock_t fd_lock; /* protect lcc list */ struct list_head fd_lccs; /* list of ll_cl_context */ }; @@ -840,6 +844,7 @@ int ll_dir_getstripe(struct inode *inode, void **lmmp, int *lmm_size, int ll_data_version(struct inode *inode, u64 *data_version, int flags); int ll_hsm_release(struct inode *inode); int ll_hsm_state_set(struct inode *inode, struct hsm_state_set *hss); +void ll_io_set_mirror(struct cl_io *io, const struct file *file); /* llite/dcache.c */ diff --git a/fs/lustre/lov/lov_io.c b/fs/lustre/lov/lov_io.c index 8e4ff6c..628057d 100644 --- a/fs/lustre/lov/lov_io.c +++ b/fs/lustre/lov/lov_io.c @@ -299,6 +299,27 @@ static int lov_io_mirror_init(struct lov_io *lio, struct lov_object *obj, return 0; } + /* find the corresponding mirror for designated mirror IO */ + if (io->ci_designated_mirror > 0) { + struct lov_mirror_entry *entry; + + LASSERT(!io->ci_ndelay); + + index = 0; + lio->lis_mirror_index = -1; + lov_foreach_mirror_entry(obj, entry) { + if (entry->lre_mirror_id == + io->ci_designated_mirror) { + lio->lis_mirror_index = index; + break; + } + + index++; + } + + return (lio->lis_mirror_index < 0) ? -EINVAL : 0; + } + result = lov_io_mirror_write_intent(lio, obj, io); if (result) return result; @@ -998,7 +1019,11 @@ static int lov_io_submit(const struct lu_env *env, if (lov_page_is_empty(page)) { cl_page_list_move(&queue->c2_qout, qin, page); - cl_page_prep(env, ios->cis_io, page, crt); + /* it could only be mirror read to get here therefore + * the pages will be transient. We don't care about + * the return code of cl_page_prep() at all. + */ + (void) cl_page_prep(env, ios->cis_io, page, crt); cl_page_completion(env, page, crt, 0); continue; } diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index fe60e67..f5cd979 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -276,7 +276,7 @@ struct ll_futimes_3 { #define LL_IOC_GET_CONNECT_FLAGS _IOWR('f', 174, __u64 *) #define LL_IOC_GET_MDTIDX _IOR('f', 175, int) #define LL_IOC_FUTIMES_3 _IOWR('f', 176, struct ll_futimes_3) - +#define LL_IOC_FLR_SET_MIRROR _IOW('f', 177, long) /* lustre_ioctl.h 177-210 */ #define LL_IOC_HSM_STATE_GET _IOR('f', 211, struct hsm_user_state) #define LL_IOC_HSM_STATE_SET _IOW('f', 212, struct hsm_state_set) From patchwork Mon Sep 30 18:55:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167215 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AD57416C1 for ; Mon, 30 Sep 2019 19:03:53 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 939F9224EF for ; Mon, 30 Sep 2019 19:03:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 939F9224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id C285E5E45C5; Mon, 30 Sep 2019 11:59:49 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 43C0C5C3B15 for ; Mon, 30 Sep 2019 11:57:22 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 1C76110058E9; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 1A5EBA9; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:34 -0400 Message-Id: <1569869810-23848-76-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 075/151] lustre: flr: resync support and test tool X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Jinshan Xiong A tool to resync mirrored file after writing. It extends the Lustre lease API to support taking file lease and then sending the MDS_REINT_RESYNC RPC to the MDT so that it can increase the file's layout version; then the client will start copying the contents from valid mirror to stale mirrors. At the end of resync, the copying client will release the lease and revalidate stale mirrors. WC-bug-id: https://jira.whamcloud.com/browse/LU-9771 Lustre-commit: 5999c0b881e8 ("LU-9771 flr: resync support and test tool") Signed-off-by: Jinshan Xiong Reviewed-on: https://review.whamcloud.com/29096 Reviewed-by: Bobi Jam Reviewed-by: Dmitry Eremin Reviewed-by: Andreas Dilger Signed-off-by: James Simmons --- fs/lustre/include/lprocfs_status.h | 1 + fs/lustre/include/lustre_req_layout.h | 1 + fs/lustre/include/lustre_swab.h | 1 + fs/lustre/include/obd.h | 2 + fs/lustre/include/obd_class.h | 15 ++ fs/lustre/llite/file.c | 289 ++++++++++++++++++++++++-------- fs/lustre/llite/llite_internal.h | 8 +- fs/lustre/llite/rw26.c | 10 ++ fs/lustre/lmv/lmv_obd.c | 21 +++ fs/lustre/lov/lov_io.c | 70 +++++++- fs/lustre/mdc/mdc_internal.h | 1 + fs/lustre/mdc/mdc_lib.c | 16 ++ fs/lustre/mdc/mdc_reint.c | 52 ++++++ fs/lustre/mdc/mdc_request.c | 38 +++-- fs/lustre/osc/osc_io.c | 3 + fs/lustre/ptlrpc/layout.c | 16 +- fs/lustre/ptlrpc/lproc_ptlrpc.c | 1 + fs/lustre/ptlrpc/pack_generic.c | 13 ++ fs/lustre/ptlrpc/wiretest.c | 94 ++++++++++- include/uapi/linux/lustre/lustre_idl.h | 47 +++++- include/uapi/linux/lustre/lustre_user.h | 35 +++- 21 files changed, 630 insertions(+), 104 deletions(-) diff --git a/fs/lustre/include/lprocfs_status.h b/fs/lustre/include/lprocfs_status.h index e923673..54e4eda 100644 --- a/fs/lustre/include/lprocfs_status.h +++ b/fs/lustre/include/lprocfs_status.h @@ -334,6 +334,7 @@ enum { MDS_REINT_RENAME, MDS_REINT_OPEN, MDS_REINT_SETXATTR, + MDS_REINT_RESYNC, BRW_READ_BYTES, BRW_WRITE_BYTES, EXTRA_LAST_OPC diff --git a/fs/lustre/include/lustre_req_layout.h b/fs/lustre/include/lustre_req_layout.h index c255648..3d86883 100644 --- a/fs/lustre/include/lustre_req_layout.h +++ b/fs/lustre/include/lustre_req_layout.h @@ -163,6 +163,7 @@ void req_capsule_shrink(struct req_capsule *pill, extern struct req_format RQF_MDS_QUOTACTL; extern struct req_format RQF_MDS_SWAP_LAYOUTS; extern struct req_format RQF_MDS_REINT_MIGRATE; +extern struct req_format RQF_MDS_REINT_RESYNC; /* MDS hsm formats */ extern struct req_format RQF_MDS_HSM_STATE_GET; extern struct req_format RQF_MDS_HSM_STATE_SET; diff --git a/fs/lustre/include/lustre_swab.h b/fs/lustre/include/lustre_swab.h index 1758dd9..79cacf4 100644 --- a/fs/lustre/include/lustre_swab.h +++ b/fs/lustre/include/lustre_swab.h @@ -99,6 +99,7 @@ void lustre_swab_lov_user_md_objects(struct lov_user_ost_data *lod, void lustre_swab_hsm_request(struct hsm_request *hr); void lustre_swab_swap_layouts(struct mdc_swap_layouts *msl); void lustre_swab_close_data(struct close_data *data); +void lustre_swab_close_data_resync_done(struct close_data_resync_done *resync); void lustre_swab_lmv_user_md(struct lmv_user_md *lum); void lustre_swab_ladvise(struct lu_ladvise *ladvise); void lustre_swab_ladvise_hdr(struct ladvise_hdr *ladvise_hdr); diff --git a/fs/lustre/include/obd.h b/fs/lustre/include/obd.h index c377a91..e377526 100644 --- a/fs/lustre/include/obd.h +++ b/fs/lustre/include/obd.h @@ -931,6 +931,8 @@ struct obd_client_handle { struct cl_attr; struct md_ops { + int (*file_resync)(struct obd_export *exp, struct md_op_data *data); + int (*get_root)(struct obd_export *exp, const char *fileset, struct lu_fid *fid); int (*null_inode)(struct obd_export *, const struct lu_fid *); diff --git a/fs/lustre/include/obd_class.h b/fs/lustre/include/obd_class.h index f26ca17..a939f17 100644 --- a/fs/lustre/include/obd_class.h +++ b/fs/lustre/include/obd_class.h @@ -1349,6 +1349,21 @@ static inline int md_fsync(struct obd_export *exp, const struct lu_fid *fid, return MDP(exp->exp_obd, fsync)(exp, fid, request); } +/* FLR: resync mirrored files. */ +static inline int md_file_resync(struct obd_export *exp, + struct md_op_data *data) +{ + int rc; + + rc = exp_check_ops(exp); + if (rc) + return rc; + + rc = MDP(exp->exp_obd, file_resync)(exp, data); + + return rc; +} + static inline int md_read_page(struct obd_export *exp, struct md_op_data *op_data, struct md_callback *cb_op, diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 335f9bf..d13c583 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -161,6 +161,23 @@ static int ll_close_inode_openhandle(struct inode *inode, op_data->op_fid2 = *ll_inode2fid(data); break; + case MDS_CLOSE_RESYNC_DONE: { + struct ll_ioc_lease *ioc = data; + + LASSERT(data); + op_data->op_attr_blocks += + ioc->lil_count * op_data->op_attr_blocks; + op_data->op_attr.ia_valid |= ATTR_SIZE; + op_data->op_xvalid |= OP_XVALID_BLOCKS; + op_data->op_bias |= MDS_CLOSE_RESYNC_DONE; + + op_data->op_lease_handle = och->och_lease_handle; + op_data->op_data = &ioc->lil_ids[0]; + op_data->op_data_size = + ioc->lil_count * sizeof(ioc->lil_ids[0]); + break; + } + case MDS_HSM_RELEASE: LASSERT(data); op_data->op_bias |= MDS_HSM_RELEASE; @@ -1007,8 +1024,10 @@ static int ll_swap_layouts_close(struct obd_client_handle *och, * Release lease and close the file. * It will check if the lease has ever broken. */ -static int ll_lease_close(struct obd_client_handle *och, struct inode *inode, - bool *lease_broken) +static int ll_lease_close_intent(struct obd_client_handle *och, + struct inode *inode, + bool *lease_broken, enum mds_op_bias bias, + void *data) { struct ldlm_lock *lock; bool cancelled = true; @@ -1021,15 +1040,60 @@ static int ll_lease_close(struct obd_client_handle *och, struct inode *inode, LDLM_LOCK_PUT(lock); } - CDEBUG(D_INODE, "lease for " DFID " broken? %d\n", - PFID(&ll_i2info(inode)->lli_fid), cancelled); + CDEBUG(D_INODE, "lease for " DFID " broken? %d, bias: %x\n", + PFID(&ll_i2info(inode)->lli_fid), cancelled, bias); - if (!cancelled) - ldlm_cli_cancel(&och->och_lease_handle, 0); if (lease_broken) *lease_broken = cancelled; - return ll_close_inode_openhandle(inode, och, 0, NULL); + if (!cancelled && !bias) + ldlm_cli_cancel(&och->och_lease_handle, 0); + if (cancelled) { /* no need to excute intent */ + bias = 0; + data = NULL; + } + + return ll_close_inode_openhandle(inode, och, bias, data); +} + +static int ll_lease_close(struct obd_client_handle *och, struct inode *inode, + bool *lease_broken) +{ + return ll_lease_close_intent(och, inode, lease_broken, 0, NULL); +} + +/** + * After lease is taken, send the RPC MDS_REINT_RESYNC to the MDT + */ +static int ll_lease_file_resync(struct obd_client_handle *och, + struct inode *inode) +{ + struct ll_sb_info *sbi = ll_i2sbi(inode); + struct md_op_data *op_data; + u64 data_version_unused; + int rc; + + op_data = ll_prep_md_op_data(NULL, inode, NULL, NULL, 0, 0, + LUSTRE_OPC_ANY, NULL); + if (IS_ERR(op_data)) + return PTR_ERR(op_data); + + /* before starting file resync, it's necessary to clean up page cache + * in client memory, otherwise once the layout version is increased, + * writing back cached data will be denied the OSTs. + */ + rc = ll_data_version(inode, &data_version_unused, LL_DV_WR_FLUSH); + if (rc) + goto out; + + op_data->op_handle = och->och_lease_handle; + rc = md_file_resync(sbi->ll_md_exp, op_data); + if (rc) + goto out; + +out: + ll_finish_md_op_data(op_data); + return rc; } int ll_merge_attr(const struct lu_env *env, struct inode *inode) @@ -1114,12 +1178,18 @@ void ll_io_set_mirror(struct cl_io *io, const struct file *file) { struct ll_file_data *fd = LUSTRE_FPRIVATE(file); + /* clear layout version for generic(non-resync) I/O in case it carries + * stale layout version due to I/O restart + */ + io->ci_layout_version = 0; + /* FLR: disable non-delay for designated mirror I/O because obviously * only one mirror is available */ if (fd->fd_designated_mirror > 0) { io->ci_ndelay = 0; io->ci_designated_mirror = fd->fd_designated_mirror; + io->ci_layout_version = fd->fd_layout_version; } CDEBUG(D_VFSTRACE, "%s: desiginated mirror: %d\n", @@ -2577,6 +2647,140 @@ int ll_ioctl_fssetxattr(struct inode *inode, unsigned int cmd, kfree(attr); out_fsxattr: ll_finish_md_op_data(op_data); + + return rc; +} + +static long ll_file_unlock_lease(struct file *file, struct ll_ioc_lease *ioc, + unsigned long arg) +{ + struct inode *inode = file_inode(file); + struct ll_file_data *fd = LUSTRE_FPRIVATE(file); + struct ll_inode_info *lli = ll_i2info(inode); + struct obd_client_handle *och = NULL; + bool lease_broken; + fmode_t fmode = 0; + enum mds_op_bias bias = 0; + void *data = NULL; + size_t data_size = 0; + long rc; + + mutex_lock(&lli->lli_och_mutex); + if (fd->fd_lease_och) { + och = fd->fd_lease_och; + fd->fd_lease_och = NULL; + } + mutex_unlock(&lli->lli_och_mutex); + + if (!och) { + rc = -ENOLCK; + goto out; + } + + fmode = och->och_flags; + + if (ioc->lil_flags & LL_LEASE_RESYNC_DONE) { + if (ioc->lil_count > IOC_IDS_MAX) { + rc = -EINVAL; + goto out; + } + + data_size = offsetof(typeof(*ioc), lil_ids[ioc->lil_count]); + data = kzalloc(data_size, GFP_KERNEL); + if (!data) { + rc = -ENOMEM; + goto out; + } + + if (copy_from_user(data, (void __user *)arg, data_size)) { + rc = -EFAULT; + goto out; + } + + bias = MDS_CLOSE_RESYNC_DONE; + } + + rc = ll_lease_close_intent(och, inode, &lease_broken, bias, data); + if (rc < 0) + goto out; + + rc = ll_lease_och_release(inode, file); + if (rc < 0) + goto out; + + if (lease_broken) + fmode = 0; + +out: + kfree(data); + if (!rc) + rc = ll_lease_type_from_fmode(fmode); + return rc; +} + +static long ll_file_set_lease(struct file *file, struct ll_ioc_lease *ioc, + unsigned long arg) +{ + struct inode *inode = file_inode(file); + struct ll_inode_info *lli = ll_i2info(inode); + struct ll_file_data *fd = LUSTRE_FPRIVATE(file); + struct obd_client_handle *och = NULL; + u64 open_flags = 0; + bool lease_broken; + fmode_t fmode; + long rc; + + switch (ioc->lil_mode) { + case LL_LEASE_WRLCK: + if (!(file->f_mode & FMODE_WRITE)) + return -EPERM; + fmode = FMODE_WRITE; + break; + case LL_LEASE_RDLCK: + if (!(file->f_mode & FMODE_READ)) + return -EPERM; + fmode = FMODE_READ; + break; + case LL_LEASE_UNLCK: + return ll_file_unlock_lease(file, ioc, arg); + default: + return -EINVAL; + } + + CDEBUG(D_INODE, "Set lease with mode %u\n", fmode); + + /* apply for lease */ + if (ioc->lil_flags & LL_LEASE_RESYNC) + open_flags = MDS_OPEN_RESYNC; + och = ll_lease_open(inode, file, fmode, open_flags); + if (IS_ERR(och)) + return PTR_ERR(och); + + if (ioc->lil_flags & LL_LEASE_RESYNC) { + rc = ll_lease_file_resync(och, inode); + if (rc) { + ll_lease_close(och, inode, NULL); + return rc; + } + rc = ll_layout_refresh(inode, &fd->fd_layout_version); + if (rc) { + ll_lease_close(och, inode, NULL); + return rc; + } + } + + rc = 0; + mutex_lock(&lli->lli_och_mutex); + if (!fd->fd_lease_och) { + fd->fd_lease_och = och; + och = NULL; + } + mutex_unlock(&lli->lli_och_mutex); + if (och) { + /* impossible now that only excl is supported for now */ + ll_lease_close(och, inode, &lease_broken); + rc = -EBUSY; + } return rc; } @@ -2805,71 +3009,18 @@ int ll_ioctl_fssetxattr(struct inode *inode, unsigned int cmd, kfree(hca); return rc; } - case LL_IOC_SET_LEASE: { - struct ll_inode_info *lli = ll_i2info(inode); - struct obd_client_handle *och = NULL; - bool lease_broken; - fmode_t fmode; - - switch (arg) { - case LL_LEASE_WRLCK: - if (!(file->f_mode & FMODE_WRITE)) - return -EPERM; - fmode = FMODE_WRITE; - break; - case LL_LEASE_RDLCK: - if (!(file->f_mode & FMODE_READ)) - return -EPERM; - fmode = FMODE_READ; - break; - case LL_LEASE_UNLCK: - mutex_lock(&lli->lli_och_mutex); - if (fd->fd_lease_och) { - och = fd->fd_lease_och; - fd->fd_lease_och = NULL; - } - mutex_unlock(&lli->lli_och_mutex); - - if (!och) - return -ENOLCK; + case LL_IOC_SET_LEASE_OLD: { + struct ll_ioc_lease ioc = { .lil_mode = (u32)arg }; - fmode = och->och_flags; - rc = ll_lease_close(och, inode, &lease_broken); - if (rc < 0) - return rc; - - rc = ll_lease_och_release(inode, file); - if (rc < 0) - return rc; - - if (lease_broken) - fmode = 0; - - return ll_lease_type_from_fmode(fmode); - default: - return -EINVAL; - } - - CDEBUG(D_INODE, "Set lease with mode %u\n", fmode); + return ll_file_set_lease(file, &ioc, 0); + } + case LL_IOC_SET_LEASE: { + struct ll_ioc_lease ioc; - /* apply for lease */ - och = ll_lease_open(inode, file, fmode, 0); - if (IS_ERR(och)) - return PTR_ERR(och); + if (copy_from_user(&ioc, (void __user *)arg, sizeof(ioc))) + return -EFAULT; - rc = 0; - mutex_lock(&lli->lli_och_mutex); - if (!fd->fd_lease_och) { - fd->fd_lease_och = och; - och = NULL; - } - mutex_unlock(&lli->lli_och_mutex); - if (och) { - /* impossible now that only excl is supported for now */ - ll_lease_close(och, inode, &lease_broken); - rc = -EBUSY; - } - return rc; + return ll_file_set_lease(file, &ioc, arg); } case LL_IOC_GET_LEASE: { struct ll_inode_info *lli = ll_i2info(inode); diff --git a/fs/lustre/llite/llite_internal.h b/fs/lustre/llite/llite_internal.h index 2d32c5e..dc81474 100644 --- a/fs/lustre/llite/llite_internal.h +++ b/fs/lustre/llite/llite_internal.h @@ -653,12 +653,16 @@ struct ll_file_data { */ bool fd_write_failed; bool ll_lock_no_expand; + rwlock_t fd_lock; /* protect lcc list */ + struct list_head fd_lccs; /* list of ll_cl_context */ /* Used by mirrored file to lead IOs to a specific mirror, usually * for mirror resync. 0 means default. */ u32 fd_designated_mirror; - rwlock_t fd_lock; /* protect lcc list */ - struct list_head fd_lccs; /* list of ll_cl_context */ + /* The layout version when resync starts. Resync I/O should carry this + * layout version for verification to OST objects + */ + u32 fd_layout_version; }; void llite_tunables_unregister(void); diff --git a/fs/lustre/llite/rw26.c b/fs/lustre/llite/rw26.c index 805ba32..35d39fe 100644 --- a/fs/lustre/llite/rw26.c +++ b/fs/lustre/llite/rw26.c @@ -443,6 +443,16 @@ static int ll_write_begin(struct file *file, struct address_space *mapping, env = lcc->lcc_env; io = lcc->lcc_io; + if (file->f_flags & O_DIRECT && io->ci_designated_mirror > 0) { + /* direct IO failed because it couldn't clean up cached pages, + * this causes a problem for mirror write because the cached + * page may belong to another mirror, which will result in + * problem submitting the I/O. + */ + result = -EBUSY; + goto out; + } + /* To avoid deadlock, try to lock page first. */ vmpage = grab_cache_page_nowait(mapping, index); if (unlikely(!vmpage || PageDirty(vmpage) || PageWriteback(vmpage))) { diff --git a/fs/lustre/lmv/lmv_obd.c b/fs/lustre/lmv/lmv_obd.c index 73ab7b6..47fc22c 100644 --- a/fs/lustre/lmv/lmv_obd.c +++ b/fs/lustre/lmv/lmv_obd.c @@ -2139,6 +2139,26 @@ static struct lu_dirent *stripe_dirent_next(struct lmv_dir_ctxt *ctxt, return ent; } +static int lmv_file_resync(struct obd_export *exp, struct md_op_data *data) +{ + struct obd_device *obd = exp->exp_obd; + struct lmv_obd *lmv = &obd->u.lmv; + struct lmv_tgt_desc *tgt; + int rc; + + rc = lmv_check_connect(obd); + if (rc != 0) + return rc; + + tgt = lmv_find_target(lmv, &data->op_fid1); + if (IS_ERR(tgt)) + return PTR_ERR(tgt); + + data->op_flags |= MF_MDC_CANCEL_FID1; + rc = md_file_resync(tgt->ltd_exp, data); + return rc; +} + /** * Get dirent with the closest hash for striped directory * @@ -3120,6 +3140,7 @@ static int lmv_merge_attr(struct obd_export *exp, .setattr = lmv_setattr, .setxattr = lmv_setxattr, .fsync = lmv_fsync, + .file_resync = lmv_file_resync, .read_page = lmv_read_page, .unlink = lmv_unlink, .init_ea_size = lmv_init_ea_size, diff --git a/fs/lustre/lov/lov_io.c b/fs/lustre/lov/lov_io.c index 628057d..9fd1b52 100644 --- a/fs/lustre/lov/lov_io.c +++ b/fs/lustre/lov/lov_io.c @@ -229,8 +229,17 @@ static int lov_io_mirror_write_intent(struct lov_io *lio, cl_io_is_mkwrite(io))) return 0; + /* FLR: check if it needs to send a write intent RPC to server. + * Writing to sync_pending file needs write intent RPC to change + * the file state back to write_pending, so that the layout version + * can be increased when the state changes to sync_pending at a later + * time. Otherwise there exists a chance that an evicted client may + * dirty the file data while resync client is working on it. + * Designated I/O is allowed for resync workload. + */ if (lov_flr_state(obj) == LCM_FL_RDONLY || - lov_flr_state(obj) == LCM_FL_SYNC_PENDING) { + (lov_flr_state(obj) == LCM_FL_SYNC_PENDING && + io->ci_designated_mirror == 0)) { io->ci_need_write_intent = 1; return 0; } @@ -299,12 +308,31 @@ static int lov_io_mirror_init(struct lov_io *lio, struct lov_object *obj, return 0; } + /* transfer the layout version for verification */ + if (io->ci_layout_version == 0) + io->ci_layout_version = obj->lo_lsm->lsm_layout_gen; + /* find the corresponding mirror for designated mirror IO */ if (io->ci_designated_mirror > 0) { struct lov_mirror_entry *entry; LASSERT(!io->ci_ndelay); + CDEBUG(D_LAYOUT, "designated I/O mirror state: %d\n", + lov_flr_state(obj)); + + if ((cl_io_is_trunc(io) || io->ci_type == CIT_WRITE) && + (io->ci_layout_version != obj->lo_lsm->lsm_layout_gen)) { + /* For resync I/O, the ci_layout_version was the layout + * version when resync starts. If it doesn't match the + * current object layout version, it means the layout + * has been changed + */ + return -ESTALE; + } + + io->ci_layout_version |= LU_LAYOUT_RESYNC; + index = 0; lio->lis_mirror_index = -1; lov_foreach_mirror_entry(obj, entry) { @@ -317,7 +345,7 @@ static int lov_io_mirror_init(struct lov_io *lio, struct lov_object *obj, index++; } - return (lio->lis_mirror_index < 0) ? -EINVAL : 0; + return lio->lis_mirror_index < 0 ? -EINVAL : 0; } result = lov_io_mirror_write_intent(lio, obj, io); @@ -333,9 +361,6 @@ static int lov_io_mirror_init(struct lov_io *lio, struct lov_object *obj, return 1; } - /* transfer the layout version for verification */ - io->ci_layout_version = obj->lo_lsm->lsm_layout_gen; - if (io->ci_ndelay_tried == 0 || /* first time to try */ /* reset the mirror index if layout has changed */ lio->lis_mirror_layout_gen != obj->lo_lsm->lsm_layout_gen) { @@ -529,11 +554,29 @@ static int lov_io_slice_init(struct lov_io *lio, struct lov_object *obj, if (!lsm_entry_inited(obj->lo_lsm, index)) { io->ci_need_write_intent = 1; io->ci_write_intent = ext; - result = 1; - goto out; + break; } } + if (io->ci_need_write_intent && io->ci_designated_mirror > 0) { + /* REINT_SYNC RPC has already tried to instantiate all of the + * components involved, obviously it didn't succeed. Skip this + * mirror for now. The server won't be able to figure out + * which mirror it should instantiate components + */ + CERROR(DFID": trying to instantiate components for designated I/O, file state: %d\n", + PFID(lu_object_fid(lov2lu(obj))), lov_flr_state(obj)); + + io->ci_need_write_intent = 0; + result = -EIO; + goto out; + } + + if (io->ci_need_write_intent) { + result = 1; + goto out; + } + out: return result; } @@ -661,7 +704,8 @@ static int lov_io_iter_init(const struct lu_env *env, ext.e_end = lio->lis_endpos; lov_foreach_io_layout(index, lio, &ext) { - struct lov_layout_raid0 *r0 = lov_r0(lio->lis_object, index); + struct lov_layout_entry *le = lov_entry(lio->lis_object, index); + struct lov_layout_raid0 *r0 = &le->lle_raid0; int stripe; u64 start; u64 end; @@ -675,6 +719,12 @@ static int lov_io_iter_init(const struct lu_env *env, continue; } + if (!le->lle_valid && !ios->cis_io->ci_designated_mirror) { + CERROR("I/O to invalid component: %d, mirror: %d\n", + index, lio->lis_mirror_index); + return -EIO; + } + for (stripe = 0; stripe < r0->lo_nr; stripe++) { if (!lov_stripe_intersects(lsm, index, stripe, &ext, &start, &end)) @@ -744,6 +794,10 @@ static int lov_io_rw_iter_init(const struct lu_env *env, return -ENODATA; } + if (!lov_entry(lio->lis_object, index)->lle_valid && + !io->ci_designated_mirror) + return io->ci_type == CIT_READ ? -EAGAIN : -EIO; + lse = lov_lse(lio->lis_object, index); next = MAX_LFS_FILESIZE; diff --git a/fs/lustre/mdc/mdc_internal.h b/fs/lustre/mdc/mdc_internal.h index 6b282b2..669cb1c 100644 --- a/fs/lustre/mdc/mdc_internal.h +++ b/fs/lustre/mdc/mdc_internal.h @@ -116,6 +116,7 @@ int mdc_setattr(struct obd_export *exp, struct md_op_data *op_data, void *ea, size_t ealen, struct ptlrpc_request **request); int mdc_unlink(struct obd_export *exp, struct md_op_data *op_data, struct ptlrpc_request **request); +int mdc_file_resync(struct obd_export *exp, struct md_op_data *data); int mdc_cancel_unused(struct obd_export *exp, const struct lu_fid *fid, union ldlm_policy_data *policy, enum ldlm_mode mode, enum ldlm_cancel_flags flags, void *opaque); diff --git a/fs/lustre/mdc/mdc_lib.c b/fs/lustre/mdc/mdc_lib.c index d0ae6f2..ff14f82 100644 --- a/fs/lustre/mdc/mdc_lib.c +++ b/fs/lustre/mdc/mdc_lib.c @@ -450,6 +450,22 @@ static void mdc_intent_close_pack(struct ptlrpc_request *req, data->cd_data_version = op_data->op_data_version; data->cd_fid = op_data->op_fid2; + + if (bias & MDS_CLOSE_RESYNC_DONE) { + struct close_data_resync_done *sync = &data->cd_resync; + + BUILD_BUG_ON(sizeof(data->cd_resync) > sizeof(data->cd_reserved)); + sync->resync_count = op_data->op_data_size / sizeof(u32); + if (sync->resync_count <= INLINE_RESYNC_ARRAY_SIZE) { + memcpy(sync->resync_ids_inline, op_data->op_data, + op_data->op_data_size); + } else { + size_t count = sync->resync_count; + + memcpy(req_capsule_client_get(&req->rq_pill, &RMF_U32), + op_data->op_data, count * sizeof(u32)); + } + } } void mdc_rename_pack(struct ptlrpc_request *req, struct md_op_data *op_data, diff --git a/fs/lustre/mdc/mdc_reint.c b/fs/lustre/mdc/mdc_reint.c index 87dabaf..8e0bd0a 100644 --- a/fs/lustre/mdc/mdc_reint.c +++ b/fs/lustre/mdc/mdc_reint.c @@ -427,3 +427,55 @@ int mdc_rename(struct obd_export *exp, struct md_op_data *op_data, return rc; } + +int mdc_file_resync(struct obd_export *exp, struct md_op_data *op_data) +{ + struct list_head cancels = LIST_HEAD_INIT(cancels); + struct ptlrpc_request *req; + struct ldlm_lock *lock; + struct mdt_rec_resync *rec; + int count = 0, rc; + + if (op_data->op_flags & MF_MDC_CANCEL_FID1 && + fid_is_sane(&op_data->op_fid1)) + count = mdc_resource_get_unused(exp, &op_data->op_fid1, + &cancels, LCK_EX, + MDS_INODELOCK_LAYOUT); + + req = ptlrpc_request_alloc(class_exp2cliimp(exp), + &RQF_MDS_REINT_RESYNC); + if (!req) { + ldlm_lock_list_put(&cancels, l_bl_ast, count); + return -ENOMEM; + } + + rc = mdc_prep_elc_req(exp, req, MDS_REINT, &cancels, count); + if (rc) { + ptlrpc_request_free(req); + return rc; + } + + BUILD_BUG_ON(sizeof(*rec) != sizeof(struct mdt_rec_reint)); + rec = req_capsule_client_get(&req->rq_pill, &RMF_REC_REINT); + rec->rs_opcode = REINT_RESYNC; + rec->rs_fsuid = op_data->op_fsuid; + rec->rs_fsgid = op_data->op_fsgid; + rec->rs_cap = op_data->op_cap.cap[0]; + rec->rs_fid = op_data->op_fid1; + rec->rs_bias = op_data->op_bias; + + lock = ldlm_handle2lock(&op_data->op_handle); + if (lock) { + rec->rs_handle = lock->l_remote_handle; + LDLM_LOCK_PUT(lock); + } + + ptlrpc_request_set_replen(req); + + rc = mdc_reint(req, LUSTRE_IMP_FULL); + if (rc == -ERESTARTSYS) + rc = 0; + + ptlrpc_req_finished(req); + return rc; +} diff --git a/fs/lustre/mdc/mdc_request.c b/fs/lustre/mdc/mdc_request.c index a1ed9bf..9bae3a5 100644 --- a/fs/lustre/mdc/mdc_request.c +++ b/fs/lustre/mdc/mdc_request.c @@ -762,23 +762,34 @@ static int mdc_close(struct obd_export *exp, struct md_op_data *op_data, struct obd_device *obd = class_exp2obd(exp); struct ptlrpc_request *req; struct req_format *req_fmt; + size_t u32_count = 0; int rc; int saved_rc = 0; - if (op_data->op_bias & MDS_HSM_RELEASE) { + CDEBUG(D_INODE, "%s: " DFID " file closed with intent: %x\n", + exp->exp_obd->obd_name, PFID(&op_data->op_fid1), + op_data->op_bias); + + if (op_data->op_bias & MDS_CLOSE_INTENT) { req_fmt = &RQF_MDS_INTENT_CLOSE; + if (op_data->op_bias & MDS_HSM_RELEASE) { + /* allocate a FID for volatile file */ + rc = mdc_fid_alloc(NULL, exp, &op_data->op_fid2, + op_data); + if (rc < 0) { + CERROR("%s: " DFID " allocating FID: rc = %d\n", + obd->obd_name, PFID(&op_data->op_fid1), + rc); + /* save the errcode and proceed to close */ + saved_rc = rc; + } + } + if (op_data->op_bias & MDS_CLOSE_RESYNC_DONE) { + size_t count = op_data->op_data_size / sizeof(u32); - /* allocate a FID for volatile file */ - rc = mdc_fid_alloc(NULL, exp, &op_data->op_fid2, op_data); - if (rc < 0) { - CERROR("%s: " DFID " failed to allocate FID: %d\n", - obd->obd_name, PFID(&op_data->op_fid1), rc); - /* save the errcode and proceed to close */ - saved_rc = rc; + if (count > INLINE_RESYNC_ARRAY_SIZE) + u32_count = count; } - } else if (op_data->op_bias & (MDS_CLOSE_LAYOUT_SWAP | - MDS_CLOSE_LAYOUT_MERGE)) { - req_fmt = &RQF_MDS_INTENT_CLOSE; } else { req_fmt = &RQF_MDS_CLOSE; } @@ -818,6 +829,10 @@ static int mdc_close(struct obd_export *exp, struct md_op_data *op_data, goto out; } + if (u32_count > 0) + req_capsule_set_size(&req->rq_pill, &RMF_U32, RCL_CLIENT, + u32_count * sizeof(u32)); + rc = ptlrpc_request_pack(req, LUSTRE_MDS_VERSION, MDS_CLOSE); if (rc) { ptlrpc_request_free(req); @@ -2627,6 +2642,7 @@ int mdc_process_config(struct obd_device *obd, u32 len, void *buf) .setxattr = mdc_setxattr, .getxattr = mdc_getxattr, .fsync = mdc_fsync, + .file_resync = mdc_file_resync, .read_page = mdc_read_page, .unlink = mdc_unlink, .cancel_unused = mdc_cancel_unused, diff --git a/fs/lustre/osc/osc_io.c b/fs/lustre/osc/osc_io.c index d75f725..0545e23 100644 --- a/fs/lustre/osc/osc_io.c +++ b/fs/lustre/osc/osc_io.c @@ -296,6 +296,9 @@ int osc_io_commit_async(const struct lu_env *env, opg = osc_cl_page_osc(page, osc); oap = &opg->ops_oap; + LASSERTF(osc == oap->oap_obj, + "obj mismatch: %p / %p\n", osc, oap->oap_obj); + if (!list_empty(&oap->oap_rpc_item)) { CDEBUG(D_CACHE, "Busy oap %p page %p for submit.\n", oap, opg); diff --git a/fs/lustre/ptlrpc/layout.c b/fs/lustre/ptlrpc/layout.c index b6476bc..ce1de5e 100644 --- a/fs/lustre/ptlrpc/layout.c +++ b/fs/lustre/ptlrpc/layout.c @@ -121,7 +121,8 @@ &RMF_MDT_EPOCH, &RMF_REC_REINT, &RMF_CAPA1, - &RMF_CLOSE_DATA + &RMF_CLOSE_DATA, + &RMF_U32 }; static const struct req_msg_field *obd_statfs_server[] = { @@ -298,6 +299,12 @@ &RMF_DLM_REQ }; +static const struct req_msg_field *mds_reint_resync[] = { + &RMF_PTLRPC_BODY, + &RMF_REC_REINT, + &RMF_DLM_REQ +}; + static const struct req_msg_field *mdt_swap_layouts[] = { &RMF_PTLRPC_BODY, &RMF_MDT_BODY, @@ -713,6 +720,7 @@ &RQF_MDS_REINT_MIGRATE, &RQF_MDS_REINT_SETATTR, &RQF_MDS_REINT_SETXATTR, + &RQF_MDS_REINT_RESYNC, &RQF_MDS_QUOTACTL, &RQF_MDS_HSM_PROGRESS, &RQF_MDS_HSM_CT_REGISTER, @@ -842,7 +850,7 @@ struct req_msg_field RMF_MGS_CONFIG_RES = EXPORT_SYMBOL(RMF_MGS_CONFIG_RES); struct req_msg_field RMF_U32 = - DEFINE_MSGF("generic u32", 0, + DEFINE_MSGF("generic u32", RMF_F_STRUCT_ARRAY, sizeof(u32), lustre_swab_generic_32s, NULL); EXPORT_SYMBOL(RMF_U32); @@ -1343,6 +1351,10 @@ struct req_format RQF_MDS_REINT_SETXATTR = mds_reint_setxattr_client, mdt_body_only); EXPORT_SYMBOL(RQF_MDS_REINT_SETXATTR); +struct req_format RQF_MDS_REINT_RESYNC = + DEFINE_REQ_FMT0("MDS_REINT_RESYNC", mds_reint_resync, mdt_body_only); +EXPORT_SYMBOL(RQF_MDS_REINT_RESYNC); + struct req_format RQF_MDS_CONNECT = DEFINE_REQ_FMT0("MDS_CONNECT", obd_connect_client, obd_connect_server); diff --git a/fs/lustre/ptlrpc/lproc_ptlrpc.c b/fs/lustre/ptlrpc/lproc_ptlrpc.c index 937a413..6ce4d9e 100644 --- a/fs/lustre/ptlrpc/lproc_ptlrpc.c +++ b/fs/lustre/ptlrpc/lproc_ptlrpc.c @@ -149,6 +149,7 @@ { MDS_REINT_RENAME, "mds_reint_rename" }, { MDS_REINT_OPEN, "mds_reint_open" }, { MDS_REINT_SETXATTR, "mds_reint_setxattr" }, + { MDS_REINT_RESYNC, "mds_reint_resync" }, { BRW_READ_BYTES, "read_bytes" }, { BRW_WRITE_BYTES, "write_bytes" }, }; diff --git a/fs/lustre/ptlrpc/pack_generic.c b/fs/lustre/ptlrpc/pack_generic.c index 0c73da6..e7662be 100644 --- a/fs/lustre/ptlrpc/pack_generic.c +++ b/fs/lustre/ptlrpc/pack_generic.c @@ -2418,6 +2418,19 @@ void lustre_swab_close_data(struct close_data *cd) __swab64s(&cd->cd_data_version); } +void lustre_swab_close_data_resync_done(struct close_data_resync_done *resync) +{ + int i; + + __swab32s(&resync->resync_count); + /* after swab, resync_count must in CPU endian */ + if (resync->resync_count <= INLINE_RESYNC_ARRAY_SIZE) { + for (i = 0; i < resync->resync_count; i++) + __swab32s(&resync->resync_ids_inline[i]); + } +} +EXPORT_SYMBOL(lustre_swab_close_data_resync_done); + void lustre_swab_ladvise(struct lu_ladvise *ladvise) { swab16s(&ladvise->lla_advice); diff --git a/fs/lustre/ptlrpc/wiretest.c b/fs/lustre/ptlrpc/wiretest.c index 0b3c6af..ff3c79a 100644 --- a/fs/lustre/ptlrpc/wiretest.c +++ b/fs/lustre/ptlrpc/wiretest.c @@ -197,7 +197,7 @@ void lustre_assert_wire_constants(void) (long long)REINT_RMENTRY); LASSERTF(REINT_MIGRATE == 9, "found %lld\n", (long long)REINT_MIGRATE); - LASSERTF(REINT_MAX == 10, "found %lld\n", + LASSERTF(REINT_MAX == 11, "found %lld\n", (long long)REINT_MAX); LASSERTF(DISP_IT_EXECD == 0x00000001UL, "found 0x%.8xUL\n", (unsigned int)DISP_IT_EXECD); @@ -2697,6 +2697,98 @@ void lustre_assert_wire_constants(void) LASSERTF((int)sizeof(((struct mdt_rec_setxattr *)0)->sx_padding_11) == 4, "found %lld\n", (long long)(int)sizeof(((struct mdt_rec_setxattr *)0)->sx_padding_11)); + /* Checks for struct mdt_rec_resync */ + LASSERTF((int)sizeof(struct mdt_rec_resync) == 136, "found %lld\n", + (long long)(int)sizeof(struct mdt_rec_resync)); + LASSERTF((int)offsetof(struct mdt_rec_resync, rs_opcode) == 0, "found %lld\n", + (long long)(int)offsetof(struct mdt_rec_resync, rs_opcode)); + LASSERTF((int)sizeof(((struct mdt_rec_resync *)0)->rs_opcode) == 4, "found %lld\n", + (long long)(int)sizeof(((struct mdt_rec_resync *)0)->rs_opcode)); + LASSERTF((int)offsetof(struct mdt_rec_resync, rs_cap) == 4, "found %lld\n", + (long long)(int)offsetof(struct mdt_rec_resync, rs_cap)); + LASSERTF((int)sizeof(((struct mdt_rec_resync *)0)->rs_cap) == 4, "found %lld\n", + (long long)(int)sizeof(((struct mdt_rec_resync *)0)->rs_cap)); + LASSERTF((int)offsetof(struct mdt_rec_resync, rs_fsuid) == 8, "found %lld\n", + (long long)(int)offsetof(struct mdt_rec_resync, rs_fsuid)); + LASSERTF((int)sizeof(((struct mdt_rec_resync *)0)->rs_fsuid) == 4, "found %lld\n", + (long long)(int)sizeof(((struct mdt_rec_resync *)0)->rs_fsuid)); + LASSERTF((int)offsetof(struct mdt_rec_resync, rs_fsuid_h) == 12, "found %lld\n", + (long long)(int)offsetof(struct mdt_rec_resync, rs_fsuid_h)); + LASSERTF((int)sizeof(((struct mdt_rec_resync *)0)->rs_fsuid_h) == 4, "found %lld\n", + (long long)(int)sizeof(((struct mdt_rec_resync *)0)->rs_fsuid_h)); + LASSERTF((int)offsetof(struct mdt_rec_resync, rs_fsgid) == 16, "found %lld\n", + (long long)(int)offsetof(struct mdt_rec_resync, rs_fsgid)); + LASSERTF((int)sizeof(((struct mdt_rec_resync *)0)->rs_fsgid) == 4, "found %lld\n", + (long long)(int)sizeof(((struct mdt_rec_resync *)0)->rs_fsgid)); + LASSERTF((int)offsetof(struct mdt_rec_resync, rs_fsgid_h) == 20, "found %lld\n", + (long long)(int)offsetof(struct mdt_rec_resync, rs_fsgid_h)); + LASSERTF((int)sizeof(((struct mdt_rec_resync *)0)->rs_fsgid_h) == 4, "found %lld\n", + (long long)(int)sizeof(((struct mdt_rec_resync *)0)->rs_fsgid_h)); + LASSERTF((int)offsetof(struct mdt_rec_resync, rs_suppgid1) == 24, "found %lld\n", + (long long)(int)offsetof(struct mdt_rec_resync, rs_suppgid1)); + LASSERTF((int)sizeof(((struct mdt_rec_resync *)0)->rs_suppgid1) == 4, "found %lld\n", + (long long)(int)sizeof(((struct mdt_rec_resync *)0)->rs_suppgid1)); + LASSERTF((int)offsetof(struct mdt_rec_resync, rs_suppgid1_h) == 28, "found %lld\n", + (long long)(int)offsetof(struct mdt_rec_resync, rs_suppgid1_h)); + LASSERTF((int)sizeof(((struct mdt_rec_resync *)0)->rs_suppgid1_h) == 4, "found %lld\n", + (long long)(int)sizeof(((struct mdt_rec_resync *)0)->rs_suppgid1_h)); + LASSERTF((int)offsetof(struct mdt_rec_resync, rs_suppgid2) == 32, "found %lld\n", + (long long)(int)offsetof(struct mdt_rec_resync, rs_suppgid2)); + LASSERTF((int)sizeof(((struct mdt_rec_resync *)0)->rs_suppgid2) == 4, "found %lld\n", + (long long)(int)sizeof(((struct mdt_rec_resync *)0)->rs_suppgid2)); + LASSERTF((int)offsetof(struct mdt_rec_resync, rs_suppgid2_h) == 36, "found %lld\n", + (long long)(int)offsetof(struct mdt_rec_resync, rs_suppgid2_h)); + LASSERTF((int)sizeof(((struct mdt_rec_resync *)0)->rs_suppgid2_h) == 4, "found %lld\n", + (long long)(int)sizeof(((struct mdt_rec_resync *)0)->rs_suppgid2_h)); + LASSERTF((int)offsetof(struct mdt_rec_resync, rs_fid) == 40, "found %lld\n", + (long long)(int)offsetof(struct mdt_rec_resync, rs_fid)); + LASSERTF((int)sizeof(((struct mdt_rec_resync *)0)->rs_fid) == 16, "found %lld\n", + (long long)(int)sizeof(((struct mdt_rec_resync *)0)->rs_fid)); + LASSERTF((int)offsetof(struct mdt_rec_resync, rs_padding0) == 56, "found %lld\n", + (long long)(int)offsetof(struct mdt_rec_resync, rs_padding0)); + LASSERTF((int)sizeof(((struct mdt_rec_resync *)0)->rs_padding0) == 16, "found %lld\n", + (long long)(int)sizeof(((struct mdt_rec_resync *)0)->rs_padding0)); + LASSERTF((int)offsetof(struct mdt_rec_resync, rs_padding1) == 80, "found %lld\n", + (long long)(int)offsetof(struct mdt_rec_resync, rs_padding1)); + LASSERTF((int)sizeof(((struct mdt_rec_resync *)0)->rs_padding1) == 8, "found %lld\n", + (long long)(int)sizeof(((struct mdt_rec_resync *)0)->rs_padding1)); + LASSERTF((int)offsetof(struct mdt_rec_resync, rs_padding2) == 88, "found %lld\n", + (long long)(int)offsetof(struct mdt_rec_resync, rs_padding2)); + LASSERTF((int)sizeof(((struct mdt_rec_resync *)0)->rs_padding2) == 8, "found %lld\n", + (long long)(int)sizeof(((struct mdt_rec_resync *)0)->rs_padding2)); + LASSERTF((int)offsetof(struct mdt_rec_resync, rs_padding3) == 96, "found %lld\n", + (long long)(int)offsetof(struct mdt_rec_resync, rs_padding3)); + LASSERTF((int)sizeof(((struct mdt_rec_resync *)0)->rs_padding3) == 8, "found %lld\n", + (long long)(int)sizeof(((struct mdt_rec_resync *)0)->rs_padding3)); + LASSERTF((int)offsetof(struct mdt_rec_resync, rs_padding4) == 104, "found %lld\n", + (long long)(int)offsetof(struct mdt_rec_resync, rs_padding4)); + LASSERTF((int)sizeof(((struct mdt_rec_resync *)0)->rs_padding4) == 8, "found %lld\n", + (long long)(int)sizeof(((struct mdt_rec_resync *)0)->rs_padding4)); + LASSERTF((int)offsetof(struct mdt_rec_resync, rs_bias) == 112, "found %lld\n", + (long long)(int)offsetof(struct mdt_rec_resync, rs_bias)); + LASSERTF((int)sizeof(((struct mdt_rec_resync *)0)->rs_bias) == 4, "found %lld\n", + (long long)(int)sizeof(((struct mdt_rec_resync *)0)->rs_bias)); + LASSERTF((int)offsetof(struct mdt_rec_resync, rs_padding5) == 116, "found %lld\n", + (long long)(int)offsetof(struct mdt_rec_resync, rs_padding5)); + LASSERTF((int)sizeof(((struct mdt_rec_resync *)0)->rs_padding5) == 4, "found %lld\n", + (long long)(int)sizeof(((struct mdt_rec_resync *)0)->rs_padding5)); + LASSERTF((int)offsetof(struct mdt_rec_resync, rs_padding6) == 120, "found %lld\n", + (long long)(int)offsetof(struct mdt_rec_resync, rs_padding6)); + LASSERTF((int)sizeof(((struct mdt_rec_resync *)0)->rs_padding6) == 4, "found %lld\n", + (long long)(int)sizeof(((struct mdt_rec_resync *)0)->rs_padding6)); + LASSERTF((int)offsetof(struct mdt_rec_resync, rs_padding7) == 124, "found %lld\n", + (long long)(int)offsetof(struct mdt_rec_resync, rs_padding7)); + LASSERTF((int)sizeof(((struct mdt_rec_resync *)0)->rs_padding7) == 4, "found %lld\n", + (long long)(int)sizeof(((struct mdt_rec_resync *)0)->rs_padding7)); + LASSERTF((int)offsetof(struct mdt_rec_resync, rs_padding8) == 128, "found %lld\n", + (long long)(int)offsetof(struct mdt_rec_resync, rs_padding8)); + LASSERTF((int)sizeof(((struct mdt_rec_resync *)0)->rs_padding8) == 4, "found %lld\n", + (long long)(int)sizeof(((struct mdt_rec_resync *)0)->rs_padding8)); + LASSERTF((int)offsetof(struct mdt_rec_resync, rs_padding9) == 132, "found %lld\n", + (long long)(int)offsetof(struct mdt_rec_resync, rs_padding9)); + LASSERTF((int)sizeof(((struct mdt_rec_resync *)0)->rs_padding9) == 4, "found %lld\n", + (long long)(int)sizeof(((struct mdt_rec_resync *)0)->rs_padding9)); + /* Checks for struct mdt_rec_reint */ LASSERTF((int)sizeof(struct mdt_rec_reint) == 136, "found %lld\n", (long long)(int)sizeof(struct mdt_rec_reint)); diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index 27146a6..0692c98 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -1396,6 +1396,7 @@ enum mdt_reint_cmd { REINT_SETXATTR = 7, REINT_RMENTRY = 8, REINT_MIGRATE = 9, + REINT_RESYNC = 10, REINT_MAX }; @@ -1671,11 +1672,12 @@ struct mdt_rec_setattr { * being opened with conflict mode. */ #define MDS_OPEN_RELEASE 02000000000000ULL /* Open the file for HSM release */ +#define MDS_OPEN_RESYNC 04000000000000ULL /* FLR: file resync */ #define MDS_OPEN_FL_INTERNAL (MDS_OPEN_HAS_EA | MDS_OPEN_HAS_OBJS | \ MDS_OPEN_OWNEROVERRIDE | MDS_OPEN_LOCK | \ MDS_OPEN_BY_FID | MDS_OPEN_LEASE | \ - MDS_OPEN_RELEASE) + MDS_OPEN_RELEASE | MDS_OPEN_RESYNC) enum mds_op_bias { MDS_CHECK_SPLIT = 1 << 0, @@ -1694,10 +1696,11 @@ enum mds_op_bias { MDS_RENAME_MIGRATE = 1 << 13, MDS_CLOSE_LAYOUT_SWAP = 1 << 14, MDS_CLOSE_LAYOUT_MERGE = 1 << 15, + MDS_CLOSE_RESYNC_DONE = 1 << 16, }; #define MDS_CLOSE_INTENT (MDS_HSM_RELEASE | MDS_CLOSE_LAYOUT_SWAP | \ - MDS_CLOSE_LAYOUT_MERGE) + MDS_CLOSE_LAYOUT_MERGE | MDS_CLOSE_RESYNC_DONE) /* instance of mdt_reint_rec */ struct mdt_rec_create { @@ -1840,6 +1843,35 @@ struct mdt_rec_setxattr { __u32 sx_padding_11; /* rr_padding_4 */ }; +/* instance of mdt_reint_rec + * FLR: for file resync MDS_REINT_RESYNC RPC. + */ +struct mdt_rec_resync { + __u32 rs_opcode; + __u32 rs_cap; + __u32 rs_fsuid; + __u32 rs_fsuid_h; + __u32 rs_fsgid; + __u32 rs_fsgid_h; + __u32 rs_suppgid1; + __u32 rs_suppgid1_h; + __u32 rs_suppgid2; + __u32 rs_suppgid2_h; + struct lu_fid rs_fid; + __u8 rs_padding0[sizeof(struct lu_fid)]; + struct lustre_handle rs_handle; /* rr_mtime */ + __s64 rs_padding1; /* rr_atime */ + __s64 rs_padding2; /* rr_ctime */ + __u64 rs_padding3; /* rr_size */ + __u64 rs_padding4; /* rr_blocks */ + __u32 rs_bias; + __u32 rs_padding5; /* rr_mode */ + __u32 rs_padding6; /* rr_flags */ + __u32 rs_padding7; /* rr_flags_h */ + __u32 rs_padding8; /* rr_umask */ + __u32 rs_padding9; /* rr_padding_4 */ +}; + /* * mdt_rec_reint is the template for all mdt_reint_xxx structures. * Do NOT change the size of various members, otherwise the value @@ -2855,11 +2887,20 @@ struct mdc_swap_layouts { __u64 msl_flags; } __packed; +#define INLINE_RESYNC_ARRAY_SIZE 15 +struct close_data_resync_done { + __u32 resync_count; + __u32 resync_ids_inline[INLINE_RESYNC_ARRAY_SIZE]; +}; + struct close_data { struct lustre_handle cd_handle; struct lu_fid cd_fid; __u64 cd_data_version; - __u64 cd_reserved[8]; + union { + __u64 cd_reserved[8]; + struct close_data_resync_done cd_resync; + }; }; /* diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index f5cd979..779b9af 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -238,6 +238,31 @@ struct ll_futimes_3 { }; /* + * Maximum number of mirrors currently implemented. + */ +#define LUSTRE_MIRROR_COUNT_MAX 16 + +/* Lease types for use as arg and return of LL_IOC_{GET,SET}_LEASE ioctl. */ +enum ll_lease_mode { + LL_LEASE_RDLCK = 0x01, + LL_LEASE_WRLCK = 0x02, + LL_LEASE_UNLCK = 0x04, +}; + +enum ll_lease_flags { + LL_LEASE_RESYNC = 0x1, + LL_LEASE_RESYNC_DONE = 0x2, +}; + +#define IOC_IDS_MAX 4096 +struct ll_ioc_lease { + __u32 lil_mode; + __u32 lil_flags; + __u32 lil_count; + __u32 lil_ids[0]; +}; + +/* * The ioctl naming rules: * LL_* - works on the currently opened filehandle instead of parent dir * *_OBD_* - gets data for both OSC or MDC (LOV, LMV indirectly) @@ -294,7 +319,8 @@ struct ll_futimes_3 { #define LL_IOC_LMV_SETSTRIPE _IOWR('f', 240, struct lmv_user_md) #define LL_IOC_LMV_GETSTRIPE _IOWR('f', 241, struct lmv_user_md) -#define LL_IOC_SET_LEASE _IOWR('f', 243, long) +#define LL_IOC_SET_LEASE _IOWR('f', 243, struct ll_ioc_lease) +#define LL_IOC_SET_LEASE_OLD _IOWR('f', 243, long) #define LL_IOC_GET_LEASE _IO('f', 244) #define LL_IOC_HSM_IMPORT _IOWR('f', 245, struct hsm_user_import) #define LL_IOC_LMV_SET_DEFAULT_STRIPE _IOWR('f', 246, struct lmv_user_md) @@ -303,13 +329,6 @@ struct ll_futimes_3 { #define LL_IOC_GETPARENT _IOWR('f', 249, struct getparent) #define LL_IOC_LADVISE _IOR('f', 250, struct llapi_lu_ladvise) -/* Lease types for use as arg and return of LL_IOC_{GET,SET}_LEASE ioctl. */ -enum ll_lease_type { - LL_LEASE_RDLCK = 0x1, - LL_LEASE_WRLCK = 0x2, - LL_LEASE_UNLCK = 0x4, -}; - #define LL_STATFS_LMV 1 #define LL_STATFS_LOV 2 #define LL_STATFS_NODELAY 4 From patchwork Mon Sep 30 18:55:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167219 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CEB4F1902 for ; Mon, 30 Sep 2019 19:04:02 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B6926224EF for ; Mon, 30 Sep 2019 19:04:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B6926224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 1C1C65E463B; Mon, 30 Sep 2019 11:59:53 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id A48E05C3BE9 for ; Mon, 30 Sep 2019 11:57:22 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 1EE6510058ED; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 1D432B4; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:35 -0400 Message-Id: <1569869810-23848-77-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 076/151] lustre: flr: randomize mirror pick X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Jinshan Xiong Pick mirror randomly for read on client side and write on the MDT. WC-bug-id: https://jira.whamcloud.com/browse/LU-9771 Lustre-commit: c4c8cd23f9a3 ("LU-9771 flr: randomize mirror pick") Signed-off-by: Jinshan Xiong Reviewed-on: https://review.whamcloud.com/29097 Reviewed-by: Bobi Jam Reviewed-by: Andreas Dilger Signed-off-by: James Simmons --- fs/lustre/include/obd_support.h | 1 + fs/lustre/llite/lcommon_cl.c | 4 ++++ fs/lustre/lov/lov_object.c | 19 +++++++++++++++++++ 3 files changed, 24 insertions(+) diff --git a/fs/lustre/include/obd_support.h b/fs/lustre/include/obd_support.h index 9d383f5..1dce1c0 100644 --- a/fs/lustre/include/obd_support.h +++ b/fs/lustre/include/obd_support.h @@ -478,6 +478,7 @@ #define OBD_FAIL_FLR_GLIMPSE_IMMUTABLE 0x1A00 #define OBD_FAIL_FLR_LV_DELAY 0x1A01 #define OBD_FAIL_FLR_LV_INC 0x1A02 +#define OBD_FAIL_FLR_RANDOM_PICK_MIRROR 0x1A03 /* Assign references to moved code to reduce code changes */ #define OBD_FAIL_PRECHECK(id) CFS_FAIL_PRECHECK(id) diff --git a/fs/lustre/llite/lcommon_cl.c b/fs/lustre/llite/lcommon_cl.c index 156b1ad..0361adc 100644 --- a/fs/lustre/llite/lcommon_cl.c +++ b/fs/lustre/llite/lcommon_cl.c @@ -188,6 +188,10 @@ int cl_file_inode_init(struct inode *inode, struct lustre_md *md) } } else { result = cl_conf_set(env, lli->lli_clob, &conf); + if (result == -EBUSY) { + /* ignore the error since I/O will handle it later */ + result = 0; + } } cl_env_put(env, &refcheck); diff --git a/fs/lustre/lov/lov_object.c b/fs/lustre/lov/lov_object.c index f606e04..49facf3 100644 --- a/fs/lustre/lov/lov_object.c +++ b/fs/lustre/lov/lov_object.c @@ -756,6 +756,25 @@ static int lov_init_composite(const struct lu_env *env, struct lov_device *dev, goto out; } + if (OBD_FAIL_CHECK(OBD_FAIL_FLR_RANDOM_PICK_MIRROR)) { + unsigned int seq; + + get_random_bytes(&seq, sizeof(seq)); + seq %= mirror_count; + + i = 0; + lov_foreach_mirror_entry(lov, lre) { + i++; + if (lre->lre_stale) + continue; + + if (!seq--) { + comp->lo_preferred_mirror = i - 1; + break; + } + } + } + LASSERT(comp->lo_preferred_mirror >= 0); out: From patchwork Mon Sep 30 18:55:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167223 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 87FF616C1 for ; Mon, 30 Sep 2019 19:04:11 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6F91D224EF for ; Mon, 30 Sep 2019 19:04:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6F91D224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 2D03F5E469E; Mon, 30 Sep 2019 11:59:56 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id E7D595C3BE9 for ; Mon, 30 Sep 2019 11:57:22 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 22CDB10058EE; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 2076DBD; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:36 -0400 Message-Id: <1569869810-23848-78-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 077/151] lustre: flr: instantiate component for truncate X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Bobi Jam , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Bobi Jam layout intent for truncate IO needs stale components for non primary mirror overlapping [trunc_size, eof), while needs to instantiate components overlapping [0, trunc_size). WC-bug-id: https://jira.whamcloud.com/browse/LU-9771 Lustre-commit: 173981742e69 ("LU-9771 flr: instantiate component for truncate") Signed-off-by: Bobi Jam Signed-off-by: Jinshan Xiong Reviewed-on: https://review.whamcloud.com/29099 Reviewed-by: Jian Yu Reviewed-by: Lai Siyao Reviewed-by: Dmitry Eremin Signed-off-by: James Simmons --- fs/lustre/llite/file.c | 27 +++++++++++++++------------ fs/lustre/llite/lcommon_cl.c | 2 +- fs/lustre/llite/llite_internal.h | 3 ++- fs/lustre/llite/vvp_io.c | 8 ++++++-- fs/lustre/lov/lov_io.c | 6 +++--- include/uapi/linux/lustre/lustre_idl.h | 2 +- 6 files changed, 28 insertions(+), 20 deletions(-) diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index d13c583..22e3494 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -1723,6 +1723,10 @@ static int ll_lov_setstripe(struct inode *inode, struct file *file, struct cl_layout cl = { .cl_is_composite = false, }; + struct lu_extent ext = { + .e_start = 0, + .e_end = OBD_OBJECT_EOF, + }; struct lu_env *env; u16 refcheck; @@ -1732,7 +1736,8 @@ static int ll_lov_setstripe(struct inode *inode, struct file *file, rc = cl_object_layout_get(env, obj, &cl); if (!rc && cl.cl_is_composite) - rc = ll_layout_write_intent(inode, 0, OBD_OBJECT_EOF); + rc = ll_layout_write_intent(inode, LAYOUT_INTENT_WRITE, + &ext); cl_env_put(env, &refcheck); if (rc) @@ -4396,25 +4401,23 @@ int ll_layout_refresh(struct inode *inode, u32 *gen) * Issue layout intent RPC indicating where in a file an IO is about to write. * * @inode: file inode. - * @start: start offset of fille in bytes where an IO is about to - * write. - * @end: exclusive end offset in bytes of the write range. + * @ext: write range with start offset of fille in bytes where + * an IO is about to write, and exclusive end offset in + * bytes. * * Returns: 0 on success * < 0 error code */ -int ll_layout_write_intent(struct inode *inode, u64 start, u64 end) +int ll_layout_write_intent(struct inode *inode, enum layout_intent_opc opc, + struct lu_extent *ext) { struct layout_intent intent = { - .li_opc = LAYOUT_INTENT_WRITE, - .li_extent.e_start = start, - .li_extent.e_end = end, + .li_opc = opc, + .li_extent.e_start = ext->e_start, + .li_extent.e_end = ext->e_end, }; - int rc; - rc = ll_layout_intent(inode, &intent); - - return rc; + return ll_layout_intent(inode, &intent); } /** diff --git a/fs/lustre/llite/lcommon_cl.c b/fs/lustre/llite/lcommon_cl.c index 0361adc..24e58c2 100644 --- a/fs/lustre/llite/lcommon_cl.c +++ b/fs/lustre/llite/lcommon_cl.c @@ -103,10 +103,10 @@ int cl_setattr_ost(struct cl_object *obj, const struct iattr *attr, io->u.ci_setattr.sa_xvalid = xvalid; io->u.ci_setattr.sa_parent_fid = lu_object_fid(&obj->co_lu); +again: if (attr->ia_valid & ATTR_FILE) ll_io_set_mirror(io, attr->ia_file); -again: if (cl_io_init(env, io, CIT_SETATTR, io->ci_obj) == 0) { struct vvp_io *vio = vvp_env_io(env); diff --git a/fs/lustre/llite/llite_internal.h b/fs/lustre/llite/llite_internal.h index dc81474..a44aec06 100644 --- a/fs/lustre/llite/llite_internal.h +++ b/fs/lustre/llite/llite_internal.h @@ -1332,7 +1332,8 @@ static inline void d_lustre_revalidate(struct dentry *dentry) int ll_layout_conf(struct inode *inode, const struct cl_object_conf *conf); int ll_layout_refresh(struct inode *inode, u32 *gen); int ll_layout_restore(struct inode *inode, loff_t start, u64 length); -int ll_layout_write_intent(struct inode *inode, u64 start, u64 end); +int ll_layout_write_intent(struct inode *inode, enum layout_intent_opc opc, + struct lu_extent *ext); int ll_xattr_init(void); void ll_xattr_fini(void); diff --git a/fs/lustre/llite/vvp_io.c b/fs/lustre/llite/vvp_io.c index dc9ccef..c8a1ff1 100644 --- a/fs/lustre/llite/vvp_io.c +++ b/fs/lustre/llite/vvp_io.c @@ -347,6 +347,8 @@ static void vvp_io_fini(const struct lu_env *env, const struct cl_io_slice *ios) * RPC. */ if (io->ci_need_write_intent) { + enum layout_intent_opc opc = LAYOUT_INTENT_WRITE; + io->ci_need_write_intent = 0; LASSERT(io->ci_type == CIT_WRITE || @@ -356,8 +358,10 @@ static void vvp_io_fini(const struct lu_env *env, const struct cl_io_slice *ios) PFID(lu_object_fid(&obj->co_lu)), io->ci_type, PEXT(&io->ci_write_intent)); - rc = ll_layout_write_intent(inode, io->ci_write_intent.e_start, - io->ci_write_intent.e_end); + if (cl_io_is_trunc(io)) + opc = LAYOUT_INTENT_TRUNC; + + rc = ll_layout_write_intent(inode, opc, &io->ci_write_intent); io->ci_result = rc; if (!rc) io->ci_need_restart = 1; diff --git a/fs/lustre/lov/lov_io.c b/fs/lustre/lov/lov_io.c index 9fd1b52..75fa646 100644 --- a/fs/lustre/lov/lov_io.c +++ b/fs/lustre/lov/lov_io.c @@ -538,9 +538,10 @@ static int lov_io_slice_init(struct lov_io *lio, struct lov_object *obj, goto out; } - ext.e_start = lio->lis_pos; - ext.e_end = lio->lis_endpos; + io->ci_write_intent.e_start = lio->lis_pos; + io->ci_write_intent.e_end = lio->lis_endpos; + ext = io->ci_write_intent; /* for truncate, it only needs to instantiate the components * before the truncated size. */ @@ -553,7 +554,6 @@ static int lov_io_slice_init(struct lov_io *lio, struct lov_object *obj, lov_foreach_io_layout(index, lio, &ext) { if (!lsm_entry_inited(obj->lo_lsm, index)) { io->ci_need_write_intent = 1; - io->ci_write_intent = ext; break; } } diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index 0692c98..b2f68edd 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -2845,7 +2845,7 @@ struct getparent { char gp_name[0]; /**< zero-terminated link name */ } __packed; -enum { +enum layout_intent_opc { LAYOUT_INTENT_ACCESS = 0, /** generic access */ LAYOUT_INTENT_READ = 1, /** not used */ LAYOUT_INTENT_WRITE = 2, /** write file, for comp layout */ From patchwork Mon Sep 30 18:55:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167225 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 70CE016C1 for ; Mon, 30 Sep 2019 19:04:16 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 592C9224EF for ; Mon, 30 Sep 2019 19:04:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 592C9224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 23F915E45F0; Mon, 30 Sep 2019 11:59:58 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 4AF035C3B10 for ; Mon, 30 Sep 2019 11:57:23 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 2548710058EF; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 23305B5; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:37 -0400 Message-Id: <1569869810-23848-79-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 078/151] lustre: hsm: don't release with wrong size X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Boyko , Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Alexander Boyko The result of ll_merge_attr operation was ignored during ll_hsm_release. For this case the released file could have the wrong size on MDS, like File: ‘released.file’ Size: 0 Blocks: 0 IO Block: 4194304 regular empty file WC-bug-id: https://jira.whamcloud.com/browse/LU-10271 Lustre-commit: 09b457fffc8a ("LU-10271 hsm: don't release with wrong size") Signed-off-by: Alexander Boyko Reviewed-on: https://review.whamcloud.com/30240 Reviewed-by: Henri Doreau Reviewed-by: Sergey Cheremencev Reviewed-by: Quentin Bouget Reviewed-by: John L. Hammond Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/obd_support.h | 1 + fs/lustre/llite/file.c | 13 +++++++++++-- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/fs/lustre/include/obd_support.h b/fs/lustre/include/obd_support.h index 1dce1c0..6a01af4 100644 --- a/fs/lustre/include/obd_support.h +++ b/fs/lustre/include/obd_support.h @@ -384,6 +384,7 @@ #define OBD_FAIL_MDC_RPCS_SEM 0x804 #define OBD_FAIL_MDC_LIGHTWEIGHT 0x805 #define OBD_FAIL_MDC_CLOSE 0x806 +#define OBD_FAIL_MDC_MERGE 0x807 #define OBD_FAIL_MGS 0x900 #define OBD_FAIL_MGS_ALL_REQUEST_NET 0x901 diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 22e3494..164948c 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -1133,7 +1133,10 @@ int ll_merge_attr(const struct lu_env *env, struct inode *inode) ctime = inode->i_ctime.tv_sec; cl_object_attr_lock(obj); - rc = cl_object_attr_get(env, obj, attr); + if (OBD_FAIL_CHECK(OBD_FAIL_MDC_MERGE)) + rc = -EINVAL; + else + rc = cl_object_attr_get(env, obj, attr); cl_object_attr_unlock(obj); if (rc != 0) { @@ -2050,9 +2053,15 @@ int ll_hsm_release(struct inode *inode) goto out; } - ll_merge_attr(env, inode); + rc = ll_merge_attr(env, inode); cl_env_put(env, &refcheck); + /* If error happen, we have the wrong size for a file. + * Don't release it. + */ + if (rc != 0) + goto out; + /* Release the file. * NB: lease lock handle is released in mdc_hsm_release_pack() because * we still need it to pack l_remote_handle to MDT. From patchwork Mon Sep 30 18:55:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167229 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 363391902 for ; Mon, 30 Sep 2019 19:04:26 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1EAE5224EF for ; Mon, 30 Sep 2019 19:04:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1EAE5224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 9135F5E4747; Mon, 30 Sep 2019 12:00:01 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8FA8D5C39C7 for ; Mon, 30 Sep 2019 11:57:23 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 27DED10058F0; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 2655CBB; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:38 -0400 Message-Id: <1569869810-23848-80-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 079/151] lustre: mdc: Add an additional set of 64 changelog flags X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Matthew Sanderson , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Matthew Sanderson This adds a new changelog extension containing 64 additional flag bits, to be used for future changelog extensions. The presence of the extension is signalled using the last remaining unused changelog flag bit. The new extension is present in all changelog records by default, but will be removed from records read by legacy changelog consumers. WC-bug-id: https://jira.whamcloud.com/browse/LU-9727 Lustre-commit: 08ffb6f1428f ("LU-9727 mdc: Add an additional set of 64 changelog flags") Signed-off-by: Matthew Sanderson Signed-off-by: Sebastien Buisson Reviewed-on: https://review.whamcloud.com/28045 Reviewed-by: Andreas Dilger Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/mdc/mdc_changelog.c | 5 +- include/uapi/linux/lustre/lustre_idl.h | 3 +- include/uapi/linux/lustre/lustre_user.h | 119 +++++++++++++++++++++++++++----- 3 files changed, 106 insertions(+), 21 deletions(-) diff --git a/fs/lustre/mdc/mdc_changelog.c b/fs/lustre/mdc/mdc_changelog.c index 011bcc9..e68ccb9 100644 --- a/fs/lustre/mdc/mdc_changelog.c +++ b/fs/lustre/mdc/mdc_changelog.c @@ -231,7 +231,10 @@ static int chlg_load(void *args) goto err_out; } - rc = llog_init_handle(NULL, llh, LLOG_F_IS_CAT | LLOG_F_EXT_JOBID, + rc = llog_init_handle(NULL, llh, + LLOG_F_IS_CAT | + LLOG_F_EXT_JOBID | + LLOG_F_EXT_EXTRA_FLAGS, NULL); if (rc) { CERROR("%s: fail to init llog handle: rc = %d\n", diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index b2f68edd..c23d1cb 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -2577,6 +2577,7 @@ enum llog_flag { LLOG_F_IS_PLAIN = 0x4, LLOG_F_EXT_JOBID = 0x8, LLOG_F_IS_FIXSIZE = 0x10, + LLOG_F_EXT_EXTRA_FLAGS = 0x20, /* * Note: Flags covered by LLOG_F_EXT_MASK will be inherited from @@ -2584,7 +2585,7 @@ enum llog_flag { * because the catlog record is usually fixed size, but its plain * log record can be variable */ - LLOG_F_EXT_MASK = LLOG_F_EXT_JOBID, + LLOG_F_EXT_MASK = LLOG_F_EXT_JOBID | LLOG_F_EXT_EXTRA_FLAGS, }; /* On-disk header structure of each log object, stored in little endian order */ diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index 779b9af..8e82359 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -885,7 +885,8 @@ enum changelog_rec_flags { CLF_VERSION = 0x1000, CLF_RENAME = 0x2000, CLF_JOBID = 0x4000, - CLF_SUPPORTED = CLF_VERSION | CLF_RENAME | CLF_JOBID + CLF_EXTRA_FLAGS = 0x8000, + CLF_SUPPORTED = CLF_VERSION | CLF_RENAME | CLF_JOBID | CLF_EXTRA_FLAGS }; /* Anything under the flagmask may be per-type (if desired) */ @@ -971,6 +972,11 @@ static inline void hsm_set_cl_error(int *flags, int error) *flags |= (error << CLF_HSM_ERR_L); } +enum changelog_rec_extra_flags { + CLFE_INVALID = 0, /* No additional flags currently implemented */ + CLFE_SUPPORTED = CLFE_INVALID +}; + enum changelog_send_flag { /* Not yet implemented */ CHANGELOG_FLAG_FOLLOW = 0x01, @@ -982,10 +988,13 @@ enum changelog_send_flag { CHANGELOG_FLAG_BLOCK = 0x02, /* Pack jobid into the changelog records if available. */ CHANGELOG_FLAG_JOBID = 0x04, + /* Pack additional flag bits into the changelog record */ + CHANGELOG_FLAG_EXTRA_FLAGS = 0x08, }; #define CR_MAXSIZE __ALIGN_KERNEL(2 * NAME_MAX + 2 + \ - changelog_rec_offset(CLF_SUPPORTED), 8) + changelog_rec_offset(CLF_SUPPORTED, \ + CLFE_SUPPORTED), 8) /* 31 usable bytes string + null terminator. */ #define LUSTRE_JOBID_SIZE 32 @@ -993,9 +1002,10 @@ enum changelog_send_flag { /* * This is the minimal changelog record. It can contain extensions * such as rename fields or process jobid. Its exact content is described - * by the cr_flags. + * by the cr_flags and cr_extra_flags. * - * Extensions are packed in the same order as their corresponding flags. + * Extensions are packed in the same order as their corresponding flags, + * then in the same order as their corresponding extra flags. */ struct changelog_rec { __u16 cr_namelen; @@ -1022,7 +1032,16 @@ struct changelog_ext_jobid { char cr_jobid[LUSTRE_JOBID_SIZE]; /**< zero-terminated string. */ }; -static inline size_t changelog_rec_offset(enum changelog_rec_flags crf) +/* Changelog extension to include additional flags. */ +struct changelog_ext_extra_flags { + __u64 cr_extra_flags; /* Additional CLFE_* flags */ +}; + +static inline struct changelog_ext_extra_flags *changelog_rec_extra_flags( + const struct changelog_rec *rec); + +static inline size_t changelog_rec_offset(enum changelog_rec_flags crf, + enum changelog_rec_extra_flags cref) { size_t size = sizeof(struct changelog_rec); @@ -1032,12 +1051,20 @@ static inline size_t changelog_rec_offset(enum changelog_rec_flags crf) if (crf & CLF_JOBID) size += sizeof(struct changelog_ext_jobid); + if (crf & CLF_EXTRA_FLAGS) + size += sizeof(struct changelog_ext_extra_flags); + return size; } static inline size_t changelog_rec_size(struct changelog_rec *rec) { - return changelog_rec_offset(rec->cr_flags); + enum changelog_rec_extra_flags cref = CLFE_INVALID; + + if (rec->cr_flags & CLF_EXTRA_FLAGS) + cref = changelog_rec_extra_flags(rec)->cr_extra_flags; + + return changelog_rec_offset(rec->cr_flags, cref); } static inline size_t changelog_rec_varsize(struct changelog_rec *rec) @@ -1051,7 +1078,8 @@ struct changelog_ext_rename *changelog_rec_rename(struct changelog_rec *rec) enum changelog_rec_flags crf = rec->cr_flags & CLF_VERSION; return (struct changelog_ext_rename *)((char *)rec + - changelog_rec_offset(crf)); + changelog_rec_offset(crf, + CLFE_INVALID)); } /* The jobid follows the rename extension, if present */ @@ -1062,14 +1090,33 @@ struct changelog_ext_jobid *changelog_rec_jobid(struct changelog_rec *rec) (CLF_VERSION | CLF_RENAME); return (struct changelog_ext_jobid *)((char *)rec + - changelog_rec_offset(crf)); + changelog_rec_offset(crf, + CLFE_INVALID)); } -/* The name follows the rename and jobid extensions, if present */ +/* The additional flags follow the rename and jobid extensions, if present */ +static inline +struct changelog_ext_extra_flags *changelog_rec_extra_flags( + const struct changelog_rec *rec) +{ + enum changelog_rec_flags crf = rec->cr_flags & + (CLF_VERSION | CLF_RENAME | CLF_JOBID); + + return (struct changelog_ext_extra_flags *)((char *)rec + + changelog_rec_offset(crf, + CLFE_INVALID)); +} + +/* The name follows the rename, jobid and extra flags extns, if present */ static inline char *changelog_rec_name(struct changelog_rec *rec) { - return (char *)rec + changelog_rec_offset(rec->cr_flags & - CLF_SUPPORTED); + enum changelog_rec_extra_flags cref = CLFE_INVALID; + + if (rec->cr_flags & CLF_EXTRA_FLAGS) + cref = changelog_rec_extra_flags(rec)->cr_extra_flags; + + return (char *)rec + changelog_rec_offset(rec->cr_flags & CLF_SUPPORTED, + cref & CLFE_SUPPORTED); } static inline size_t changelog_rec_snamelen(struct changelog_rec *rec) @@ -1099,31 +1146,57 @@ static inline char *changelog_rec_sname(struct changelog_rec *rec) * The following assumptions are being made: * - CLF_RENAME will not be removed * - CLF_JOBID will not be added without CLF_RENAME being added too + * - CLF_EXTRA_FLAGS will not be added without CLF_JOBID being added too * * @rec The record to remap. * @crf_wanted Flags describing the desired extensions. + * @cref_want Flags describing the desired extra extensions. */ static inline void changelog_remap_rec(struct changelog_rec *rec, - enum changelog_rec_flags crf_wanted) + enum changelog_rec_flags crf_wanted, + enum changelog_rec_extra_flags cref_want) { + char *ef_mov; char *jid_mov, *rnm_mov; crf_wanted &= CLF_SUPPORTED; - - if ((rec->cr_flags & CLF_SUPPORTED) == crf_wanted) - return; + cref_want &= CLFE_SUPPORTED; + + if ((rec->cr_flags & CLF_SUPPORTED) == crf_wanted) { + if (!(rec->cr_flags & CLF_EXTRA_FLAGS) || + (rec->cr_flags & CLF_EXTRA_FLAGS && + (changelog_rec_extra_flags(rec)->cr_extra_flags & + CLFE_SUPPORTED) == + cref_want)) + return; + } /* First move the variable-length name field */ - memmove((char *)rec + changelog_rec_offset(crf_wanted), + memmove((char *)rec + changelog_rec_offset(crf_wanted, cref_want), changelog_rec_name(rec), rec->cr_namelen); - /* Locations of jobid and rename extensions in the remapped record */ + /* Locations of extensions in the remapped record */ + ef_mov = (char *)rec + + changelog_rec_offset(crf_wanted & ~CLF_EXTRA_FLAGS, + CLFE_INVALID); jid_mov = (char *)rec + - changelog_rec_offset(crf_wanted & ~CLF_JOBID); + changelog_rec_offset(crf_wanted & + ~(CLF_EXTRA_FLAGS | CLF_JOBID), + CLFE_INVALID); rnm_mov = (char *)rec + - changelog_rec_offset(crf_wanted & ~(CLF_JOBID | CLF_RENAME)); + changelog_rec_offset(crf_wanted & + ~(CLF_EXTRA_FLAGS | + CLF_JOBID | + CLF_RENAME), + CLFE_INVALID); /* Move the extension fields to the desired positions */ + if ((crf_wanted & CLF_EXTRA_FLAGS) && + (rec->cr_flags & CLF_EXTRA_FLAGS)) { + memmove(ef_mov, changelog_rec_extra_flags(rec), + sizeof(struct changelog_ext_extra_flags)); + } + if ((crf_wanted & CLF_JOBID) && (rec->cr_flags & CLF_JOBID)) memmove(jid_mov, changelog_rec_jobid(rec), sizeof(struct changelog_ext_jobid)); @@ -1133,6 +1206,10 @@ static inline void changelog_remap_rec(struct changelog_rec *rec, sizeof(struct changelog_ext_rename)); /* Clear newly added fields */ + if ((crf_wanted & CLF_EXTRA_FLAGS) && + !(rec->cr_flags & CLF_EXTRA_FLAGS)) + memset(ef_mov, 0, sizeof(struct changelog_ext_extra_flags)); + if ((crf_wanted & CLF_JOBID) && !(rec->cr_flags & CLF_JOBID)) memset(jid_mov, 0, sizeof(struct changelog_ext_jobid)); @@ -1141,6 +1218,10 @@ static inline void changelog_remap_rec(struct changelog_rec *rec, /* Update the record's flags accordingly */ rec->cr_flags = (rec->cr_flags & CLF_FLAGMASK) | crf_wanted; + if (rec->cr_flags & CLF_EXTRA_FLAGS) + changelog_rec_extra_flags(rec)->cr_extra_flags = + changelog_rec_extra_flags(rec)->cr_extra_flags | + cref_want; } enum changelog_message_type { From patchwork Mon Sep 30 18:55:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167227 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7495616C1 for ; Mon, 30 Sep 2019 19:04:21 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5CEE3224EF for ; Mon, 30 Sep 2019 19:04:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5CEE3224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id BEC3C5E4717; Mon, 30 Sep 2019 11:59:59 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id E8E855C3A82 for ; Mon, 30 Sep 2019 11:57:23 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 2B9E310058F1; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 29CE9A9; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:39 -0400 Message-Id: <1569869810-23848-81-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 080/151] lustre: ldlm: assume OBD_CONNECT_IBITS X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: "John L. Hammond" Clients and MDSs have supported and required OBD_CONNECT_IBITS since before 1.6 so remove obsolete code to handle clients that do not support this flag. WC-bug-id: https://jira.whamcloud.com/browse/LU-8402 Lustre-commit: c7a833830de6 ("LU-8402 ldlm: assume OBD_CONNECT_IBITS") Signed-off-by: John L. Hammond Reviewed-on: https://review.whamcloud.com/30009 Reviewed-by: Andreas Dilger Reviewed-by: Dmitry Eremin Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/ldlm/ldlm_request.c | 17 +++++++++-------- fs/lustre/ptlrpc/import.c | 10 ---------- include/uapi/linux/lustre/lustre_idl.h | 4 ++-- 3 files changed, 11 insertions(+), 20 deletions(-) diff --git a/fs/lustre/ldlm/ldlm_request.c b/fs/lustre/ldlm/ldlm_request.c index 65c3558..eb99279 100644 --- a/fs/lustre/ldlm/ldlm_request.c +++ b/fs/lustre/ldlm/ldlm_request.c @@ -445,14 +445,15 @@ int ldlm_cli_enqueue_fini(struct obd_export *exp, struct ptlrpc_request *req, } LDLM_DEBUG(lock, "client-side enqueue, new resource"); } - if (with_policy) - if (!(type == LDLM_IBITS && - !(exp_connect_flags(exp) & OBD_CONNECT_IBITS))) - /* We assume lock type cannot change on server*/ - ldlm_convert_policy_to_local(exp, - lock->l_resource->lr_type, - &reply->lock_desc.l_policy_data, - &lock->l_policy_data); + + if (with_policy) { + /* We assume lock type cannot change on server*/ + ldlm_convert_policy_to_local(exp, + lock->l_resource->lr_type, + &reply->lock_desc.l_policy_data, + &lock->l_policy_data); + } + if (type != LDLM_PLAIN) LDLM_DEBUG(lock, "client-side enqueue, new policy data"); diff --git a/fs/lustre/ptlrpc/import.c b/fs/lustre/ptlrpc/import.c index 32aceb9..6e2af7f 100644 --- a/fs/lustre/ptlrpc/import.c +++ b/fs/lustre/ptlrpc/import.c @@ -754,16 +754,6 @@ static int ptlrpc_connect_set_flags(struct obd_import *imp, struct client_obd *cli = &imp->imp_obd->u.cli; static bool warned; - if ((imp->imp_connect_flags_orig & OBD_CONNECT_IBITS) && - !(ocd->ocd_connect_flags & OBD_CONNECT_IBITS)) { - LCONSOLE_WARN("%s: MDS %s does not support ibits lock, either very old or invalid: requested %#llx, replied %#llx\n", - imp->imp_obd->obd_name, - imp->imp_connection->c_remote_uuid.uuid, - imp->imp_connect_flags_orig, - ocd->ocd_connect_flags); - return -EPROTO; - } - spin_lock(&imp->imp_lock); list_del(&imp->imp_conn_current->oic_item); list_add(&imp->imp_conn_current->oic_item, &imp->imp_conn_list); diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index c23d1cb..078ad6c 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -702,7 +702,7 @@ struct ptlrpc_body_v2 { #define OBD_CONNECT_LARGE_ACL 0x200ULL /* more than 32 ACL entries */ #define OBD_CONNECT_TRUNCLOCK 0x400ULL /*locks on server for punch */ #define OBD_CONNECT_TRANSNO 0x800ULL /*replay sends init transno */ -#define OBD_CONNECT_IBITS 0x1000ULL /*support for inodebits locks*/ +#define OBD_CONNECT_IBITS 0x1000ULL /* not checked in 2.11+ */ #define OBD_CONNECT_JOIN 0x2000ULL /*files can be concatenated. *We do not support JOIN FILE *anymore, reserve this flags @@ -809,7 +809,7 @@ struct ptlrpc_body_v2 { (!!((ocd)->ocd_connect_flags & OBD_CONNECT_##flg)) /* Features required for this version of the client to work with server */ -#define CLIENT_CONNECT_MDT_REQD (OBD_CONNECT_IBITS | OBD_CONNECT_FID | \ +#define CLIENT_CONNECT_MDT_REQD (OBD_CONNECT_FID | \ OBD_CONNECT_FULL20) /* This structure is used for both request and reply. From patchwork Mon Sep 30 18:55:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167189 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A5354912 for ; Mon, 30 Sep 2019 19:02:27 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8D912224EF for ; Mon, 30 Sep 2019 19:02:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8D912224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id A610D5E446C; Mon, 30 Sep 2019 11:59:21 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 4EA495C3BF6 for ; Mon, 30 Sep 2019 11:57:24 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 2E8D210058F6; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 2CA3BB4; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:40 -0400 Message-Id: <1569869810-23848-82-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 081/151] lustre: llite: assume OBD_CONNECT_ATTRFID X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: "John L. Hammond" OBD_CONNECT_ATTRFID has been supported by MDSs since before 1.6 so add it to the client side required flags and remove some code to handle servers that do not support it. WC-bug-id: https://jira.whamcloud.com/browse/LU-8402 Lustre-commit: deaef32d1fd6 ("LU-8402 llite: assume OBD_CONNECT_ATTRFID") Signed-off-by: John L. Hammond Reviewed-on: https://review.whamcloud.com/30010 Reviewed-by: Andreas Dilger Reviewed-by: Dmitry Eremin Signed-off-by: James Simmons --- fs/lustre/llite/file.c | 112 +++++++++++---------------------- include/uapi/linux/lustre/lustre_idl.h | 1 + 2 files changed, 39 insertions(+), 74 deletions(-) diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 164948c..7f04e4f 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -3733,10 +3733,15 @@ static int ll_inode_revalidate_fini(struct inode *inode, int rc) return rc; } -static int __ll_inode_revalidate(struct dentry *dentry, u64 ibits) +static int __ll_inode_revalidate(struct dentry *dentry, + enum ldlm_intent_flags op) { struct inode *inode = d_inode(dentry); + struct lookup_intent oit = { + .it_op = op, + }; struct ptlrpc_request *req = NULL; + struct md_op_data *op_data; struct obd_export *exp; int rc = 0; @@ -3745,79 +3750,40 @@ static int __ll_inode_revalidate(struct dentry *dentry, u64 ibits) exp = ll_i2mdexp(inode); - /* XXX: Enable OBD_CONNECT_ATTRFID to reduce unnecessary getattr RPC. - * But under CMD case, it caused some lock issues, should be fixed - * with new CMD ibits lock. See bug 12718 - */ - if (exp_connect_flags(exp) & OBD_CONNECT_ATTRFID) { - struct lookup_intent oit = { .it_op = IT_GETATTR }; - struct md_op_data *op_data; - - if (ibits == MDS_INODELOCK_LOOKUP) - oit.it_op = IT_LOOKUP; - - /* Call getattr by fid, so do not provide name at all. */ - op_data = ll_prep_md_op_data(NULL, inode, - inode, NULL, 0, 0, - LUSTRE_OPC_ANY, NULL); - if (IS_ERR(op_data)) - return PTR_ERR(op_data); - - rc = md_intent_lock(exp, op_data, &oit, &req, - &ll_md_blocking_ast, 0); - ll_finish_md_op_data(op_data); - if (rc < 0) { - rc = ll_inode_revalidate_fini(inode, rc); - goto out; - } - - rc = ll_revalidate_it_finish(req, &oit, inode); - if (rc != 0) { - ll_intent_release(&oit); - goto out; - } - - /* Unlinked? Unhash dentry, so it is not picked up later by - * do_lookup() -> ll_revalidate_it(). We cannot use d_drop - * here to preserve get_cwd functionality on 2.6. - * Bug 10503 - */ - if (!d_inode(dentry)->i_nlink) { - spin_lock(&inode->i_lock); - d_lustre_invalidate(dentry, 0); - spin_unlock(&inode->i_lock); - } - - ll_lookup_finish_locks(&oit, inode); - } else if (!ll_have_md_lock(d_inode(dentry), &ibits, LCK_MINMODE)) { - struct ll_sb_info *sbi = ll_i2sbi(d_inode(dentry)); - u64 valid = OBD_MD_FLGETATTR; - struct md_op_data *op_data; - int ealen = 0; - - if (S_ISREG(inode->i_mode)) { - rc = ll_get_default_mdsize(sbi, &ealen); - if (rc) - return rc; - valid |= OBD_MD_FLEASIZE | OBD_MD_FLMODEASIZE; - } + /* Call getattr by fid, so do not provide name at all. */ + op_data = ll_prep_md_op_data(NULL, d_inode(dentry), d_inode(dentry), + NULL, 0, 0, LUSTRE_OPC_ANY, NULL); + if (IS_ERR(op_data)) + return PTR_ERR(op_data); - op_data = ll_prep_md_op_data(NULL, inode, NULL, NULL, - 0, ealen, LUSTRE_OPC_ANY, - NULL); - if (IS_ERR(op_data)) - return PTR_ERR(op_data); + rc = md_intent_lock(exp, op_data, &oit, &req, &ll_md_blocking_ast, 0); + ll_finish_md_op_data(op_data); + if (rc < 0) { + rc = ll_inode_revalidate_fini(inode, rc); + goto out; + } - op_data->op_valid = valid; - rc = md_getattr(sbi->ll_md_exp, op_data, &req); - ll_finish_md_op_data(op_data); - if (rc) - return ll_inode_revalidate_fini(inode, rc); + rc = ll_revalidate_it_finish(req, &oit, inode); + if (rc != 0) { + ll_intent_release(&oit); + goto out; + } - rc = ll_prep_inode(&inode, req, NULL, NULL); + /* Unlinked? Unhash dentry, so it is not picked up later by + * do_lookup() -> ll_revalidate_it(). We cannot use d_drop + * here to preserve get_cwd functionality on 2.6. + * Bug 10503 + */ + if (!d_inode(dentry)->i_nlink) { + spin_lock(&inode->i_lock); + d_lustre_invalidate(dentry, 0); + spin_unlock(&inode->i_lock); } + + ll_lookup_finish_locks(&oit, inode); out: ptlrpc_req_finished(req); + return rc; } @@ -3843,12 +3809,12 @@ static int ll_merge_md_attr(struct inode *inode) return 0; } -static int ll_inode_revalidate(struct dentry *dentry, u64 ibits) +static int ll_inode_revalidate(struct dentry *dentry, enum ldlm_intent_flags op) { struct inode *inode = d_inode(dentry); int rc; - rc = __ll_inode_revalidate(dentry, ibits); + rc = __ll_inode_revalidate(dentry, op); if (rc != 0) return rc; @@ -3888,8 +3854,7 @@ int ll_getattr(const struct path *path, struct kstat *stat, struct ll_inode_info *lli = ll_i2info(inode); int res; - res = ll_inode_revalidate(path->dentry, - MDS_INODELOCK_UPDATE | MDS_INODELOCK_LOOKUP); + res = ll_inode_revalidate(path->dentry, IT_GETATTR); ll_stats_ops_tally(sbi, LPROC_LL_GETATTR, 1); if (res) @@ -3983,8 +3948,7 @@ int ll_inode_permission(struct inode *inode, int mask) */ if (is_root_inode(inode)) { - rc = __ll_inode_revalidate(inode->i_sb->s_root, - MDS_INODELOCK_LOOKUP); + rc = __ll_inode_revalidate(inode->i_sb->s_root, IT_LOOKUP); if (rc) return rc; } diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index 078ad6c..15b0710 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -810,6 +810,7 @@ struct ptlrpc_body_v2 { /* Features required for this version of the client to work with server */ #define CLIENT_CONNECT_MDT_REQD (OBD_CONNECT_FID | \ + OBD_CONNECT_ATTRFID | \ OBD_CONNECT_FULL20) /* This structure is used for both request and reply. From patchwork Mon Sep 30 18:55:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167183 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1400E912 for ; Mon, 30 Sep 2019 19:02:08 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EFEBD224EF for ; Mon, 30 Sep 2019 19:02:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EFEBD224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id D862A5E4377; Mon, 30 Sep 2019 11:59:14 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id A529C5C33AB for ; Mon, 30 Sep 2019 11:57:24 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 3155910058F7; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 2FC4EBD; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:41 -0400 Message-Id: <1569869810-23848-83-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 082/151] lustre: llite: simplify ll_inode_revalidate() X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: "John L. Hammond" ll_inode_revalidate() is only called by ll_getattr() so move the contents of ll_inode_revalidate() to ll_getattr() and rename __ll_inode_revalidate() to ll_inode_revalidate(). WC-bug-id: https://jira.whamcloud.com/browse/LU-8402 Lustre-commit: a430c3376c6b ("LU-8402 llite: simplify ll_inode_revalidate()") Signed-off-by: John L. Hammond Reviewed-on: https://review.whamcloud.com/30036 Reviewed-by: Dmitry Eremin Reviewed-by: Andreas Dilger Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/file.c | 75 +++++++++++++++++++++----------------------------- 1 file changed, 31 insertions(+), 44 deletions(-) diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 7f04e4f..47e270d 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -3733,26 +3733,23 @@ static int ll_inode_revalidate_fini(struct inode *inode, int rc) return rc; } -static int __ll_inode_revalidate(struct dentry *dentry, - enum ldlm_intent_flags op) +static int ll_inode_revalidate(struct dentry *dentry, enum ldlm_intent_flags op) { struct inode *inode = d_inode(dentry); + struct obd_export *exp = ll_i2mdexp(inode); struct lookup_intent oit = { .it_op = op, }; struct ptlrpc_request *req = NULL; struct md_op_data *op_data; - struct obd_export *exp; int rc = 0; CDEBUG(D_VFSTRACE, "VFS Op:inode=" DFID "(%p),name=%pd\n", PFID(ll_inode2fid(inode)), inode, dentry); - exp = ll_i2mdexp(inode); - /* Call getattr by fid, so do not provide name at all. */ - op_data = ll_prep_md_op_data(NULL, d_inode(dentry), d_inode(dentry), - NULL, 0, 0, LUSTRE_OPC_ANY, NULL); + op_data = ll_prep_md_op_data(NULL, inode, inode, NULL, 0, 0, + LUSTRE_OPC_ANY, NULL); if (IS_ERR(op_data)) return PTR_ERR(op_data); @@ -3809,30 +3806,21 @@ static int ll_merge_md_attr(struct inode *inode) return 0; } -static int ll_inode_revalidate(struct dentry *dentry, enum ldlm_intent_flags op) +int ll_getattr(const struct path *path, struct kstat *stat, + u32 request_mask, unsigned int flags) { - struct inode *inode = d_inode(dentry); + struct inode *inode = d_inode(path->dentry); + struct ll_sb_info *sbi = ll_i2sbi(inode); + struct ll_inode_info *lli = ll_i2info(inode); int rc; - rc = __ll_inode_revalidate(dentry, op); - if (rc != 0) - return rc; - - /* if object isn't regular file, don't validate size */ - if (!S_ISREG(inode->i_mode)) { - if (S_ISDIR(inode->i_mode) && - ll_i2info(inode)->lli_lsm_md) { - rc = ll_merge_md_attr(inode); - if (rc) - return rc; - } + ll_stats_ops_tally(sbi, LPROC_LL_GETATTR, 1); - inode->i_atime.tv_sec = ll_i2info(inode)->lli_atime; - inode->i_mtime.tv_sec = ll_i2info(inode)->lli_mtime; - inode->i_ctime.tv_sec = ll_i2info(inode)->lli_ctime; - } else { - struct ll_inode_info *lli = ll_i2info(inode); + rc = ll_inode_revalidate(path->dentry, IT_GETATTR); + if (rc < 0) + return rc; + if (S_ISREG(inode->i_mode)) { /* In case of restore, the MDT has the right size and has * already send it back without granting the layout lock, * inode is up-to-date so glimpse is useless. @@ -3840,25 +3828,24 @@ static int ll_inode_revalidate(struct dentry *dentry, enum ldlm_intent_flags op) * restore the MDT holds the layout lock so the glimpse will * block up to the end of restore (getattr will block) */ - if (!test_bit(LLIF_FILE_RESTORING, &lli->lli_flags)) + if (!test_bit(LLIF_FILE_RESTORING, &lli->lli_flags)) { rc = ll_glimpse_size(inode); - } - return rc; -} - -int ll_getattr(const struct path *path, struct kstat *stat, - u32 request_mask, unsigned int flags) -{ - struct inode *inode = d_inode(path->dentry); - struct ll_sb_info *sbi = ll_i2sbi(inode); - struct ll_inode_info *lli = ll_i2info(inode); - int res; - - res = ll_inode_revalidate(path->dentry, IT_GETATTR); - ll_stats_ops_tally(sbi, LPROC_LL_GETATTR, 1); + if (rc < 0) + return rc; + } + } else { + /* If object isn't regular a file then don't validate size. */ + if (S_ISDIR(inode->i_mode) && + lli->lli_lsm_md != NULL) { + rc = ll_merge_md_attr(inode); + if (rc < 0) + return rc; + } - if (res) - return res; + inode->i_atime.tv_sec = lli->lli_atime; + inode->i_mtime.tv_sec = lli->lli_mtime; + inode->i_ctime.tv_sec = lli->lli_ctime; + } OBD_FAIL_TIMEOUT(OBD_FAIL_GETATTR_DELAY, 30); @@ -3948,7 +3935,7 @@ int ll_inode_permission(struct inode *inode, int mask) */ if (is_root_inode(inode)) { - rc = __ll_inode_revalidate(inode->i_sb->s_root, IT_LOOKUP); + rc = ll_inode_revalidate(inode->i_sb->s_root, IT_LOOKUP); if (rc) return rc; } From patchwork Mon Sep 30 18:55:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167193 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0CF98912 for ; Mon, 30 Sep 2019 19:02:40 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E9B3D224F0 for ; Mon, 30 Sep 2019 19:02:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E9B3D224F0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 343C75E44C7; Mon, 30 Sep 2019 11:59:25 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 06C0D5C3B47 for ; Mon, 30 Sep 2019 11:57:25 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 355D010058F8; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 330FEB5; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:42 -0400 Message-Id: <1569869810-23848-84-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 083/151] lustre: obd: free obd_svc_stats when all users are gone X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Bruno Faccini During object device shutdown obd_svc_stats must only be freed after all access methods from user-land are no longer possible to prevent any race and further crash. WC-bug-id: https://jira.whamcloud.com/browse/LU-10224 Lustre-commit: ffc843a0aacd (LU-10224 obd: free obd_svc_stats when all users are gone") Signed-off-by: Bruno Faccini Reviewed-on: https://review.whamcloud.com/30249 Reviewed-by: James Simmons Reviewed-by: Andreas Dilger Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/mdc/mdc_request.c | 1 - fs/lustre/osc/osc_request.c | 1 - fs/lustre/ptlrpc/lproc_ptlrpc.c | 4 ++++ 3 files changed, 4 insertions(+), 2 deletions(-) diff --git a/fs/lustre/mdc/mdc_request.c b/fs/lustre/mdc/mdc_request.c index 9bae3a5..1a0f05a 100644 --- a/fs/lustre/mdc/mdc_request.c +++ b/fs/lustre/mdc/mdc_request.c @@ -2584,7 +2584,6 @@ static int mdc_precleanup(struct obd_device *obd) obd_cleanup_client_import(obd); ptlrpc_lprocfs_unregister_obd(obd); - lprocfs_obd_cleanup(obd); ldebugfs_free_md_stats(obd); mdc_llog_finish(obd); return 0; diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index 5581f42..2dce087 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -3070,7 +3070,6 @@ static int osc_precleanup(struct obd_device *obd) osc_precleanup_common(obd); ptlrpc_lprocfs_unregister_obd(obd); - lprocfs_obd_cleanup(obd); return 0; } diff --git a/fs/lustre/ptlrpc/lproc_ptlrpc.c b/fs/lustre/ptlrpc/lproc_ptlrpc.c index 6ce4d9e..02713076 100644 --- a/fs/lustre/ptlrpc/lproc_ptlrpc.c +++ b/fs/lustre/ptlrpc/lproc_ptlrpc.c @@ -1172,6 +1172,10 @@ void ptlrpc_lprocfs_unregister_service(struct ptlrpc_service *svc) void ptlrpc_lprocfs_unregister_obd(struct obd_device *obd) { + /* cleanup first to allow concurrent access to device's + * stats via debugfs to complete safely + */ + lprocfs_obd_cleanup(obd); debugfs_remove_recursive(obd->obd_svc_debugfs_entry); if (obd->obd_svc_stats) From patchwork Mon Sep 30 18:55:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167231 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B31D213B1 for ; Mon, 30 Sep 2019 19:04:31 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9B1F8224EF for ; Mon, 30 Sep 2019 19:04:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9B1F8224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 3C3EE5E4785; Mon, 30 Sep 2019 12:00:03 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 4C3F35C33AB for ; Mon, 30 Sep 2019 11:57:25 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 376B410058FA; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 35FC5BB; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:43 -0400 Message-Id: <1569869810-23848-85-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 084/151] lustre: mdc: add uid/gid to Changelogs entries X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Sebastien Buisson Add a new changelog extension named changelog_ext_uidgid to hold uid/gid information. uid/gid info is added to every Changelog entry type except MARK, in the form 'u=:': 5 01CREAT 15:44:32.385864793 2017.07.18 0x0 t=[0x200000402:0x3:0x0] ef=0x1 u=500:500 p=[0x200000402:0x2:0x0] file1 WC-bug-id: https://jira.whamcloud.com/browse/LU-9727 Lustre-commit: 361edea47072 ("LU-9727 mdc: add uid/gid to Changelogs entries") Signed-off-by: Sebastien Buisson Reviewed-on: https://review.whamcloud.com/28114 Reviewed-by: Andreas Dilger Reviewed-by: Henri Doreau Reviewed-by: Olaf Weber Signed-off-by: James Simmons --- fs/lustre/mdc/mdc_changelog.c | 3 +- include/uapi/linux/lustre/lustre_idl.h | 4 ++- include/uapi/linux/lustre/lustre_user.h | 51 +++++++++++++++++++++++++++++++-- 3 files changed, 53 insertions(+), 5 deletions(-) diff --git a/fs/lustre/mdc/mdc_changelog.c b/fs/lustre/mdc/mdc_changelog.c index e68ccb9..b6a26db 100644 --- a/fs/lustre/mdc/mdc_changelog.c +++ b/fs/lustre/mdc/mdc_changelog.c @@ -234,7 +234,8 @@ static int chlg_load(void *args) rc = llog_init_handle(NULL, llh, LLOG_F_IS_CAT | LLOG_F_EXT_JOBID | - LLOG_F_EXT_EXTRA_FLAGS, + LLOG_F_EXT_EXTRA_FLAGS | + LLOG_F_EXT_X_UIDGID, NULL); if (rc) { CERROR("%s: fail to init llog handle: rc = %d\n", diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index 15b0710..2c729ad 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -2579,6 +2579,7 @@ enum llog_flag { LLOG_F_EXT_JOBID = 0x8, LLOG_F_IS_FIXSIZE = 0x10, LLOG_F_EXT_EXTRA_FLAGS = 0x20, + LLOG_F_EXT_X_UIDGID = 0x40, /* * Note: Flags covered by LLOG_F_EXT_MASK will be inherited from @@ -2586,7 +2587,8 @@ enum llog_flag { * because the catlog record is usually fixed size, but its plain * log record can be variable */ - LLOG_F_EXT_MASK = LLOG_F_EXT_JOBID | LLOG_F_EXT_EXTRA_FLAGS, + LLOG_F_EXT_MASK = LLOG_F_EXT_JOBID | LLOG_F_EXT_EXTRA_FLAGS | + LLOG_F_EXT_X_UIDGID, }; /* On-disk header structure of each log object, stored in little endian order */ diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index 8e82359..ad50f95 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -973,8 +973,9 @@ static inline void hsm_set_cl_error(int *flags, int error) } enum changelog_rec_extra_flags { - CLFE_INVALID = 0, /* No additional flags currently implemented */ - CLFE_SUPPORTED = CLFE_INVALID + CLFE_INVALID = 0, + CLFE_UIDGID = 0x0001, + CLFE_SUPPORTED = CLFE_UIDGID }; enum changelog_send_flag { @@ -992,6 +993,11 @@ enum changelog_send_flag { CHANGELOG_FLAG_EXTRA_FLAGS = 0x08, }; +enum changelog_send_extra_flag { + /* Pack uid/gid into the changelog record */ + CHANGELOG_EXTRA_FLAG_UIDGID = 0x01, +}; + #define CR_MAXSIZE __ALIGN_KERNEL(2 * NAME_MAX + 2 + \ changelog_rec_offset(CLF_SUPPORTED, \ CLFE_SUPPORTED), 8) @@ -1037,6 +1043,12 @@ struct changelog_ext_extra_flags { __u64 cr_extra_flags; /* Additional CLFE_* flags */ }; +/* Changelog extra extension to include UID/GID. */ +struct changelog_ext_uidgid { + __u64 cr_uid; + __u64 cr_gid; +}; + static inline struct changelog_ext_extra_flags *changelog_rec_extra_flags( const struct changelog_rec *rec); @@ -1051,8 +1063,11 @@ static inline size_t changelog_rec_offset(enum changelog_rec_flags crf, if (crf & CLF_JOBID) size += sizeof(struct changelog_ext_jobid); - if (crf & CLF_EXTRA_FLAGS) + if (crf & CLF_EXTRA_FLAGS) { size += sizeof(struct changelog_ext_extra_flags); + if (cref & CLFE_UIDGID) + size += sizeof(struct changelog_ext_uidgid); + } return size; } @@ -1107,6 +1122,19 @@ struct changelog_ext_extra_flags *changelog_rec_extra_flags( CLFE_INVALID)); } +/* The uid/gid is the first extra extension */ +static inline +struct changelog_ext_uidgid *changelog_rec_uidgid( + const struct changelog_rec *rec) +{ + enum changelog_rec_flags crf = rec->cr_flags & + (CLF_VERSION | CLF_RENAME | CLF_JOBID | CLF_EXTRA_FLAGS); + + return (struct changelog_ext_uidgid *)((char *)rec + + changelog_rec_offset(crf, + CLFE_INVALID)); +} + /* The name follows the rename, jobid and extra flags extns, if present */ static inline char *changelog_rec_name(struct changelog_rec *rec) { @@ -1156,8 +1184,10 @@ static inline void changelog_remap_rec(struct changelog_rec *rec, enum changelog_rec_flags crf_wanted, enum changelog_rec_extra_flags cref_want) { + char *uidgid_mov = NULL; char *ef_mov; char *jid_mov, *rnm_mov; + enum changelog_rec_extra_flags cref = CLFE_INVALID; crf_wanted &= CLF_SUPPORTED; cref_want &= CLFE_SUPPORTED; @@ -1176,6 +1206,13 @@ static inline void changelog_remap_rec(struct changelog_rec *rec, changelog_rec_name(rec), rec->cr_namelen); /* Locations of extensions in the remapped record */ + if (rec->cr_flags & CLF_EXTRA_FLAGS) { + uidgid_mov = (char *)rec + + changelog_rec_offset(crf_wanted & CLF_SUPPORTED, + CLFE_INVALID); + cref = changelog_rec_extra_flags(rec)->cr_extra_flags; + } + ef_mov = (char *)rec + changelog_rec_offset(crf_wanted & ~CLF_EXTRA_FLAGS, CLFE_INVALID); @@ -1193,6 +1230,10 @@ static inline void changelog_remap_rec(struct changelog_rec *rec, /* Move the extension fields to the desired positions */ if ((crf_wanted & CLF_EXTRA_FLAGS) && (rec->cr_flags & CLF_EXTRA_FLAGS)) { + if ((cref_want & CLFE_UIDGID) && (cref & CLFE_UIDGID)) + memmove(uidgid_mov, changelog_rec_uidgid(rec), + sizeof(struct changelog_ext_uidgid)); + memmove(ef_mov, changelog_rec_extra_flags(rec), sizeof(struct changelog_ext_extra_flags)); } @@ -1206,6 +1247,10 @@ static inline void changelog_remap_rec(struct changelog_rec *rec, sizeof(struct changelog_ext_rename)); /* Clear newly added fields */ + if (uidgid_mov && (cref_want & CLFE_UIDGID) && + !(cref & CLFE_UIDGID)) + memset(uidgid_mov, 0, sizeof(struct changelog_ext_uidgid)); + if ((crf_wanted & CLF_EXTRA_FLAGS) && !(rec->cr_flags & CLF_EXTRA_FLAGS)) memset(ef_mov, 0, sizeof(struct changelog_ext_extra_flags)); From patchwork Mon Sep 30 18:55:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167233 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 694DA16C1 for ; Mon, 30 Sep 2019 19:04:38 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 517BD224EF for ; Mon, 30 Sep 2019 19:04:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 517BD224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8483F5E47CA; Mon, 30 Sep 2019 12:00:06 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id A1B235C33AB for ; Mon, 30 Sep 2019 11:57:25 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 3A80F10058FB; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 39099A9; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:44 -0400 Message-Id: <1569869810-23848-86-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 085/151] lustre: scrub: general framework for OI scrub X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fan Yong , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Fan Yong Reconstruct OI scrub code to make some data structure and functions can be shared between osd-ldiskfs and osd-ZFS. This cleanup is mainly server side but it does lead to a enum removal since in the linux client. WC-bug-id: https://jira.whamcloud.com/browse/LU-7585 Lustre-commit: 7e21ea59edcc ("LU-7585 scrub: general framework for OI scrub") Signed-off-by: Fan Yong Reviewed-on: https://review.whamcloud.com/28607 Reviewed-by: Andreas Dilger Reviewed-by: Alex Zhuravlev Signed-off-by: James Simmons --- fs/lustre/include/lustre_fid.h | 7 ------- 1 file changed, 7 deletions(-) diff --git a/fs/lustre/include/lustre_fid.h b/fs/lustre/include/lustre_fid.h index 5108864..b469295 100644 --- a/fs/lustre/include/lustre_fid.h +++ b/fs/lustre/include/lustre_fid.h @@ -195,13 +195,6 @@ enum { LUSTRE_SEQ_SUPER_WIDTH = ((1ULL << 30ULL) * LUSTRE_SEQ_META_WIDTH) }; -enum { - /** 2^6 FIDs for OI containers */ - OSD_OI_FID_OID_BITS = 6, - /** reserve enough FIDs in case we want more in the future */ - OSD_OI_FID_OID_BITS_MAX = 10, -}; - /** special OID for local objects */ enum local_oid { /** \see fld_mod_init */ From patchwork Mon Sep 30 18:55:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167195 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9CD4F912 for ; Mon, 30 Sep 2019 19:02:50 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 83E84224EF for ; Mon, 30 Sep 2019 19:02:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 83E84224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 11C3121EB5F; Mon, 30 Sep 2019 11:59:29 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id E31855C33AB for ; Mon, 30 Sep 2019 11:57:25 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 3E26A10058FC; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 3C14FB4; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:45 -0400 Message-Id: <1569869810-23848-87-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 086/151] lustre: idl: clean up and document ptlrpc structures X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Andreas Dilger Miscellaneous cleanups for wire-related structures found during the Lustre protocol documentation project. Remove obsolete and unused since 2.3 OBD_QC_CALLBACK RPC. Move definition of constants used by lustre_msg_v2 and ptlrpc_body fields to immediately before their respective structures. Add comments for struct members for lustre_msg_v2, ptlrpc_body_v3, ldlm_request, and obd_statfs. Rename RQF_MDS_INTENT_CLOSE to RQF_MDS_CLOSE_INTENT to make it more easily found with RQF_MDS_CLOSE. Rename mdt_intent_close_client() and mdc_intent_close_pack() to mdt_close_intent_client() and mdc_close_intent_pack() to match. Rename RQF_LDLM_GL_DESC_CALLBACK to RQF_LDLM_GL_CALLBACK_DESC to make it match RQF_LDLM_FL_CALLBACK and RQF_LDLM_BL_CALLBACK. Remove unused MSG_OP_FLAG_MASK, MSG_OP_FLAG_SHIFT, MSG_GEN_FLAG_MASK. WC-bug-id: https://jira.whamcloud.com/browse/LU-6349 Lustre-commit: e8bdd1a48dd8 ("LU-6349 idl: clean up and document ptlrpc structures") Signed-off-by: Andreas Dilger Reviewed-on: https://review.whamcloud.com/17830 Reviewed-by: Ben Evans Reviewed-by: Steve Guminski Reviewed-by: James Simmons Reviewed-by: John L. Hammond Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_req_layout.h | 4 +- fs/lustre/include/obd_class.h | 2 +- fs/lustre/mdc/mdc_lib.c | 6 +- fs/lustre/mdc/mdc_request.c | 2 +- fs/lustre/mgc/mgc_request.c | 2 +- fs/lustre/ptlrpc/layout.c | 18 ++-- fs/lustre/ptlrpc/pack_generic.c | 60 +++++------ fs/lustre/ptlrpc/wiretest.c | 6 -- include/uapi/linux/lustre/lustre_idl.h | 176 ++++++++++++++++---------------- include/uapi/linux/lustre/lustre_user.h | 27 ++--- 10 files changed, 149 insertions(+), 154 deletions(-) diff --git a/fs/lustre/include/lustre_req_layout.h b/fs/lustre/include/lustre_req_layout.h index 3d86883..48f454c 100644 --- a/fs/lustre/include/lustre_req_layout.h +++ b/fs/lustre/include/lustre_req_layout.h @@ -143,7 +143,7 @@ void req_capsule_shrink(struct req_capsule *pill, */ extern struct req_format RQF_MDS_GETATTR_NAME; extern struct req_format RQF_MDS_CLOSE; -extern struct req_format RQF_MDS_INTENT_CLOSE; +extern struct req_format RQF_MDS_CLOSE_INTENT; extern struct req_format RQF_MDS_CONNECT; extern struct req_format RQF_MDS_DISCONNECT; extern struct req_format RQF_MDS_GET_INFO; @@ -210,7 +210,7 @@ void req_capsule_shrink(struct req_capsule *pill, extern struct req_format RQF_LDLM_CP_CALLBACK; extern struct req_format RQF_LDLM_BL_CALLBACK; extern struct req_format RQF_LDLM_GL_CALLBACK; -extern struct req_format RQF_LDLM_GL_DESC_CALLBACK; +extern struct req_format RQF_LDLM_GL_CALLBACK_DESC; /* LOG req_format */ extern struct req_format RQF_LOG_CANCEL; extern struct req_format RQF_LLOG_ORIGIN_HANDLE_CREATE; diff --git a/fs/lustre/include/obd_class.h b/fs/lustre/include/obd_class.h index a939f17..38b69e8 100644 --- a/fs/lustre/include/obd_class.h +++ b/fs/lustre/include/obd_class.h @@ -183,7 +183,7 @@ int class_config_parse_llog(const struct lu_env *env, struct llog_ctxt *ctxt, struct config_llog_data { struct ldlm_res_id cld_resid; struct config_llog_instance cld_cfg; - struct list_head cld_list_chain; + struct list_head cld_list_chain; /* on config_llog_list */ atomic_t cld_refcount; struct config_llog_data *cld_sptlrpc; /* depended sptlrpc log */ struct config_llog_data *cld_params; /* common parameters log */ diff --git a/fs/lustre/mdc/mdc_lib.c b/fs/lustre/mdc/mdc_lib.c index ff14f82..df66ed1 100644 --- a/fs/lustre/mdc/mdc_lib.c +++ b/fs/lustre/mdc/mdc_lib.c @@ -428,7 +428,7 @@ void mdc_link_pack(struct ptlrpc_request *req, struct md_op_data *op_data) mdc_pack_name(req, &RMF_NAME, op_data->op_name, op_data->op_namelen); } -static void mdc_intent_close_pack(struct ptlrpc_request *req, +static void mdc_close_intent_pack(struct ptlrpc_request *req, struct md_op_data *op_data) { enum mds_op_bias bias = op_data->op_bias; @@ -501,7 +501,7 @@ void mdc_rename_pack(struct ptlrpc_request *req, struct md_op_data *op_data, op_data->op_bias & MDS_RENAME_MIGRATE) { struct mdt_ioepoch *epoch; - mdc_intent_close_pack(req, op_data); + mdc_close_intent_pack(req, op_data); epoch = req_capsule_client_get(&req->rq_pill, &RMF_MDT_EPOCH); mdc_ioepoch_pack(epoch, op_data); } @@ -553,5 +553,5 @@ void mdc_close_pack(struct ptlrpc_request *req, struct md_op_data *op_data) rec->sa_valid &= ~MDS_ATTR_ATIME; mdc_ioepoch_pack(epoch, op_data); - mdc_intent_close_pack(req, op_data); + mdc_close_intent_pack(req, op_data); } diff --git a/fs/lustre/mdc/mdc_request.c b/fs/lustre/mdc/mdc_request.c index 1a0f05a..efa6516 100644 --- a/fs/lustre/mdc/mdc_request.c +++ b/fs/lustre/mdc/mdc_request.c @@ -771,7 +771,7 @@ static int mdc_close(struct obd_export *exp, struct md_op_data *op_data, op_data->op_bias); if (op_data->op_bias & MDS_CLOSE_INTENT) { - req_fmt = &RQF_MDS_INTENT_CLOSE; + req_fmt = &RQF_MDS_CLOSE_INTENT; if (op_data->op_bias & MDS_HSM_RELEASE) { /* allocate a FID for volatile file */ rc = mdc_fid_alloc(NULL, exp, &op_data->op_fid2, diff --git a/fs/lustre/mgc/mgc_request.c b/fs/lustre/mgc/mgc_request.c index 581ffb0..ca7f47a 100644 --- a/fs/lustre/mgc/mgc_request.c +++ b/fs/lustre/mgc/mgc_request.c @@ -116,7 +116,7 @@ static int mgc_logname2resid(char *logname, struct ldlm_res_id *res_id, /********************** config llog list **********************/ static LIST_HEAD(config_llog_list); -static DEFINE_SPINLOCK(config_list_lock); +static DEFINE_SPINLOCK(config_list_lock); /* protects config_llog_list */ /* Take a reference to a config log */ static int config_log_get(struct config_llog_data *cld) diff --git a/fs/lustre/ptlrpc/layout.c b/fs/lustre/ptlrpc/layout.c index ce1de5e..0978329 100644 --- a/fs/lustre/ptlrpc/layout.c +++ b/fs/lustre/ptlrpc/layout.c @@ -116,7 +116,7 @@ &RMF_CAPA1 }; -static const struct req_msg_field *mdt_intent_close_client[] = { +static const struct req_msg_field *mdt_close_intent_client[] = { &RMF_PTLRPC_BODY, &RMF_MDT_EPOCH, &RMF_REC_REINT, @@ -705,7 +705,7 @@ &RQF_MDS_GETXATTR, &RQF_MDS_SYNC, &RQF_MDS_CLOSE, - &RQF_MDS_INTENT_CLOSE, + &RQF_MDS_CLOSE_INTENT, &RQF_MDS_READPAGE, &RQF_MDS_WRITEPAGE, &RQF_MDS_REINT, @@ -757,7 +757,7 @@ &RQF_LDLM_CP_CALLBACK, &RQF_LDLM_BL_CALLBACK, &RQF_LDLM_GL_CALLBACK, - &RQF_LDLM_GL_DESC_CALLBACK, + &RQF_LDLM_GL_CALLBACK_DESC, &RQF_LDLM_INTENT, &RQF_LDLM_INTENT_BASIC, &RQF_LDLM_INTENT_LAYOUT, @@ -1405,10 +1405,10 @@ struct req_format RQF_LDLM_GL_CALLBACK = ldlm_gl_callback_server); EXPORT_SYMBOL(RQF_LDLM_GL_CALLBACK); -struct req_format RQF_LDLM_GL_DESC_CALLBACK = +struct req_format RQF_LDLM_GL_CALLBACK_DESC = DEFINE_REQ_FMT0("LDLM_GL_CALLBACK", ldlm_gl_callback_desc_client, ldlm_gl_callback_server); -EXPORT_SYMBOL(RQF_LDLM_GL_DESC_CALLBACK); +EXPORT_SYMBOL(RQF_LDLM_GL_CALLBACK_DESC); struct req_format RQF_LDLM_INTENT_BASIC = DEFINE_REQ_FMT0("LDLM_INTENT_BASIC", @@ -1421,7 +1421,7 @@ struct req_format RQF_LDLM_INTENT = EXPORT_SYMBOL(RQF_LDLM_INTENT); struct req_format RQF_LDLM_INTENT_LAYOUT = - DEFINE_REQ_FMT0("LDLM_INTENT_LAYOUT ", + DEFINE_REQ_FMT0("LDLM_INTENT_LAYOUT", ldlm_intent_layout_client, ldlm_enqueue_lvb_server); EXPORT_SYMBOL(RQF_LDLM_INTENT_LAYOUT); @@ -1456,10 +1456,10 @@ struct req_format RQF_MDS_CLOSE = mdt_close_client, mds_last_unlink_server); EXPORT_SYMBOL(RQF_MDS_CLOSE); -struct req_format RQF_MDS_INTENT_CLOSE = +struct req_format RQF_MDS_CLOSE_INTENT = DEFINE_REQ_FMT0("MDS_CLOSE", - mdt_intent_close_client, mds_last_unlink_server); -EXPORT_SYMBOL(RQF_MDS_INTENT_CLOSE); + mdt_close_intent_client, mds_last_unlink_server); +EXPORT_SYMBOL(RQF_MDS_CLOSE_INTENT); struct req_format RQF_MDS_READPAGE = DEFINE_REQ_FMT0("MDS_READPAGE", diff --git a/fs/lustre/ptlrpc/pack_generic.c b/fs/lustre/ptlrpc/pack_generic.c index e7662be..7fa2cf7 100644 --- a/fs/lustre/ptlrpc/pack_generic.c +++ b/fs/lustre/ptlrpc/pack_generic.c @@ -832,7 +832,7 @@ void lustre_msg_clear_flags(struct lustre_msg *msg, u32 flags) struct ptlrpc_body *pb = lustre_msg_ptlrpc_body(msg); LASSERTF(pb, "invalid msg %p: no ptlrpc body!\n", msg); - pb->pb_flags &= ~(flags & MSG_GEN_FLAG_MASK); + pb->pb_flags &= ~flags; return; } default: @@ -1504,40 +1504,41 @@ int do_set_info_async(struct obd_import *imp, /* byte flipping routines for all wire types declared in * lustre_idl.h implemented here. */ -void lustre_swab_ptlrpc_body(struct ptlrpc_body *b) -{ - __swab32s(&b->pb_type); - __swab32s(&b->pb_version); - __swab32s(&b->pb_opc); - __swab32s(&b->pb_status); - __swab64s(&b->pb_last_xid); - __swab16s(&b->pb_tag); - __swab64s(&b->pb_last_committed); - __swab64s(&b->pb_transno); - __swab32s(&b->pb_flags); - __swab32s(&b->pb_op_flags); - __swab32s(&b->pb_conn_cnt); - __swab32s(&b->pb_timeout); - __swab32s(&b->pb_service_time); - __swab32s(&b->pb_limit); - __swab64s(&b->pb_slv); - __swab64s(&b->pb_pre_versions[0]); - __swab64s(&b->pb_pre_versions[1]); - __swab64s(&b->pb_pre_versions[2]); - __swab64s(&b->pb_pre_versions[3]); - __swab64s(&b->pb_mbits); - BUILD_BUG_ON(offsetof(typeof(*b), pb_padding0) == 0); - BUILD_BUG_ON(offsetof(typeof(*b), pb_padding1) == 0); - BUILD_BUG_ON(offsetof(typeof(*b), pb_padding64_0) == 0); - BUILD_BUG_ON(offsetof(typeof(*b), pb_padding64_1) == 0); - BUILD_BUG_ON(offsetof(typeof(*b), pb_padding64_2) == 0); +void lustre_swab_ptlrpc_body(struct ptlrpc_body *body) +{ + __swab32s(&body->pb_type); + __swab32s(&body->pb_version); + __swab32s(&body->pb_opc); + __swab32s(&body->pb_status); + __swab64s(&body->pb_last_xid); + __swab16s(&body->pb_tag); + BUILD_BUG_ON(offsetof(typeof(*body), pb_padding0) == 0); + BUILD_BUG_ON(offsetof(typeof(*body), pb_padding1) == 0); + __swab64s(&body->pb_last_committed); + __swab64s(&body->pb_transno); + __swab32s(&body->pb_flags); + __swab32s(&body->pb_op_flags); + __swab32s(&body->pb_conn_cnt); + __swab32s(&body->pb_timeout); + __swab32s(&body->pb_service_time); + __swab32s(&body->pb_limit); + __swab64s(&body->pb_slv); + __swab64s(&body->pb_pre_versions[0]); + __swab64s(&body->pb_pre_versions[1]); + __swab64s(&body->pb_pre_versions[2]); + __swab64s(&body->pb_pre_versions[3]); + __swab64s(&body->pb_mbits); + BUILD_BUG_ON(offsetof(typeof(*body), pb_padding64_0) == 0); + BUILD_BUG_ON(offsetof(typeof(*body), pb_padding64_1) == 0); + BUILD_BUG_ON(offsetof(typeof(*body), pb_padding64_2) == 0); /* While we need to maintain compatibility between * clients and servers without ptlrpc_body_v2 (< 2.3) * do not swab any fields beyond pb_jobid, as we are * using this swab function for both ptlrpc_body * and ptlrpc_body_v2. */ - BUILD_BUG_ON(offsetof(typeof(*b), pb_jobid) == 0); + /* pb_jobid is an ASCII string and should not be swabbed */ + BUILD_BUG_ON(offsetof(typeof(*body), pb_jobid) == 0); } void lustre_swab_connect(struct obd_connect_data *ocd) @@ -1642,6 +1643,7 @@ void lustre_swab_obd_statfs(struct obd_statfs *os) __swab32s(&os->os_namelen); __swab64s(&os->os_maxbytes); __swab32s(&os->os_state); + __swab32s(&os->os_fprecreated); BUILD_BUG_ON(offsetof(typeof(*os), os_fprecreated) == 0); BUILD_BUG_ON(offsetof(typeof(*os), os_spare2) == 0); BUILD_BUG_ON(offsetof(typeof(*os), os_spare3) == 0); diff --git a/fs/lustre/ptlrpc/wiretest.c b/fs/lustre/ptlrpc/wiretest.c index ff3c79a..76fb656 100644 --- a/fs/lustre/ptlrpc/wiretest.c +++ b/fs/lustre/ptlrpc/wiretest.c @@ -827,12 +827,6 @@ void lustre_assert_wire_constants(void) (long long)MSGHDR_AT_SUPPORT); LASSERTF(MSGHDR_CKSUM_INCOMPAT18 == 2, "found %lld\n", (long long)MSGHDR_CKSUM_INCOMPAT18); - LASSERTF(MSG_OP_FLAG_MASK == 0xffff0000UL, "found 0x%.8xUL\n", - (unsigned int)MSG_OP_FLAG_MASK); - LASSERTF(MSG_OP_FLAG_SHIFT == 16, "found %lld\n", - (long long)MSG_OP_FLAG_SHIFT); - LASSERTF(MSG_GEN_FLAG_MASK == 0x0000ffffUL, "found 0x%.8xUL\n", - (unsigned int)MSG_GEN_FLAG_MASK); LASSERTF(MSG_LAST_REPLAY == 0x00000001UL, "found 0x%.8xUL\n", (unsigned int)MSG_LAST_REPLAY); LASSERTF(MSG_RESENT == 0x00000002UL, "found 0x%.8xUL\n", diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index 2c729ad..2a7f419 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -123,29 +123,8 @@ #define SEQ_CONTROLLER_PORTAL 32 #define MGS_BULK_PORTAL 33 -/* Portal 63 is reserved for the Cray Inc DVS - nic@cray.com, roe@cray.com, - * n8851@cray.com - */ - -/* packet types */ -#define PTL_RPC_MSG_REQUEST 4711 -#define PTL_RPC_MSG_ERR 4712 -#define PTL_RPC_MSG_REPLY 4713 - -/* DON'T use swabbed values of MAGIC as magic! */ -#define LUSTRE_MSG_MAGIC_V2 0x0BD00BD3 -#define LUSTRE_MSG_MAGIC_V2_SWABBED 0xD30BD00B - -#define LUSTRE_MSG_MAGIC LUSTRE_MSG_MAGIC_V2 - -#define PTLRPC_MSG_VERSION 0x00000003 -#define LUSTRE_VERSION_MASK 0xffff0000 -#define LUSTRE_OBD_VERSION 0x00010000 -#define LUSTRE_MDS_VERSION 0x00020000 -#define LUSTRE_OST_VERSION 0x00030000 -#define LUSTRE_DLM_VERSION 0x00040000 -#define LUSTRE_LOG_VERSION 0x00050000 -#define LUSTRE_MGS_VERSION 0x00060000 +/* #define DVS_PORTAL 63 */ +/* reserved for Cray DVS - spitzcor@cray.com, roe@cray.com, n8851@cray.com */ /** * Describes a range of sequence, lsr_start is included but lsr_end is @@ -550,55 +529,107 @@ static inline void lustre_handle_copy(struct lustre_handle *tgt, tgt->cookie = src->cookie; } +/* lustre_msg struct magic. DON'T use swabbed values of MAGIC as magic! */ +#define LUSTRE_MSG_MAGIC_V2 0x0BD00BD3 +#define LUSTRE_MSG_MAGIC_V2_SWABBED 0xD30BD00B +#define LUSTRE_MSG_MAGIC LUSTRE_MSG_MAGIC_V2 + /* flags for lm_flags */ -#define MSGHDR_AT_SUPPORT 0x1 -#define MSGHDR_CKSUM_INCOMPAT18 0x2 +#define MSGHDR_AT_SUPPORT 0x1 /* adaptive timeouts, lm_cksum valid + * in early reply messages + */ +#define MSGHDR_CKSUM_INCOMPAT18 0x2 /* compat for 1.8, needs to be set + * well beyond 2.8.0 for + * compatibility + */ #define lustre_msg lustre_msg_v2 /* we depend on this structure to be 8-byte aligned */ /* this type is only endian-adjusted in lustre_unpack_msg() */ struct lustre_msg_v2 { - __u32 lm_bufcount; - __u32 lm_secflvr; - __u32 lm_magic; - __u32 lm_repsize; - __u32 lm_cksum; - __u32 lm_flags; - __u32 lm_padding_2; - __u32 lm_padding_3; - __u32 lm_buflens[0]; + __u32 lm_bufcount; /* number of buffers in lm_buflens[] */ + __u32 lm_secflvr; /* 0 = no crypto, or sptlrpc security flavour */ + __u32 lm_magic; /* RPC version magic = LUSTRE_MSG_MAGIC_V2 */ + __u32 lm_repsize; /* size of preallocated reply buffer */ + __u32 lm_cksum; /* CRC32 of ptlrpc_body early reply messages */ + __u32 lm_flags; /* MSGHDR_* flags */ + __u32 lm_padding_2; /* unused */ + __u32 lm_padding_3; /* unused */ + __u32 lm_buflens[0]; /* length of additional buffers in bytes, + * padded to a multiple of 8 bytes. + */ + /* + * message buffers are packed after padded lm_buflens[] array, + * padded to a multiple of 8 bytes each to align contents. + */ }; -/* without gss, ptlrpc_body is put at the first buffer. */ -#define PTLRPC_NUM_VERSIONS 4 +/* ptlrpc_body packet pb_types */ +#define PTL_RPC_MSG_REQUEST 4711 /* normal RPC request message */ +#define PTL_RPC_MSG_ERR 4712 /* error reply if request unprocessed */ +#define PTL_RPC_MSG_REPLY 4713 /* normal RPC reply message */ +/* ptlrpc_body pb_version ((target_version << 16) | rpc_version) */ +#define PTLRPC_MSG_VERSION 0x00000003 +#define LUSTRE_VERSION_MASK 0xffff0000 +#define LUSTRE_OBD_VERSION 0x00010000 +#define LUSTRE_MDS_VERSION 0x00020000 +#define LUSTRE_OST_VERSION 0x00030000 +#define LUSTRE_DLM_VERSION 0x00040000 +#define LUSTRE_LOG_VERSION 0x00050000 +#define LUSTRE_MGS_VERSION 0x00060000 + +/* pb_flags that apply to all requests */ +#define MSG_LAST_REPLAY 0x0001 +#define MSG_RESENT 0x0002 +#define MSG_REPLAY 0x0004 +/* #define MSG_AT_SUPPORT 0x0008 obsolete 1.5 */ +#define MSG_DELAY_REPLAY 0x0010 +#define MSG_VERSION_REPLAY 0x0020 +#define MSG_REQ_REPLAY_DONE 0x0040 +#define MSG_LOCK_REPLAY_DONE 0x0080 + +/* pb_op_flags for all connect opcodes (MDS_CONNECT, OST_CONNECT) */ +#define MSG_CONNECT_RECOVERING 0x00000001 +#define MSG_CONNECT_RECONNECT 0x00000002 +#define MSG_CONNECT_REPLAYABLE 0x00000004 +/* #define MSG_CONNECT_PEER 0x00000008 removed 1.5 */ +#define MSG_CONNECT_LIBCLIENT 0x00000010 +#define MSG_CONNECT_INITIAL 0x00000020 +#define MSG_CONNECT_ASYNC 0x00000040 +#define MSG_CONNECT_NEXT_VER 0x00000080 /* use next version of lustre_msg */ +#define MSG_CONNECT_TRANSNO 0x00000100 /* report transno */ + +/* number of previous object versions in pb_pre_versions[] */ +#define PTLRPC_NUM_VERSIONS 4 +/* without gss, ptlrpc_body is put at the first buffer. */ struct ptlrpc_body_v3 { struct lustre_handle pb_handle; - __u32 pb_type; - __u32 pb_version; - __u32 pb_opc; - __u32 pb_status; + __u32 pb_type; /* request/reply/err type: PTL_RPC_MSG_* */ + __u32 pb_version; /* LUSTRE_*_VERSION | PTLRPC_MSG_VERSION */ + __u32 pb_opc; /* RPC opcodes: MDS_*, OST_*, LDLM_, ... */ + __u32 pb_status; /* negative Linux x86 error number */ __u64 pb_last_xid; /* highest replied XID without lower unreplied XID */ __u16 pb_tag; /* virtual slot idx for multiple modifying RPCs */ __u16 pb_padding0; __u32 pb_padding1; - __u64 pb_last_committed; - __u64 pb_transno; - __u32 pb_flags; - __u32 pb_op_flags; - __u32 pb_conn_cnt; - __u32 pb_timeout; /* for req, the deadline, for rep, the service est */ - __u32 pb_service_time; /* for rep, actual service time */ - __u32 pb_limit; - __u64 pb_slv; - /* VBR: pre-versions */ + __u64 pb_last_committed;/* rep: highest pb_transno committed to disk */ + __u64 pb_transno; /* server-assigned transno for modifying RPCs */ + __u32 pb_flags; /* req: MSG_* flags */ + __u32 pb_op_flags; /* req: MSG_CONNECT_* flags */ + __u32 pb_conn_cnt; /* connect instance of this client on server */ + __u32 pb_timeout; /* req: max wait time; rep: service estimate */ + __u32 pb_service_time; /* rep: server arrival to reply in seconds */ + __u32 pb_limit; /* rep: dynamic DLM LRU lock count limit */ + __u64 pb_slv; /* rep: dynamic DLM LRU server lock volume */ + /* VBR: rep: previous pb_version(s) of objects modified by this RPC */ __u64 pb_pre_versions[PTLRPC_NUM_VERSIONS]; __u64 pb_mbits; /**< match bits for bulk request */ - /* padding for future needs */ + /* padding for future needs - fix lustre_swab_ptlrpc_body() also */ __u64 pb_padding64_0; __u64 pb_padding64_1; __u64 pb_padding64_2; - char pb_jobid[LUSTRE_JOBID_SIZE]; + char pb_jobid[LUSTRE_JOBID_SIZE]; /* req: ASCII MPI jobid from env */ }; #define ptlrpc_body ptlrpc_body_v3 @@ -656,39 +687,6 @@ struct ptlrpc_body_v2 { /** only use in req->rq_{req,rep}_swab_mask */ #define MSG_PTLRPC_HEADER_OFF 31 -/* Flags that are operation-specific go in the top 16 bits. */ -#define MSG_OP_FLAG_MASK 0xffff0000 -#define MSG_OP_FLAG_SHIFT 16 - -/* Flags that apply to all requests are in the bottom 16 bits */ -#define MSG_GEN_FLAG_MASK 0x0000ffff -#define MSG_LAST_REPLAY 0x0001 -#define MSG_RESENT 0x0002 -#define MSG_REPLAY 0x0004 -/* #define MSG_AT_SUPPORT 0x0008 - * This was used in early prototypes of adaptive timeouts, and while there - * shouldn't be any users of that code there also isn't a need for using this - * bits. Defer usage until at least 1.10 to avoid potential conflict. - */ -#define MSG_DELAY_REPLAY 0x0010 -#define MSG_VERSION_REPLAY 0x0020 -#define MSG_REQ_REPLAY_DONE 0x0040 -#define MSG_LOCK_REPLAY_DONE 0x0080 - -/* - * Flags for all connect opcodes (MDS_CONNECT, OST_CONNECT) - */ - -#define MSG_CONNECT_RECOVERING 0x00000001 -#define MSG_CONNECT_RECONNECT 0x00000002 -#define MSG_CONNECT_REPLAYABLE 0x00000004 -/*#define MSG_CONNECT_PEER 0x8 */ -#define MSG_CONNECT_LIBCLIENT 0x00000010 -#define MSG_CONNECT_INITIAL 0x00000020 -#define MSG_CONNECT_ASYNC 0x00000040 -#define MSG_CONNECT_NEXT_VER 0x00000080 /* use next version of lustre_msg */ -#define MSG_CONNECT_TRANSNO 0x00000100 /* report transno */ - /* Connect flags */ #define OBD_CONNECT_RDONLY 0x1ULL /*client has read-only access*/ #define OBD_CONNECT_INDEX 0x2ULL /*connect specific LOV idx */ @@ -2225,9 +2223,9 @@ struct ldlm_lock_desc { #define LDLM_ENQUEUE_CANCEL_OFF 1 struct ldlm_request { - __u32 lock_flags; - __u32 lock_count; - struct ldlm_lock_desc lock_desc; + __u32 lock_flags; /* LDLM_FL_*, see lustre_dlm_flags.h */ + __u32 lock_count; /* number of locks in lock_handle[] */ + struct ldlm_lock_desc lock_desc;/* lock descriptor */ struct lustre_handle lock_handle[LDLM_LOCKREQ_HANDLES]; }; @@ -2273,7 +2271,7 @@ struct mgs_target_info { __u32 mti_lustre_ver; __u32 mti_stripe_index; __u32 mti_config_ver; - __u32 mti_flags; + __u32 mti_flags; /* LDD_F_* */ __u32 mti_nid_count; __u32 mti_instance; /* Running instance of target */ char mti_fsname[MTI_NAME_MAXLEN]; diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index ad50f95..e9bb3d6 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -107,21 +107,22 @@ enum obd_statfs_state { }; struct obd_statfs { - __u64 os_type; - __u64 os_blocks; - __u64 os_bfree; - __u64 os_bavail; - __u64 os_files; - __u64 os_ffree; - __u8 os_fsid[40]; - __u32 os_bsize; - __u32 os_namelen; - __u64 os_maxbytes; + __u64 os_type; /* EXT4_SUPER_MAGIC, UBERBLOCK_MAGIC */ + __u64 os_blocks; /* total size in #os_bsize blocks */ + __u64 os_bfree; /* number of unused blocks */ + __u64 os_bavail; /* blocks available for allocation */ + __u64 os_files; /* total number of objects */ + __u64 os_ffree; /* # objects that could be created */ + __u8 os_fsid[40]; /* identifier for filesystem */ + __u32 os_bsize; /* block size in bytes for os_blocks */ + __u32 os_namelen; /* maximum length of filename in bytes*/ + __u64 os_maxbytes; /* maximum object size in bytes */ __u32 os_state; /**< obd_statfs_state OS_STATE_* flag */ __u32 os_fprecreated; /* objs available now to the caller - * used in QoS code to find preferred OSTs */ - __u32 os_spare2; - __u32 os_spare3; + * used in QoS code to find preferred OSTs + */ + __u32 os_spare2; /* Unused padding fields. Remember */ + __u32 os_spare3; /* to fix lustre_swab_obd_statfs() */ __u32 os_spare4; __u32 os_spare5; __u32 os_spare6; From patchwork Mon Sep 30 18:55:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167197 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D358D912 for ; Mon, 30 Sep 2019 19:02:59 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BB796224EF for ; Mon, 30 Sep 2019 19:02:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BB796224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 69A3B21F421; Mon, 30 Sep 2019 11:59:32 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 441695C33AB for ; Mon, 30 Sep 2019 11:57:26 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 40C4D10058FD; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 3F280BD; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:46 -0400 Message-Id: <1569869810-23848-88-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 087/151] lustre: idl: remove obsolete RPC MSG flags X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Andreas Dilger Add comments for MSG_* flags, and delete obsolete flags MSG_LAST_REPLAY, MSG_AT_SUPPORT, MSG_DELAY_REPLAY, MSG_VERSION_REPLAY. Remove now-unused imp_no_lock_replay field. Add comments for MSG_CONNECT_* flags, and delete obsolete flag MSG_CONNECT_ASYNC. Delete them from wirecheck file. WC-bug-id: https://jira.whamcloud.com/browse/LU-6349 Lustre-commit: 455ce0cef704 ("LU-6349 idl: remove obsolete RPC MSG flags") Signed-off-by: Andreas Dilger Reviewed-on: https://review.whamcloud.com/17831 Reviewed-by: Ben Evans Reviewed-by: Steve Guminski Reviewed-by: John L. Hammond Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_import.h | 3 --- fs/lustre/ptlrpc/client.c | 4 ---- fs/lustre/ptlrpc/wiretest.c | 8 ------- include/uapi/linux/lustre/lustre_idl.h | 38 +++++++++++++++++----------------- 4 files changed, 19 insertions(+), 34 deletions(-) diff --git a/fs/lustre/include/lustre_import.h b/fs/lustre/include/lustre_import.h index 6e22274..d9a8d9a 100644 --- a/fs/lustre/include/lustre_import.h +++ b/fs/lustre/include/lustre_import.h @@ -277,9 +277,6 @@ struct obd_import { imp_server_timeout:1, /* VBR: imp in delayed recovery */ imp_delayed_recovery:1, - /* VBR: if gap was found then no lock replays - */ - imp_no_lock_replay:1, /* recovery by versions was failed */ imp_vbr_failed:1, /* force an immediate ping */ diff --git a/fs/lustre/ptlrpc/client.c b/fs/lustre/ptlrpc/client.c index da7077a..0499368 100644 --- a/fs/lustre/ptlrpc/client.c +++ b/fs/lustre/ptlrpc/client.c @@ -2857,7 +2857,6 @@ static int ptlrpc_replay_interpret(const struct lu_env *env, DEBUG_REQ(D_WARNING, req, "Version mismatch during replay\n"); spin_lock(&imp->imp_lock); imp->imp_vbr_failed = 1; - imp->imp_no_lock_replay = 1; spin_unlock(&imp->imp_lock); lustre_msg_set_status(req->rq_repmsg, aa->praa_old_status); } else { @@ -2871,9 +2870,6 @@ static int ptlrpc_replay_interpret(const struct lu_env *env, } spin_lock(&imp->imp_lock); - /** if replays by version then gap occur on server, no trust to locks */ - if (lustre_msg_get_flags(req->rq_repmsg) & MSG_VERSION_REPLAY) - imp->imp_no_lock_replay = 1; imp->imp_last_replay_transno = lustre_msg_get_transno(req->rq_reqmsg); spin_unlock(&imp->imp_lock); LASSERT(imp->imp_last_replay_transno); diff --git a/fs/lustre/ptlrpc/wiretest.c b/fs/lustre/ptlrpc/wiretest.c index 76fb656..e161fe6 100644 --- a/fs/lustre/ptlrpc/wiretest.c +++ b/fs/lustre/ptlrpc/wiretest.c @@ -827,16 +827,10 @@ void lustre_assert_wire_constants(void) (long long)MSGHDR_AT_SUPPORT); LASSERTF(MSGHDR_CKSUM_INCOMPAT18 == 2, "found %lld\n", (long long)MSGHDR_CKSUM_INCOMPAT18); - LASSERTF(MSG_LAST_REPLAY == 0x00000001UL, "found 0x%.8xUL\n", - (unsigned int)MSG_LAST_REPLAY); LASSERTF(MSG_RESENT == 0x00000002UL, "found 0x%.8xUL\n", (unsigned int)MSG_RESENT); LASSERTF(MSG_REPLAY == 0x00000004UL, "found 0x%.8xUL\n", (unsigned int)MSG_REPLAY); - LASSERTF(MSG_DELAY_REPLAY == 0x00000010UL, "found 0x%.8xUL\n", - (unsigned int)MSG_DELAY_REPLAY); - LASSERTF(MSG_VERSION_REPLAY == 0x00000020UL, "found 0x%.8xUL\n", - (unsigned int)MSG_VERSION_REPLAY); LASSERTF(MSG_REQ_REPLAY_DONE == 0x00000040UL, "found 0x%.8xUL\n", (unsigned int)MSG_REQ_REPLAY_DONE); LASSERTF(MSG_LOCK_REPLAY_DONE == 0x00000080UL, "found 0x%.8xUL\n", @@ -851,8 +845,6 @@ void lustre_assert_wire_constants(void) (unsigned int)MSG_CONNECT_LIBCLIENT); LASSERTF(MSG_CONNECT_INITIAL == 0x00000020UL, "found 0x%.8xUL\n", (unsigned int)MSG_CONNECT_INITIAL); - LASSERTF(MSG_CONNECT_ASYNC == 0x00000040UL, "found 0x%.8xUL\n", - (unsigned int)MSG_CONNECT_ASYNC); LASSERTF(MSG_CONNECT_NEXT_VER == 0x00000080UL, "found 0x%.8xUL\n", (unsigned int)MSG_CONNECT_NEXT_VER); LASSERTF(MSG_CONNECT_TRANSNO == 0x00000100UL, "found 0x%.8xUL\n", diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index 2a7f419..41d0abb 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -579,26 +579,26 @@ struct lustre_msg_v2 { #define LUSTRE_LOG_VERSION 0x00050000 #define LUSTRE_MGS_VERSION 0x00060000 -/* pb_flags that apply to all requests */ -#define MSG_LAST_REPLAY 0x0001 -#define MSG_RESENT 0x0002 -#define MSG_REPLAY 0x0004 -/* #define MSG_AT_SUPPORT 0x0008 obsolete 1.5 */ -#define MSG_DELAY_REPLAY 0x0010 -#define MSG_VERSION_REPLAY 0x0020 -#define MSG_REQ_REPLAY_DONE 0x0040 -#define MSG_LOCK_REPLAY_DONE 0x0080 - -/* pb_op_flags for all connect opcodes (MDS_CONNECT, OST_CONNECT) */ -#define MSG_CONNECT_RECOVERING 0x00000001 -#define MSG_CONNECT_RECONNECT 0x00000002 -#define MSG_CONNECT_REPLAYABLE 0x00000004 -/* #define MSG_CONNECT_PEER 0x00000008 removed 1.5 */ -#define MSG_CONNECT_LIBCLIENT 0x00000010 -#define MSG_CONNECT_INITIAL 0x00000020 -#define MSG_CONNECT_ASYNC 0x00000040 +/* pb_flags that apply to all request messages */ +/* #define MSG_LAST_REPLAY 0x0001 obsolete 2.0 => {REQ,LOCK}_REPLAY_DONE */ +#define MSG_RESENT 0x0002 /* was previously sent, no reply seen */ +#define MSG_REPLAY 0x0004 /* was processed, got reply, recovery */ +/* #define MSG_AT_SUPPORT 0x0008 obsolete since 1.5, AT always enabled */ +/* #define MSG_DELAY_REPLAY 0x0010 obsolete since 2.0 */ +/* #define MSG_VERSION_REPLAY 0x0020 obsolete since 1.8.2, VBR always on */ +#define MSG_REQ_REPLAY_DONE 0x0040 /* request replay over, locks next */ +#define MSG_LOCK_REPLAY_DONE 0x0080 /* lock replay over, client done */ + +/* pb_op_flags for connect opcodes: MDS_CONNECT, OST_CONNECT, MGS_CONNECT */ +#define MSG_CONNECT_RECOVERING 0x00000001 /* target is in recovery */ +#define MSG_CONNECT_RECONNECT 0x00000002 /* tgt already has client import */ +#define MSG_CONNECT_REPLAYABLE 0x00000004 /* target supports RPC replay */ +/* #define MSG_CONNECT_PEER 0x00000008 obsolete since 1.2, removed in 1.5 */ +#define MSG_CONNECT_LIBCLIENT 0x00000010 /* obsolete since 2.3, removed 2.6 */ +#define MSG_CONNECT_INITIAL 0x00000020 /* first client connection attempt */ +/* #define MSG_CONNECT_ASYNC 0x00000040 obsolete since 1.5 */ #define MSG_CONNECT_NEXT_VER 0x00000080 /* use next version of lustre_msg */ -#define MSG_CONNECT_TRANSNO 0x00000100 /* report transno */ +#define MSG_CONNECT_TRANSNO 0x00000100 /* client sent transno in replay */ /* number of previous object versions in pb_pre_versions[] */ #define PTLRPC_NUM_VERSIONS 4 From patchwork Mon Sep 30 18:55:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167199 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2B05016B1 for ; Mon, 30 Sep 2019 19:03:11 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 12F37224F0 for ; Mon, 30 Sep 2019 19:03:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 12F37224F0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 089EA5E45B1; Mon, 30 Sep 2019 11:59:36 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 999225C3A09 for ; Mon, 30 Sep 2019 11:57:26 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 43DB61005C83; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 42836B5; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:47 -0400 Message-Id: <1569869810-23848-89-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 088/151] lnet: libcfs: call proper crypto algo when keys are passed in X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: James Simmons , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" In most cases keys are not passed to cfs_crypto_hash_alloc() but if they are then crypto_ahash_setkey() will fail. Keys are only handled by the hmac version of the algorithm requested. If a key is passed into cfs_crypto_hash_alloc() then we should request the hmac version of the algorithm when calling crypto_alloc_ahash(). WC-bug-id: https://jira.whamcloud.com/browse/LU-8602 Lustre-commit: 53e967746f37 ("LU-8602 libcfs: call proper crypto algo when keys are passed in") Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/25199 Reviewed-by: Sebastien Buisson Reviewed-by: Alexandr Boyko Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/libcfs/linux-crypto.c | 31 +++++++++++++++++++++---------- 1 file changed, 21 insertions(+), 10 deletions(-) diff --git a/net/lnet/libcfs/linux-crypto.c b/net/lnet/libcfs/linux-crypto.c index 1c96dc6..53285c2 100644 --- a/net/lnet/libcfs/linux-crypto.c +++ b/net/lnet/libcfs/linux-crypto.c @@ -70,14 +70,27 @@ static int cfs_crypto_hash_alloc(enum cfs_crypto_hash_alg hash_alg, int err = 0; *type = cfs_crypto_hash_type(hash_alg); - if (!*type) { CWARN("Unsupported hash algorithm id = %d, max id is %d\n", hash_alg, CFS_HASH_ALG_MAX); return -EINVAL; } - tfm = crypto_alloc_ahash((*type)->cht_name, 0, CRYPTO_ALG_ASYNC); + /* Keys are only supported for the hmac version */ + if (key && key_len > 0) { + char *algo_name; + + algo_name = kasprintf(GFP_KERNEL, "hmac(%s)", + (*type)->cht_name); + if (!algo_name) + return -ENOMEM; + + tfm = crypto_alloc_ahash(algo_name, 0, CRYPTO_ALG_ASYNC); + kfree(algo_name); + } else { + tfm = crypto_alloc_ahash((*type)->cht_name, 0, + CRYPTO_ALG_ASYNC); + } if (IS_ERR(tfm)) { CDEBUG(D_INFO, "Failed to alloc crypto hash %s\n", (*type)->cht_name); @@ -88,8 +101,8 @@ static int cfs_crypto_hash_alloc(enum cfs_crypto_hash_alg hash_alg, if (!*req) { CDEBUG(D_INFO, "Failed to alloc ahash_request for %s\n", (*type)->cht_name); - crypto_free_ahash(tfm); - return -ENOMEM; + err = -ENOMEM; + goto out_free_tfm; } ahash_request_set_callback(*req, 0, NULL, NULL); @@ -100,12 +113,8 @@ static int cfs_crypto_hash_alloc(enum cfs_crypto_hash_alg hash_alg, err = crypto_ahash_setkey(tfm, (unsigned char *)&((*type)->cht_key), (*type)->cht_size); - - if (err) { - ahash_request_free(*req); - crypto_free_ahash(tfm); - return err; - } + if (err) + goto out_free_req; CDEBUG(D_INFO, "Using crypto hash: %s (%s) speed %d MB/s\n", crypto_ahash_alg_name(tfm), crypto_ahash_driver_name(tfm), @@ -113,7 +122,9 @@ static int cfs_crypto_hash_alloc(enum cfs_crypto_hash_alg hash_alg, err = crypto_ahash_init(*req); if (err) { +out_free_req: ahash_request_free(*req); +out_free_tfm: crypto_free_ahash(tfm); } return err; From patchwork Mon Sep 30 18:55:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167235 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 70C9116C1 for ; Mon, 30 Sep 2019 19:04:43 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5907E224EF for ; Mon, 30 Sep 2019 19:04:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5907E224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 6F9035E4805; Mon, 30 Sep 2019 12:00:09 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id DA0AB5C3C2B for ; Mon, 30 Sep 2019 11:57:26 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 473491005C84; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 45A26A9; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:48 -0400 Message-Id: <1569869810-23848-90-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 089/151] lustre: clio: remove unused cl_lock layers X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: "John L. Hammond" Remove the struct vvp_lock and struct lovsub_lock, omitting these layers from cl_lock. Adjust cl_lock_enqueue() to allow for empty locks (corresponding to unstriped files). WC-bug-id: https://jira.whamcloud.com/browse/LU-10257 Lustre-commit: f168371f40e1 (LU-10257 clio: remove unused cl_lock layers") Signed-off-by: John L. Hammond Reviewed-on: https://review.whamcloud.com/30192 Reviewed-by: Bobi Jam Reviewed-by: Jinshan Xiong Reviewed-by: Mike Pershin Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/cl_object.h | 14 +++---- fs/lustre/llite/Makefile | 2 +- fs/lustre/llite/vvp_dev.c | 6 --- fs/lustre/llite/vvp_internal.h | 12 ------ fs/lustre/llite/vvp_lock.c | 87 ----------------------------------------- fs/lustre/llite/vvp_object.c | 1 - fs/lustre/lov/Makefile | 2 +- fs/lustre/lov/lov_cl_internal.h | 27 ------------- fs/lustre/lov/lov_dev.c | 6 --- fs/lustre/lov/lovsub_lock.c | 81 -------------------------------------- fs/lustre/lov/lovsub_object.c | 4 +- fs/lustre/obdclass/cl_lock.c | 7 +++- 12 files changed, 13 insertions(+), 236 deletions(-) delete mode 100644 fs/lustre/llite/vvp_lock.c delete mode 100644 fs/lustre/lov/lovsub_lock.c diff --git a/fs/lustre/include/cl_object.h b/fs/lustre/include/cl_object.h index 99534f5..946e6e1 100644 --- a/fs/lustre/include/cl_object.h +++ b/fs/lustre/include/cl_object.h @@ -1058,15 +1058,13 @@ static inline bool __page_in_use(const struct cl_page *page, int refc) * (struct cl_lock) and a list of layers (struct cl_lock_slice), linked to * cl_lock::cll_layers list through cl_lock_slice::cls_linkage. * - * Typical cl_lock consists of the two layers: + * Typical cl_lock consists of one layer: * - * - vvp_lock (vvp specific data), and * - lov_lock (lov specific data). * * lov_lock contains an array of sub-locks. Each of these sub-locks is a * normal cl_lock: it has a header (struct cl_lock) and a list of layers: * - * - lovsub_lock, and * - osc_lock * * Each sub-lock is associated with a cl_object (representing stripe @@ -1183,7 +1181,7 @@ struct cl_lock { /** * Per-layer part of cl_lock * - * \see vvp_lock, lov_lock, lovsub_lock, osc_lock + * \see lov_lock, osc_lock */ struct cl_lock_slice { struct cl_lock *cls_lock; @@ -1198,7 +1196,7 @@ struct cl_lock_slice { /** * - * \see vvp_lock_ops, lov_lock_ops, lovsub_lock_ops, osc_lock_ops + * \see lov_lock_ops, osc_lock_ops */ struct cl_lock_operations { /** @{ */ @@ -1210,8 +1208,7 @@ struct cl_lock_operations { * wait on @anchor for resources * -ve for failure * - * \see vvp_lock_enqueue(), lov_lock_enqueue(), lovsub_lock_enqueue(), - * \see osc_lock_enqueue() + * \see lov_lock_enqueue(), osc_lock_enqueue() */ int (*clo_enqueue)(const struct lu_env *env, const struct cl_lock_slice *slice, @@ -1226,8 +1223,7 @@ struct cl_lock_operations { /** * Destructor. Frees resources and the slice. * - * \see vvp_lock_fini(), lov_lock_fini(), lovsub_lock_fini(), - * \see osc_lock_fini() + * \see lov_lock_fini(), osc_lock_fini() */ void (*clo_fini)(const struct lu_env *env, struct cl_lock_slice *slice); /** diff --git a/fs/lustre/llite/Makefile b/fs/lustre/llite/Makefile index 06e9c05..6d8a33b 100644 --- a/fs/lustre/llite/Makefile +++ b/fs/lustre/llite/Makefile @@ -6,7 +6,7 @@ lustre-y := dcache.o dir.o file.o llite_lib.o llite_nfs.o \ rw.o rw26.o namei.o symlink.o llite_mmap.o range_lock.o \ xattr.o xattr_cache.o xattr_security.o \ super25.o statahead.o glimpse.o lcommon_cl.o lcommon_misc.o \ - vvp_dev.o vvp_page.o vvp_lock.o vvp_io.o vvp_object.o \ + vvp_dev.o vvp_page.o vvp_io.o vvp_object.o \ lproc_llite.o lustre-$(CONFIG_FS_POSIX_ACL) += acl.o diff --git a/fs/lustre/llite/vvp_dev.c b/fs/lustre/llite/vvp_dev.c index 4c5a29d..9f793e9 100644 --- a/fs/lustre/llite/vvp_dev.c +++ b/fs/lustre/llite/vvp_dev.c @@ -55,7 +55,6 @@ */ static struct kmem_cache *ll_thread_kmem; -struct kmem_cache *vvp_lock_kmem; struct kmem_cache *vvp_object_kmem; static struct kmem_cache *vvp_session_kmem; static struct kmem_cache *vvp_thread_kmem; @@ -67,11 +66,6 @@ .ckd_size = sizeof(struct ll_thread_info), }, { - .ckd_cache = &vvp_lock_kmem, - .ckd_name = "vvp_lock_kmem", - .ckd_size = sizeof(struct vvp_lock), - }, - { .ckd_cache = &vvp_object_kmem, .ckd_name = "vvp_object_kmem", .ckd_size = sizeof(struct vvp_object), diff --git a/fs/lustre/llite/vvp_internal.h b/fs/lustre/llite/vvp_internal.h index f9a4552..923f866 100644 --- a/fs/lustre/llite/vvp_internal.h +++ b/fs/lustre/llite/vvp_internal.h @@ -119,7 +119,6 @@ struct vvp_io { extern struct lu_context_key vvp_session_key; extern struct lu_context_key vvp_thread_key; -extern struct kmem_cache *vvp_lock_kmem; extern struct kmem_cache *vvp_object_kmem; struct vvp_thread_info { @@ -247,10 +246,6 @@ struct vvp_device { struct cl_device *vdv_next; }; -struct vvp_lock { - struct cl_lock_slice vlk_cl; -}; - void *ccc_key_init(const struct lu_context *ctx, struct lu_context_key *key); void ccc_key_fini(const struct lu_context *ctx, @@ -296,19 +291,12 @@ static inline struct page *cl2vm_page(const struct cl_page_slice *slice) return cl2vvp_page(slice)->vpg_page; } -static inline struct vvp_lock *cl2vvp_lock(const struct cl_lock_slice *slice) -{ - return container_of(slice, struct vvp_lock, vlk_cl); -} - # define CLOBINVRNT(env, clob, expr) \ ((void)sizeof(env), (void)sizeof(clob), (void)sizeof(!!(expr))) int vvp_io_init(const struct lu_env *env, struct cl_object *obj, struct cl_io *io); int vvp_io_write_commit(const struct lu_env *env, struct cl_io *io); -int vvp_lock_init(const struct lu_env *env, struct cl_object *obj, - struct cl_lock *lock, const struct cl_io *io); int vvp_page_init(const struct lu_env *env, struct cl_object *obj, struct cl_page *page, pgoff_t index); struct lu_object *vvp_object_alloc(const struct lu_env *env, diff --git a/fs/lustre/llite/vvp_lock.c b/fs/lustre/llite/vvp_lock.c deleted file mode 100644 index 4b6c714..0000000 --- a/fs/lustre/llite/vvp_lock.c +++ /dev/null @@ -1,87 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * GPL HEADER START - * - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 only, - * as published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, but - * WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * General Public License version 2 for more details (a copy is included - * in the LICENSE file that accompanied this code). - * - * You should have received a copy of the GNU General Public License - * version 2 along with this program; If not, see - * http://www.gnu.org/licenses/gpl-2.0.html - * - * GPL HEADER END - */ -/* - * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. - * Use is subject to license terms. - * - * Copyright (c) 2014, 2015, Intel Corporation. - */ -/* - * This file is part of Lustre, http://www.lustre.org/ - * Lustre is a trademark of Sun Microsystems, Inc. - * - * Implementation of cl_lock for VVP layer. - * - * Author: Nikita Danilov - */ - -#define DEBUG_SUBSYSTEM S_LLITE - -#include - -#include "vvp_internal.h" - -/***************************************************************************** - * - * Vvp lock functions. - * - */ - -static void vvp_lock_fini(const struct lu_env *env, struct cl_lock_slice *slice) -{ - struct vvp_lock *vlk = cl2vvp_lock(slice); - - kmem_cache_free(vvp_lock_kmem, vlk); -} - -static int vvp_lock_enqueue(const struct lu_env *env, - const struct cl_lock_slice *slice, - struct cl_io *unused, struct cl_sync_io *anchor) -{ - CLOBINVRNT(env, slice->cls_obj, vvp_object_invariant(slice->cls_obj)); - - return 0; -} - -static const struct cl_lock_operations vvp_lock_ops = { - .clo_fini = vvp_lock_fini, - .clo_enqueue = vvp_lock_enqueue, -}; - -int vvp_lock_init(const struct lu_env *env, struct cl_object *obj, - struct cl_lock *lock, const struct cl_io *unused) -{ - struct vvp_lock *vlk; - int result; - - CLOBINVRNT(env, obj, vvp_object_invariant(obj)); - - vlk = kmem_cache_zalloc(vvp_lock_kmem, GFP_NOFS); - if (vlk) { - cl_lock_slice_add(lock, &vlk->vlk_cl, obj, &vvp_lock_ops); - result = 0; - } else { - result = -ENOMEM; - } - return result; -} diff --git a/fs/lustre/llite/vvp_object.c b/fs/lustre/llite/vvp_object.c index 549837f..6606459 100644 --- a/fs/lustre/llite/vvp_object.c +++ b/fs/lustre/llite/vvp_object.c @@ -217,7 +217,6 @@ static void vvp_req_attr_set(const struct lu_env *env, struct cl_object *obj, static const struct cl_object_operations vvp_ops = { .coo_page_init = vvp_page_init, - .coo_lock_init = vvp_lock_init, .coo_io_init = vvp_io_init, .coo_attr_get = vvp_attr_get, .coo_attr_update = vvp_attr_update, diff --git a/fs/lustre/lov/Makefile b/fs/lustre/lov/Makefile index 62eb769..abdaac0 100644 --- a/fs/lustre/lov/Makefile +++ b/fs/lustre/lov/Makefile @@ -5,4 +5,4 @@ obj-$(CONFIG_LUSTRE_FS) += lov.o lov-y := lov_obd.o lov_pack.o lov_offset.o lov_merge.o \ lov_request.o lov_ea.o lov_dev.o lov_object.o lov_page.o \ lov_lock.o lov_io.o lovsub_dev.o lovsub_object.o lovsub_page.o \ - lovsub_lock.o lov_pool.o lproc_lov.o + lov_pool.o lproc_lov.o diff --git a/fs/lustre/lov/lov_cl_internal.h b/fs/lustre/lov/lov_cl_internal.h index 095bca0..4472d6f 100644 --- a/fs/lustre/lov/lov_cl_internal.h +++ b/fs/lustre/lov/lov_cl_internal.h @@ -81,7 +81,6 @@ struct lovsub_device; struct lovsub_object; -struct lovsub_lock; enum lov_device_flags { LOV_DEV_INITIALIZED = 1 << 0 @@ -460,13 +459,6 @@ struct lovsub_object { }; /** - * Lock state at lovsub layer. - */ -struct lovsub_lock { - struct cl_lock_slice lss_cl; -}; - -/** * Describe the environment settings for sublocks. */ struct lov_sublock_env { @@ -607,7 +599,6 @@ struct lov_session { extern struct kmem_cache *lov_thread_kmem; extern struct kmem_cache *lov_session_kmem; -extern struct kmem_cache *lovsub_lock_kmem; extern struct kmem_cache *lovsub_object_kmem; int lov_object_init(const struct lu_env *env, struct lu_object *obj, @@ -618,8 +609,6 @@ int lov_lock_init(const struct lu_env *env, struct cl_object *obj, struct cl_lock *lock, const struct cl_io *io); int lov_io_init(const struct lu_env *env, struct cl_object *obj, struct cl_io *io); -int lovsub_lock_init(const struct lu_env *env, struct cl_object *obj, - struct cl_lock *lock, const struct cl_io *io); int lov_lock_init_composite(const struct lu_env *env, struct cl_object *obj, struct cl_lock *lock, const struct cl_io *io); @@ -781,22 +770,6 @@ static inline struct lovsub_object *lu2lovsub(const struct lu_object *obj) return container_of(obj, struct lovsub_object, lso_cl.co_lu); } -static inline struct lovsub_lock * -cl2lovsub_lock(const struct cl_lock_slice *slice) -{ - LINVRNT(lovsub_is_object(&slice->cls_obj->co_lu)); - return container_of(slice, struct lovsub_lock, lss_cl); -} - -static inline struct lovsub_lock *cl2sub_lock(const struct cl_lock *lock) -{ - const struct cl_lock_slice *slice; - - slice = cl_lock_at(lock, &lovsub_device_type); - LASSERT(slice); - return cl2lovsub_lock(slice); -} - static inline struct lov_lock *cl2lov_lock(const struct cl_lock_slice *slice) { LINVRNT(lov_is_object(&slice->cls_obj->co_lu)); diff --git a/fs/lustre/lov/lov_dev.c b/fs/lustre/lov/lov_dev.c index 5ddf49a..46d2e1e 100644 --- a/fs/lustre/lov/lov_dev.c +++ b/fs/lustre/lov/lov_dev.c @@ -48,7 +48,6 @@ struct kmem_cache *lov_thread_kmem; struct kmem_cache *lov_session_kmem; -struct kmem_cache *lovsub_lock_kmem; struct kmem_cache *lovsub_object_kmem; struct lu_kmem_descr lov_caches[] = { @@ -73,11 +72,6 @@ struct lu_kmem_descr lov_caches[] = { .ckd_size = sizeof(struct lov_session) }, { - .ckd_cache = &lovsub_lock_kmem, - .ckd_name = "lovsub_lock_kmem", - .ckd_size = sizeof(struct lovsub_lock) - }, - { .ckd_cache = &lovsub_object_kmem, .ckd_name = "lovsub_object_kmem", .ckd_size = sizeof(struct lovsub_object) diff --git a/fs/lustre/lov/lovsub_lock.c b/fs/lustre/lov/lovsub_lock.c deleted file mode 100644 index 7b67c92..0000000 --- a/fs/lustre/lov/lovsub_lock.c +++ /dev/null @@ -1,81 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * GPL HEADER START - * - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 only, - * as published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, but - * WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * General Public License version 2 for more details (a copy is included - * in the LICENSE file that accompanied this code). - * - * You should have received a copy of the GNU General Public License - * version 2 along with this program; If not, see - * http://www.gnu.org/licenses/gpl-2.0.html - * - * GPL HEADER END - */ -/* - * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. - * Use is subject to license terms. - * - * Copyright (c) 2011, 2012, Intel Corporation. - */ -/* - * This file is part of Lustre, http://www.lustre.org/ - * Lustre is a trademark of Sun Microsystems, Inc. - * - * Implementation of cl_lock for LOVSUB layer. - * - * Author: Nikita Danilov - */ - -#define DEBUG_SUBSYSTEM S_LOV - -#include "lov_cl_internal.h" - -/** \addtogroup lov - * @{ - */ - -/***************************************************************************** - * - * Lovsub lock operations. - * - */ - -static void lovsub_lock_fini(const struct lu_env *env, - struct cl_lock_slice *slice) -{ - struct lovsub_lock *lsl; - - lsl = cl2lovsub_lock(slice); - kmem_cache_free(lovsub_lock_kmem, lsl); -} - -static const struct cl_lock_operations lovsub_lock_ops = { - .clo_fini = lovsub_lock_fini, -}; - -int lovsub_lock_init(const struct lu_env *env, struct cl_object *obj, - struct cl_lock *lock, const struct cl_io *io) -{ - struct lovsub_lock *lsk; - int result; - - lsk = kmem_cache_zalloc(lovsub_lock_kmem, GFP_NOFS); - if (lsk) { - cl_lock_slice_add(lock, &lsk->lss_cl, obj, &lovsub_lock_ops); - result = 0; - } else { - result = -ENOMEM; - } - return result; -} - -/** @} lov */ diff --git a/fs/lustre/lov/lovsub_object.c b/fs/lustre/lov/lovsub_object.c index 6ba09f1..03a6002 100644 --- a/fs/lustre/lov/lovsub_object.c +++ b/fs/lustre/lov/lovsub_object.c @@ -61,7 +61,7 @@ int lovsub_object_init(const struct lu_env *env, struct lu_object *obj, below = under->ld_ops->ldo_object_alloc(env, obj->lo_header, under); if (below) { lu_object_add(obj, below); - cl_object_page_init(lu2cl(obj), sizeof(struct lovsub_page)); + cl_object_page_init(lu2cl(obj), 0); result = 0; } else { result = -ENOMEM; @@ -145,8 +145,6 @@ static void lovsub_req_attr_set(const struct lu_env *env, struct cl_object *obj, } static const struct cl_object_operations lovsub_ops = { - .coo_page_init = lovsub_page_init, - .coo_lock_init = lovsub_lock_init, .coo_attr_update = lovsub_attr_update, .coo_glimpse = lovsub_object_glimpse, .coo_req_attr_set = lovsub_req_attr_set diff --git a/fs/lustre/obdclass/cl_lock.c b/fs/lustre/obdclass/cl_lock.c index fe24379..07d0dc8 100644 --- a/fs/lustre/obdclass/cl_lock.c +++ b/fs/lustre/obdclass/cl_lock.c @@ -105,7 +105,10 @@ int cl_lock_init(const struct lu_env *env, struct cl_lock *lock, INIT_LIST_HEAD(&lock->cll_layers); cl_object_for_each(scan, obj) { - result = scan->co_ops->coo_lock_init(env, scan, lock, io); + if (scan->co_ops->coo_lock_init) + result = scan->co_ops->coo_lock_init(env, scan, lock, + io); + if (result != 0) { cl_lock_fini(env, lock); break; @@ -159,7 +162,7 @@ int cl_lock_enqueue(const struct lu_env *env, struct cl_io *io, struct cl_lock *lock, struct cl_sync_io *anchor) { const struct cl_lock_slice *slice; - int rc = -EBUSY; + int rc = 0; list_for_each_entry(slice, &lock->cll_layers, cls_linkage) { if (!slice->cls_ops->clo_enqueue) From patchwork Mon Sep 30 18:55:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167237 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D530413B1 for ; Mon, 30 Sep 2019 19:04:43 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BB1B6224EF for ; Mon, 30 Sep 2019 19:04:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BB1B6224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id A70C75E480C; Mon, 30 Sep 2019 12:00:09 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 3B5825C3C2B for ; Mon, 30 Sep 2019 11:57:27 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 4A0951005C85; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 489FCBB; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:49 -0400 Message-Id: <1569869810-23848-91-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 090/151] lustre: sec: migrate to 64 bit time X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: James Simmons , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" Replace cfs_time_current_sec() to avoid the overflow issues in 2038 with ktime_get_real_seconds(). Mirgate the rest of the gss code to time64_t to avoid the 2038 overflow issue. Currently in encrypt_page_pools we are reporting the jiffy cycles for "max wait time" which not only doesn't make sense but can vary from platform to platform. Instead we will report in terms of milliseconds. That requires changing epp_st_max_wait into ktime_t since we need better than seconds precision. Lastly the time in encrypt_page_pools for "last access" and "last shrink" was showing up negative. This was due to epp_last_* field being set to the number of seconds since the epoch instead of the number of seconds since the node booted. Change epp_last_* to being set by ktime_get_seconds() instead of ktime_get_real_seconds() resolves this problem. WC-bug-id: https://jira.whamcloud.com/browse/LU-9019 Lustre-commit: cc759278f18f (LU-9019 sec: migrate to 64 bit time") Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/29859 Reviewed-by: Andreas Dilger Reviewed-by: John L. Hammond Reviewed-by: Sebastien Buisson Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_sec.h | 2 +- fs/lustre/ptlrpc/sec_bulk.c | 22 ++++++++++------------ 2 files changed, 11 insertions(+), 13 deletions(-) diff --git a/fs/lustre/include/lustre_sec.h b/fs/lustre/include/lustre_sec.h index 66054d5..dabc663 100644 --- a/fs/lustre/include/lustre_sec.h +++ b/fs/lustre/include/lustre_sec.h @@ -484,7 +484,7 @@ struct ptlrpc_cli_ctx { atomic_t cc_refcount; struct ptlrpc_sec *cc_sec; struct ptlrpc_ctx_ops *cc_ops; - unsigned long cc_expire; /* in seconds */ + time64_t cc_expire; /* in seconds */ unsigned int cc_early_expire:1; unsigned long cc_flags; struct vfs_cred cc_vcred; diff --git a/fs/lustre/ptlrpc/sec_bulk.c b/fs/lustre/ptlrpc/sec_bulk.c index 74cfdd8..755781c 100644 --- a/fs/lustre/ptlrpc/sec_bulk.c +++ b/fs/lustre/ptlrpc/sec_bulk.c @@ -108,7 +108,7 @@ unsigned long epp_st_missings; /* # of cache missing */ unsigned long epp_st_lowfree; /* lowest free pages reached */ unsigned int epp_st_max_wqlen; /* highest waitqueue length */ - unsigned long epp_st_max_wait; /* in jiffies */ + ktime_t epp_st_max_wait; /* in nanoseconds */ unsigned long epp_st_outofmem; /* # of out of mem requests */ /* * pointers to pools @@ -131,8 +131,8 @@ int sptlrpc_proc_enc_pool_seq_show(struct seq_file *m, void *v) "total pages: %lu\n" "total free: %lu\n" "idle index: %lu/100\n" - "last shrink: %lds\n" - "last access: %lds\n" + "last shrink: %llds\n" + "last access: %llds\n" "max pages reached: %lu\n" "grows: %u\n" "grows failure: %u\n" @@ -141,7 +141,7 @@ int sptlrpc_proc_enc_pool_seq_show(struct seq_file *m, void *v) "cache missing: %lu\n" "low free mark: %lu\n" "max waitqueue depth: %u\n" - "max wait time: %ld/%lu\n" + "max wait time ms: %lld\n" "out of mem: %lu\n", totalram_pages(), PAGES_PER_POOL, @@ -150,8 +150,8 @@ int sptlrpc_proc_enc_pool_seq_show(struct seq_file *m, void *v) page_pools.epp_total_pages, page_pools.epp_free_pages, page_pools.epp_idle_idx, - (long)(ktime_get_seconds() - page_pools.epp_last_shrink), - (long)(ktime_get_seconds() - page_pools.epp_last_access), + ktime_get_seconds() - page_pools.epp_last_shrink, + ktime_get_seconds() - page_pools.epp_last_access, page_pools.epp_st_max_pages, page_pools.epp_st_grows, page_pools.epp_st_grow_fails, @@ -160,8 +160,7 @@ int sptlrpc_proc_enc_pool_seq_show(struct seq_file *m, void *v) page_pools.epp_st_missings, page_pools.epp_st_lowfree, page_pools.epp_st_max_wqlen, - page_pools.epp_st_max_wait, - msecs_to_jiffies(MSEC_PER_SEC), + ktime_to_ms(page_pools.epp_st_max_wait), page_pools.epp_st_outofmem); spin_unlock(&page_pools.epp_lock); @@ -432,7 +431,7 @@ int sptlrpc_enc_pool_init(void) page_pools.epp_st_missings = 0; page_pools.epp_st_lowfree = 0; page_pools.epp_st_max_wqlen = 0; - page_pools.epp_st_max_wait = 0; + page_pools.epp_st_max_wait = ktime_set(0, 0); page_pools.epp_st_outofmem = 0; enc_pools_alloc(); @@ -463,13 +462,12 @@ void sptlrpc_enc_pool_fini(void) if (page_pools.epp_st_access > 0) { CDEBUG(D_SEC, - "max pages %lu, grows %u, grow fails %u, shrinks %u, access %lu, missing %lu, max qlen %u, max wait %ld/%ld, out of mem %lu\n", + "max pages %lu, grows %u, grow fails %u, shrinks %u, access %lu, missing %lu, max qlen %u, max wait ms %lld, out of mem %lu\n", page_pools.epp_st_max_pages, page_pools.epp_st_grows, page_pools.epp_st_grow_fails, page_pools.epp_st_shrinks, page_pools.epp_st_access, page_pools.epp_st_missings, page_pools.epp_st_max_wqlen, - page_pools.epp_st_max_wait, - msecs_to_jiffies(MSEC_PER_SEC), + ktime_to_ms(page_pools.epp_st_max_wait), page_pools.epp_st_outofmem); } } From patchwork Mon Sep 30 18:55:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167241 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EC5B216C1 for ; Mon, 30 Sep 2019 19:04:54 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D5215224EF for ; Mon, 30 Sep 2019 19:04:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D5215224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 1900D5E48A7; Mon, 30 Sep 2019 12:00:21 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 91FFA5C3C36 for ; Mon, 30 Sep 2019 11:57:27 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 4CCE01005C86; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 4B868B4; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:50 -0400 Message-Id: <1569869810-23848-92-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 091/151] lustre: llite: avoid live-lock when concurrent mmap()s X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Bruno Faccini Patch is an attempt to delay page-fault retry from Client side for inode with extents being mmap()'ed, to prevent live-lock situation to occur with other page's competitors. WC-bug-id: https://jira.whamcloud.com/browse/LU-10149 Lustre-commit: cd793b4067b6 ("LU-10149 llite: avoid live-lock when concurrent mmap()s") Signed-off-by: Bruno Faccini Reviewed-on: https://review.whamcloud.com/30465 Reviewed-by: Andreas Dilger Reviewed-by: Bobi Jam Reviewed-by: Jinshan Xiong Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/llite_mmap.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/fs/lustre/llite/llite_mmap.c b/fs/lustre/llite/llite_mmap.c index 59fb400..14080b6 100644 --- a/fs/lustre/llite/llite_mmap.c +++ b/fs/lustre/llite/llite_mmap.c @@ -38,6 +38,7 @@ #include #include #include +#include #include #include @@ -153,7 +154,7 @@ static int __ll_page_mkwrite(struct vm_area_struct *vma, struct page *vmpage, int result; u16 refcheck; sigset_t old, new; - struct inode *inode; + struct inode *inode = NULL; struct ll_inode_info *lli; env = cl_env_get(&refcheck); @@ -229,6 +230,17 @@ static int __ll_page_mkwrite(struct vm_area_struct *vma, struct page *vmpage, CDEBUG(D_MMAP, "%s mkwrite with %d\n", current->comm, result); LASSERT(ergo(result == 0, PageLocked(vmpage))); + /* if page has been unmapped, presumably due to lock reclaim for + * concurrent usage, add some delay before retrying to prevent + * entering live-lock situation with competitors + */ + if (result == -ENODATA && inode) { + CDEBUG(D_MMAP, + "delaying new page-fault for inode %p to prevent live-lock\n", + inode); + msleep(20); + } + return result; } From patchwork Mon Sep 30 18:55:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167239 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3406C16C1 for ; Mon, 30 Sep 2019 19:04:51 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1C33E224EF for ; Mon, 30 Sep 2019 19:04:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1C33E224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 848785E485D; Mon, 30 Sep 2019 12:00:18 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id D3E965C3C42 for ; Mon, 30 Sep 2019 11:57:27 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 516EC1005C8B; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 4ECAEB5; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:51 -0400 Message-Id: <1569869810-23848-93-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 092/151] lustre: llite: change lli_glimpse_time to ktime X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: James Simmons , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" Currently lli_glimpse_time is in jiffies which can vary between platforms. Migrate to ktime since we need more than second time resolution that is consistent on any platform. Replace the last cfs_time_current_sec() with ktime_get_real_seconds(). WC-bug-id: https://jira.whamcloud.com/browse/LU-9019 Lustre-commit: 9a484620bdd9 ("LU-9019 llite: change lli_glimpse_time to ktime") Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/30601 Reviewed-by: Dmitry Eremin Reviewed-by: Mike Pershin Reviewed-by: John L. Hammond Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/llite_internal.h | 4 ++-- fs/lustre/llite/llite_lib.c | 2 +- fs/lustre/llite/statahead.c | 9 +++++---- 3 files changed, 8 insertions(+), 7 deletions(-) diff --git a/fs/lustre/llite/llite_internal.h b/fs/lustre/llite/llite_internal.h index a44aec06..bdc90bb 100644 --- a/fs/lustre/llite/llite_internal.h +++ b/fs/lustre/llite/llite_internal.h @@ -194,7 +194,7 @@ struct ll_inode_info { struct range_lock_tree lli_write_tree; struct rw_semaphore lli_glimpse_sem; - unsigned long lli_glimpse_time; + ktime_t lli_glimpse_time; struct list_head lli_agl_list; u64 lli_agl_index; @@ -1193,7 +1193,7 @@ static inline int ll_glimpse_size(struct inode *inode) down_read(&lli->lli_glimpse_sem); rc = cl_glimpse_size(inode); - lli->lli_glimpse_time = jiffies; + lli->lli_glimpse_time = ktime_get(); up_read(&lli->lli_glimpse_sem); return rc; } diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index 12a68873..9e4c797 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -926,7 +926,7 @@ void ll_lli_init(struct ll_inode_info *lli) init_rwsem(&lli->lli_trunc_sem); range_lock_tree_init(&lli->lli_write_tree); init_rwsem(&lli->lli_glimpse_sem); - lli->lli_glimpse_time = 0; + lli->lli_glimpse_time = ktime_set(0, 0); INIT_LIST_HEAD(&lli->lli_agl_list); lli->lli_agl_index = 0; lli->lli_async_rc = 0; diff --git a/fs/lustre/llite/statahead.c b/fs/lustre/llite/statahead.c index 8fc4471..2b7015d 100644 --- a/fs/lustre/llite/statahead.c +++ b/fs/lustre/llite/statahead.c @@ -494,10 +494,10 @@ static void ll_agl_trigger(struct inode *inode, struct ll_statahead_info *sai) { struct ll_inode_info *lli = ll_i2info(inode); u64 index = lli->lli_agl_index; + ktime_t expire; int rc; LASSERT(list_empty(&lli->lli_agl_list)); - /* AGL maybe fall behind statahead with one entry */ if (is_omitted_entry(sai, index + 1)) { lli->lli_agl_index = 0; @@ -539,8 +539,9 @@ static void ll_agl_trigger(struct inode *inode, struct ll_statahead_info *sai) * relative rare. AGL can ignore such case, and it will not muchly * affect the performance. */ - if (lli->lli_glimpse_time != 0 && - time_before(jiffies - 1 * HZ, lli->lli_glimpse_time)) { + expire = ktime_sub_ns(ktime_get(), NSEC_PER_SEC); + if (ktime_to_ns(lli->lli_glimpse_time) && + ktime_before(expire, lli->lli_glimpse_time)) { up_write(&lli->lli_glimpse_sem); lli->lli_agl_index = 0; iput(inode); @@ -552,7 +553,7 @@ static void ll_agl_trigger(struct inode *inode, struct ll_statahead_info *sai) cl_agl(inode); lli->lli_agl_index = 0; - lli->lli_glimpse_time = jiffies; + lli->lli_glimpse_time = ktime_get(); up_write(&lli->lli_glimpse_sem); CDEBUG(D_READA, "Handled (init) async glimpse: inode= " From patchwork Mon Sep 30 18:55:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167243 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1DBE713B1 for ; Mon, 30 Sep 2019 19:04:56 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 06410224EF for ; Mon, 30 Sep 2019 19:04:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 06410224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 9FF615E48B8; Mon, 30 Sep 2019 12:00:21 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 3708E5C3B25 for ; Mon, 30 Sep 2019 11:57:28 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 530F21005C8C; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 51A8ABD; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:52 -0400 Message-Id: <1569869810-23848-94-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 093/151] lustre: hsm: filter kkuc write by client UUID X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: "John L. Hammond" Add a struct obd_uuid kr_uuid member to struct kkuc_reg to hold the UUID of the client (super block) that owns the kkuc pipe. Modify libcfs_kkuc_group_{put,rem,foreach}() to accept a UUID pointer which filters the kkuc pipes operated on. Modify mdc_hsm_copytool_send() to pass the UUID of the MDC device when calling libcfs_kkuc_group_put(). The effect of all this is that HALs received by a given MDC will only be delivered to copytools registered on the corresponding mount point. Remove the cluuid member of struct lmv_obd since it is always the same as the obd_uuid member of the corresponding struct obd_device. Remove the kcd_uuid member of struct kkuc_ct_data as it is no longer needed. WC-bug-id: https://jira.whamcloud.com/browse/LU-10341 Lustre-commit: c73736545faf ("LU-10341 hsm: filter kkuc write by client UUID") Signed-off-by: John L. Hammond Reviewed-on: https://review.whamcloud.com/30419 Reviewed-by: Dmitry Eremin Reviewed-by: Faccini Bruno Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_export.h | 1 - fs/lustre/include/lustre_kernelcomm.h | 12 +++++------ fs/lustre/include/obd.h | 1 - fs/lustre/lmv/lmv_obd.c | 38 +++++++++++++++++------------------ fs/lustre/mdc/mdc_request.c | 22 +++++++++----------- fs/lustre/obdclass/kernelcomm.c | 26 ++++++++++++++---------- 6 files changed, 50 insertions(+), 50 deletions(-) diff --git a/fs/lustre/include/lustre_export.h b/fs/lustre/include/lustre_export.h index 4ac996b..9e2878e 100644 --- a/fs/lustre/include/lustre_export.h +++ b/fs/lustre/include/lustre_export.h @@ -267,7 +267,6 @@ static inline int exp_connect_lockahead(struct obd_export *exp) #define KKUC_CT_DATA_MAGIC 0x092013cea struct kkuc_ct_data { u32 kcd_magic; - struct obd_uuid kcd_uuid; u32 kcd_archive; }; diff --git a/fs/lustre/include/lustre_kernelcomm.h b/fs/lustre/include/lustre_kernelcomm.h index 1ed4184..2c8ce19 100644 --- a/fs/lustre/include/lustre_kernelcomm.h +++ b/fs/lustre/include/lustre_kernelcomm.h @@ -46,11 +46,11 @@ /* Kernel methods */ void libcfs_kkuc_init(void); -int libcfs_kkuc_group_put(unsigned int group, void *payload); -int libcfs_kkuc_group_add(struct file *fp, int uid, unsigned int group, - void *data, size_t data_len); -int libcfs_kkuc_group_rem(int uid, unsigned int group); -int libcfs_kkuc_group_foreach(unsigned int group, libcfs_kkuc_cb_t cb_func, - void *cb_arg); +int libcfs_kkuc_group_put(const struct obd_uuid *uuid, int group, void *data); +int libcfs_kkuc_group_add(struct file *fp, const struct obd_uuid *uuid, int uid, + int group, void *data, size_t data_len); +int libcfs_kkuc_group_rem(const struct obd_uuid *uuid, int uid, int group); +int libcfs_kkuc_group_foreach(const struct obd_uuid *uuid, int group, + libcfs_kkuc_cb_t cb_func, void *cb_arg); #endif /* __LUSTRE_KERNELCOMM_H__ */ diff --git a/fs/lustre/include/obd.h b/fs/lustre/include/obd.h index e377526..f252ff0 100644 --- a/fs/lustre/include/obd.h +++ b/fs/lustre/include/obd.h @@ -430,7 +430,6 @@ struct lmv_obd { struct lu_client_fld lmv_fld; spinlock_t lmv_lock; struct lmv_desc desc; - struct obd_uuid cluuid; struct mutex lmv_init_mutex; int connected; diff --git a/fs/lustre/lmv/lmv_obd.c b/fs/lustre/lmv/lmv_obd.c index 47fc22c..b6459f5 100644 --- a/fs/lustre/lmv/lmv_obd.c +++ b/fs/lustre/lmv/lmv_obd.c @@ -206,7 +206,6 @@ static int lmv_connect(const struct lu_env *env, exp = class_conn2export(&conn); lmv->connected = 0; - lmv->cluuid = *cluuid; lmv->conn_data = *data; lmv->lmv_tgts_kobj = kobject_create_and_add("target_obds", @@ -276,8 +275,6 @@ static int lmv_init_ea_size(struct obd_export *exp, u32 easize, u32 def_easize) static int lmv_connect_mdc(struct obd_device *obd, struct lmv_tgt_desc *tgt) { struct lmv_obd *lmv = &obd->u.lmv; - struct obd_uuid *cluuid = &lmv->cluuid; - struct obd_uuid lmv_mdc_uuid = { "LMV_MDC_UUID" }; struct obd_device *mdc_obd; struct obd_export *mdc_exp; struct lu_fld_target target; @@ -290,16 +287,16 @@ static int lmv_connect_mdc(struct obd_device *obd, struct lmv_tgt_desc *tgt) return -EINVAL; } - CDEBUG(D_CONFIG, "connect to %s(%s) - %s, %s FOR %s\n", + CDEBUG(D_CONFIG, "connect to %s(%s) - %s, %s\n", mdc_obd->obd_name, mdc_obd->obd_uuid.uuid, - tgt->ltd_uuid.uuid, obd->obd_uuid.uuid, cluuid->uuid); + tgt->ltd_uuid.uuid, obd->obd_uuid.uuid); if (!mdc_obd->obd_set_up) { CERROR("target %s is not set up\n", tgt->ltd_uuid.uuid); return -EINVAL; } - rc = obd_connect(NULL, &mdc_exp, mdc_obd, &lmv_mdc_uuid, + rc = obd_connect(NULL, &mdc_exp, mdc_obd, &obd->obd_uuid, &lmv->conn_data, NULL); if (rc) { CERROR("target %s connect error %d\n", tgt->ltd_uuid.uuid, rc); @@ -499,7 +496,7 @@ static int lmv_check_connect(struct obd_device *obd) } CDEBUG(D_CONFIG, "Time to connect %s to %s\n", - lmv->cluuid.uuid, obd->obd_name); + obd->obd_uuid.uuid, obd->obd_name); for (i = 0; i < lmv->desc.ld_tgt_count; i++) { tgt = lmv->tgts[i]; @@ -753,10 +750,11 @@ static int lmv_hsm_req_build(struct lmv_obd *lmv, return 0; } -static int lmv_hsm_ct_unregister(struct lmv_obd *lmv, unsigned int cmd, int len, - struct lustre_kernelcomm *lk, +static int lmv_hsm_ct_unregister(struct obd_device *obd, unsigned int cmd, + int len, struct lustre_kernelcomm *lk, void __user *uarg) { + struct lmv_obd *lmv = &obd->u.lmv; u32 i; /* unregister request (call from llapi_hsm_copytool_fini) */ @@ -776,20 +774,21 @@ static int lmv_hsm_ct_unregister(struct lmv_obd *lmv, unsigned int cmd, int len, * Unreached coordinators will get EPIPE on next requests * and will unregister automatically. */ - return libcfs_kkuc_group_rem(lk->lk_uid, lk->lk_group); + return libcfs_kkuc_group_rem(&obd->obd_uuid, lk->lk_uid, lk->lk_group); } -static int lmv_hsm_ct_register(struct lmv_obd *lmv, unsigned int cmd, int len, - struct lustre_kernelcomm *lk, void __user *uarg) +static int lmv_hsm_ct_register(struct obd_device *obd, unsigned int cmd, + int len, struct lustre_kernelcomm *lk, + void __user *uarg) { + struct lmv_obd *lmv = &obd->u.lmv; struct file *filp; u32 i, j; int err; bool any_set = false; struct kkuc_ct_data kcd = { .kcd_magic = KKUC_CT_DATA_MAGIC, - .kcd_uuid = lmv->cluuid, - .kcd_archive = lk->lk_data + .kcd_archive = lk->lk_data, }; int rc = 0; @@ -797,8 +796,8 @@ static int lmv_hsm_ct_register(struct lmv_obd *lmv, unsigned int cmd, int len, if (!filp) return -EBADF; - rc = libcfs_kkuc_group_add(filp, lk->lk_uid, lk->lk_group, - &kcd, sizeof(kcd)); + rc = libcfs_kkuc_group_add(filp, &obd->obd_uuid, lk->lk_uid, + lk->lk_group, &kcd, sizeof(kcd)); if (rc) goto err_fput; @@ -848,7 +847,7 @@ static int lmv_hsm_ct_register(struct lmv_obd *lmv, unsigned int cmd, int len, return 0; err_kkuc_rem: - libcfs_kkuc_group_rem(lk->lk_uid, lk->lk_group); + libcfs_kkuc_group_rem(&obd->obd_uuid, lk->lk_uid, lk->lk_group); err_fput: fput(filp); return rc; @@ -1084,9 +1083,9 @@ static int lmv_iocontrol(unsigned int cmd, struct obd_export *exp, struct lustre_kernelcomm *lk = karg; if (lk->lk_flags & LK_FLG_STOP) - rc = lmv_hsm_ct_unregister(lmv, cmd, len, lk, uarg); + rc = lmv_hsm_ct_unregister(obddev, cmd, len, lk, uarg); else - rc = lmv_hsm_ct_register(lmv, cmd, len, lk, uarg); + rc = lmv_hsm_ct_register(obddev, cmd, len, lk, uarg); break; } default: @@ -2562,6 +2561,7 @@ static int lmv_unlink(struct obd_export *exp, struct md_op_data *op_data, static int lmv_precleanup(struct obd_device *obd) { + libcfs_kkuc_group_rem(&obd->obd_uuid, 0, KUC_GRP_HSM); fld_client_debugfs_fini(&obd->u.lmv.lmv_fld); lprocfs_obd_cleanup(obd); ldebugfs_free_md_stats(obd); diff --git a/fs/lustre/mdc/mdc_request.c b/fs/lustre/mdc/mdc_request.c index efa6516..29a4886 100644 --- a/fs/lustre/mdc/mdc_request.c +++ b/fs/lustre/mdc/mdc_request.c @@ -2173,10 +2173,12 @@ static int mdc_ioc_hsm_ct_start(struct obd_export *exp, /** * Send a message to any listening copytools * + * @uuid: obd device UUID * @val: KUC message (kuc_hdr + hsm_action_list) * @len: total length of message */ -static int mdc_hsm_copytool_send(size_t len, void *val) +static int mdc_hsm_copytool_send(const struct obd_uuid *uuid, + size_t len, void *val) { struct kuc_hdr *lh = (struct kuc_hdr *)val; struct hsm_action_list *hal = (struct hsm_action_list *)(lh + 1); @@ -2200,7 +2202,7 @@ static int mdc_hsm_copytool_send(size_t len, void *val) lh->kuc_msglen, hal->hal_count, hal->hal_fsname); /* Broadcast to HSM listeners */ - return libcfs_kkuc_group_put(KUC_GRP_HSM, lh); + return libcfs_kkuc_group_put(uuid, KUC_GRP_HSM, lh); } /** @@ -2219,9 +2221,6 @@ static int mdc_hsm_ct_reregister(void *data, void *cb_arg) if (!kcd || kcd->kcd_magic != KKUC_CT_DATA_MAGIC) return -EPROTO; - if (!obd_uuid_equals(&kcd->kcd_uuid, &imp->imp_obd->obd_uuid)) - return 0; - CDEBUG(D_HA, "%s: recover copytool registration to MDT (archive=%#x)\n", imp->imp_obd->obd_name, kcd->kcd_archive); rc = mdc_ioc_hsm_ct_register(imp, kcd->kcd_archive); @@ -2264,7 +2263,8 @@ static int mdc_set_info_async(const struct lu_env *env, return rc; } if (KEY_IS(KEY_HSM_COPYTOOL_SEND)) { - rc = mdc_hsm_copytool_send(vallen, val); + rc = mdc_hsm_copytool_send(&imp->imp_obd->obd_uuid, vallen, + val); return rc; } if (KEY_IS(KEY_DEFAULT_EASIZE)) { @@ -2403,11 +2403,13 @@ static int mdc_import_event(struct obd_device *obd, struct obd_import *imp, case IMP_EVENT_ACTIVE: rc = obd_notify_observer(obd, obd, OBD_NOTIFY_ACTIVE); /* redo the kuc registration after reconnecting */ - if (rc == 0) + if (rc == 0) { /* re-register HSM agents */ - rc = libcfs_kkuc_group_foreach(KUC_GRP_HSM, + rc = libcfs_kkuc_group_foreach(&imp->imp_obd->obd_uuid, + KUC_GRP_HSM, mdc_hsm_ct_reregister, (void *)imp); + } break; case IMP_EVENT_OCD: { struct obd_connect_data *ocd = &imp->imp_connect_data; @@ -2576,10 +2578,6 @@ static int mdc_precleanup(struct obd_device *obd) { osc_precleanup_common(obd); - /* Failsafe, ok if racy */ - if (atomic_read(&obd->obd_type->typ_refcnt) <= 1) - libcfs_kkuc_group_rem(0, KUC_GRP_HSM); - mdc_changelog_cdev_finish(obd); obd_cleanup_client_import(obd); diff --git a/fs/lustre/obdclass/kernelcomm.c b/fs/lustre/obdclass/kernelcomm.c index ee998ba..e92a52c 100644 --- a/fs/lustre/obdclass/kernelcomm.c +++ b/fs/lustre/obdclass/kernelcomm.c @@ -90,6 +90,7 @@ static int libcfs_kkuc_msg_put(struct file *filp, void *payload) /** A single group registration has a uid and a file pointer */ struct kkuc_reg { struct list_head kr_chain; + struct obd_uuid kr_uuid; int kr_uid; struct file *kr_fp; char kr_data[0]; @@ -115,12 +116,13 @@ void libcfs_kkuc_init(void) /** Add a receiver to a broadcast group * * @filp: pipe to write into + * @uuid: obd device UUID * @uid: identifier for this receiver * @group: group number * @data: user data */ -int libcfs_kkuc_group_add(struct file *filp, int uid, unsigned int group, - void *data, size_t data_len) +int libcfs_kkuc_group_add(struct file *filp, const struct obd_uuid *uuid, + int uid, int group, void *data, size_t data_len) { struct kkuc_reg *reg; @@ -134,10 +136,11 @@ int libcfs_kkuc_group_add(struct file *filp, int uid, unsigned int group, return -EBADF; /* freed in group_rem */ - reg = kmalloc(sizeof(*reg) + data_len, GFP_KERNEL); + reg = kzalloc(sizeof(*reg) + data_len, GFP_KERNEL); if (!reg) return -ENOMEM; + reg->kr_uuid = *uuid; reg->kr_fp = filp; reg->kr_uid = uid; memcpy(reg->kr_data, data, data_len); @@ -152,7 +155,7 @@ int libcfs_kkuc_group_add(struct file *filp, int uid, unsigned int group, } EXPORT_SYMBOL(libcfs_kkuc_group_add); -int libcfs_kkuc_group_rem(int uid, unsigned int group) +int libcfs_kkuc_group_rem(const struct obd_uuid *uuid, int uid, int group) { struct kkuc_reg *reg, *next; @@ -169,12 +172,13 @@ int libcfs_kkuc_group_rem(int uid, unsigned int group) lh.kuc_transport = KUC_TRANSPORT_GENERIC; lh.kuc_msgtype = KUC_MSG_SHUTDOWN; lh.kuc_msglen = sizeof(lh); - libcfs_kkuc_group_put(group, &lh); + libcfs_kkuc_group_put(uuid, group, &lh); } down_write(&kg_sem); list_for_each_entry_safe(reg, next, &kkuc_groups[group], kr_chain) { - if (!uid || (uid == reg->kr_uid)) { + if (obd_uuid_equals(uuid, ®->kr_uuid) && + (!uid || uid == reg->kr_uid)) { list_del(®->kr_chain); CDEBUG(D_HSM, "Removed uid=%d fp=%p from group %d\n", reg->kr_uid, reg->kr_fp, group); @@ -189,7 +193,7 @@ int libcfs_kkuc_group_rem(int uid, unsigned int group) } EXPORT_SYMBOL(libcfs_kkuc_group_rem); -int libcfs_kkuc_group_put(unsigned int group, void *payload) +int libcfs_kkuc_group_put(const struct obd_uuid *uuid, int group, void *payload) { struct kkuc_reg *reg; int rc = 0; @@ -210,7 +214,7 @@ int libcfs_kkuc_group_put(unsigned int group, void *payload) } list_for_each_entry(reg, &kkuc_groups[group], kr_chain) { - if (reg->kr_fp) { + if (obd_uuid_equals(uuid, ®->kr_uuid) && reg->kr_fp) { rc = libcfs_kkuc_msg_put(reg->kr_fp, payload); if (!rc) { one_success = 1; @@ -240,8 +244,8 @@ int libcfs_kkuc_group_put(unsigned int group, void *payload) * @cb_func: the function to be called. * @cb_arg: extra argument to be passed to the callback function. */ -int libcfs_kkuc_group_foreach(unsigned int group, libcfs_kkuc_cb_t cb_func, - void *cb_arg) +int libcfs_kkuc_group_foreach(const struct obd_uuid *uuid, int group, + libcfs_kkuc_cb_t cb_func, void *cb_arg) { struct kkuc_reg *reg; int rc = 0; @@ -253,7 +257,7 @@ int libcfs_kkuc_group_foreach(unsigned int group, libcfs_kkuc_cb_t cb_func, down_read(&kg_sem); list_for_each_entry(reg, &kkuc_groups[group], kr_chain) { - if (reg->kr_fp) + if (obd_uuid_equals(uuid, ®->kr_uuid) && reg->kr_fp) rc = cb_func(reg->kr_data, cb_arg); } up_read(&kg_sem); From patchwork Mon Sep 30 18:55:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167247 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4571313B1 for ; Mon, 30 Sep 2019 19:05:03 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2E0CD224EF for ; Mon, 30 Sep 2019 19:05:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2E0CD224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id AEF945E4919; Mon, 30 Sep 2019 12:00:24 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8D4235C3B25 for ; Mon, 30 Sep 2019 11:57:28 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 560851005C8D; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 54C6BA9; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:53 -0400 Message-Id: <1569869810-23848-95-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 094/151] lustre: dne: allow mkdir with specific MDTs X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lai Siyao , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Lai Siyao Silimiar to create file on specific OSTs, allow 'lfs mkdir' to mkdir on specific MDTs. This is achieved by allowing 'lfs mkdir -i' specify multiple MDTs, and adding LMV_USER_MAGIC_SPECIFIC. WC-bug-id: https://jira.whamcloud.com/browse/LU-8616 Lustre-commit: 087ec276de0b ("LU-8616 dne: allow mkdir with specific MDTs") Signed-off-by: Lai Siyao Reviewed-on: https://review.whamcloud.com/30566 Reviewed-by: Andreas Dilger Reviewed-by: Fan Yong Signed-off-by: James Simmons --- fs/lustre/llite/dir.c | 17 ++++++++++------- fs/lustre/ptlrpc/pack_generic.c | 23 +++++++++++++++++++++++ include/uapi/linux/lustre/lustre_user.h | 2 ++ 3 files changed, 35 insertions(+), 7 deletions(-) diff --git a/fs/lustre/llite/dir.c b/fs/lustre/llite/dir.c index 8968110..8556fa2 100644 --- a/fs/lustre/llite/dir.c +++ b/fs/lustre/llite/dir.c @@ -402,7 +402,7 @@ static int ll_send_mgc_param(struct obd_export *mgc, char *string) * <0 if the creation is failed. */ static int ll_dir_setdirstripe(struct dentry *dparent, struct lmv_user_md *lump, - const char *dirname, umode_t mode) + size_t len, const char *dirname, umode_t mode) { struct inode *parent = dparent->d_inode; struct ptlrpc_request *request = NULL; @@ -420,7 +420,8 @@ static int ll_dir_setdirstripe(struct dentry *dparent, struct lmv_user_md *lump, }; int err; - if (unlikely(lump->lum_magic != LMV_USER_MAGIC)) + if (unlikely(lump->lum_magic != LMV_USER_MAGIC && + lump->lum_magic != LMV_USER_MAGIC_SPECIFIC)) return -EINVAL; CDEBUG(D_VFSTRACE, @@ -432,7 +433,8 @@ static int ll_dir_setdirstripe(struct dentry *dparent, struct lmv_user_md *lump, !(exp_connect_flags(sbi->ll_md_exp) & OBD_CONNECT_DIR_STRIPE)) return -EINVAL; - if (lump->lum_magic != cpu_to_le32(LMV_USER_MAGIC)) + if (lump->lum_magic != cpu_to_le32(LMV_USER_MAGIC) && + lump->lum_magic != cpu_to_le32(LMV_USER_MAGIC_SPECIFIC)) lustre_swab_lmv_user_md(lump); if (!IS_POSIXACL(parent) || !exp_connect_umask(ll_i2mdexp(parent))) @@ -459,7 +461,7 @@ static int ll_dir_setdirstripe(struct dentry *dparent, struct lmv_user_md *lump, } op_data->op_cli_flags |= CLI_SET_MEA; - err = md_create(sbi->ll_md_exp, op_data, lump, sizeof(*lump), mode, + err = md_create(sbi->ll_md_exp, op_data, lump, len, mode, from_kuid(&init_user_ns, current_fsuid()), from_kgid(&init_user_ns, current_fsgid()), current_cap(), 0, &request); @@ -1184,8 +1186,9 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) lum = (struct lmv_user_md *)data->ioc_inlbuf2; lumlen = data->ioc_inllen2; - if (lum->lum_magic != LMV_USER_MAGIC || - lumlen != sizeof(*lum)) { + if ((lum->lum_magic != LMV_USER_MAGIC && + lum->lum_magic != LMV_USER_MAGIC_SPECIFIC) || + lumlen < sizeof(*lum)) { CERROR("%s: wrong lum magic %x or size %d: rc = %d\n", filename, lum->lum_magic, lumlen, -EFAULT); rc = -EINVAL; @@ -1197,7 +1200,7 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) #else mode = data->ioc_type; #endif - rc = ll_dir_setdirstripe(dentry, lum, filename, mode); + rc = ll_dir_setdirstripe(dentry, lum, lumlen, filename, mode); lmv_out_free: kvfree(data); return rc; diff --git a/fs/lustre/ptlrpc/pack_generic.c b/fs/lustre/ptlrpc/pack_generic.c index 7fa2cf7..b9dd3abe 100644 --- a/fs/lustre/ptlrpc/pack_generic.c +++ b/fs/lustre/ptlrpc/pack_generic.c @@ -1960,14 +1960,37 @@ void lustre_swab_lmv_mds_md(union lmv_mds_md *lmm) } EXPORT_SYMBOL(lustre_swab_lmv_mds_md); +void lustre_swab_lmv_user_md_objects(struct lmv_user_mds_data *lmd, + int stripe_count) +{ + int i; + + for (i = 0; i < stripe_count; i++) + __swab32s(&(lmd[i].lum_mds)); +} +EXPORT_SYMBOL(lustre_swab_lmv_user_md_objects); + + void lustre_swab_lmv_user_md(struct lmv_user_md *lum) { + u32 count = lum->lum_stripe_count; + __swab32s(&lum->lum_magic); __swab32s(&lum->lum_stripe_count); __swab32s(&lum->lum_stripe_offset); __swab32s(&lum->lum_hash_type); __swab32s(&lum->lum_type); BUILD_BUG_ON(!offsetof(typeof(*lum), lum_padding1)); + switch (lum->lum_magic) { + case LMV_USER_MAGIC_SPECIFIC: + count = lum->lum_stripe_count; + /* fallthrough */ + case __swab32(LMV_USER_MAGIC_SPECIFIC): + lustre_swab_lmv_user_md_objects(lum->lum_objects, count); + break; + default: + break; + } } EXPORT_SYMBOL(lustre_swab_lmv_user_md); diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index e9bb3d6..7a44038 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -365,8 +365,10 @@ struct ll_ioc_lease { #define LOV_USER_MAGIC_COMP_V1 0x0BD60BD0 #define LMV_USER_MAGIC 0x0CD30CD0 /*default lmv magic*/ +#define LMV_USER_MAGIC_SPECIFIC 0x0CD40CD0 #define LOV_PATTERN_RAID0 0x001 + #define LOV_PATTERN_RAID1 0x002 #define LOV_PATTERN_MDT 0x100 #define LOV_PATTERN_CMOBD 0x200 From patchwork Mon Sep 30 18:55:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167187 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 214B516B1 for ; Mon, 30 Sep 2019 19:02:17 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 097D2224EF for ; Mon, 30 Sep 2019 19:02:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 097D2224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 2FD635E43E8; Mon, 30 Sep 2019 11:59:18 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id E35FF5C3B25 for ; Mon, 30 Sep 2019 11:57:28 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 5B9D31005C8E; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 588B9B4; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:54 -0400 Message-Id: <1569869810-23848-96-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 095/151] lustre: misc: update Intel copyright messages for 2017 X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Andreas Dilger Update copyright messages for files updated in 2016, excluding trivial patches. This brings use closely to the OpenSFS tree. WC-bug-id: https://jira.whamcloud.com/browse/LU-10308 Lustre-commit: ccabce23bd9e ("LU-10308 misc: update Intel copyright messages for 2017") Signed-off-by: Andreas Dilger Reviewed-on: https://review.whamcloud.com/30341 Reviewed-by: Jian Yu Reviewed-by: James Nunez Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/fid/fid_internal.h | 2 +- fs/lustre/fid/fid_request.c | 2 +- fs/lustre/fid/lproc_fid.c | 2 +- fs/lustre/include/cl_object.h | 2 +- fs/lustre/include/lprocfs_status.h | 2 +- fs/lustre/include/lu_object.h | 2 +- fs/lustre/include/lu_ref.h | 2 +- fs/lustre/include/lustre_acl.h | 2 ++ fs/lustre/include/lustre_disk.h | 2 +- fs/lustre/include/lustre_dlm.h | 2 +- fs/lustre/include/lustre_export.h | 2 +- fs/lustre/include/lustre_fid.h | 2 +- fs/lustre/include/lustre_handles.h | 2 +- fs/lustre/include/lustre_import.h | 2 +- fs/lustre/include/lustre_kernelcomm.h | 2 +- fs/lustre/include/lustre_lib.h | 2 +- fs/lustre/include/lustre_linkea.h | 2 +- fs/lustre/include/lustre_log.h | 2 +- fs/lustre/include/lustre_mdc.h | 2 +- fs/lustre/include/lustre_mds.h | 2 +- fs/lustre/include/lustre_net.h | 2 +- fs/lustre/include/lustre_osc.h | 2 +- fs/lustre/include/lustre_req_layout.h | 2 +- fs/lustre/include/lustre_sec.h | 2 +- fs/lustre/include/lustre_swab.h | 2 +- fs/lustre/include/obd.h | 2 +- fs/lustre/include/obd_class.h | 2 +- fs/lustre/include/obd_support.h | 2 +- fs/lustre/ldlm/ldlm_extent.c | 2 +- fs/lustre/ldlm/ldlm_flock.c | 2 +- fs/lustre/ldlm/ldlm_internal.h | 2 +- fs/lustre/ldlm/ldlm_lib.c | 2 +- fs/lustre/ldlm/ldlm_lock.c | 2 +- fs/lustre/ldlm/ldlm_lockd.c | 2 +- fs/lustre/ldlm/ldlm_plain.c | 2 +- fs/lustre/ldlm/ldlm_pool.c | 2 +- fs/lustre/ldlm/ldlm_request.c | 2 +- fs/lustre/ldlm/ldlm_resource.c | 2 +- fs/lustre/llite/dcache.c | 2 +- fs/lustre/llite/dir.c | 2 +- fs/lustre/llite/file.c | 2 +- fs/lustre/llite/glimpse.c | 2 +- fs/lustre/llite/lcommon_cl.c | 2 +- fs/lustre/llite/lcommon_misc.c | 2 +- fs/lustre/llite/llite_internal.h | 2 +- fs/lustre/llite/llite_lib.c | 2 +- fs/lustre/llite/llite_nfs.c | 2 +- fs/lustre/llite/lproc_llite.c | 2 +- fs/lustre/llite/namei.c | 2 +- fs/lustre/llite/rw.c | 2 +- fs/lustre/llite/rw26.c | 2 +- fs/lustre/llite/statahead.c | 2 +- fs/lustre/llite/vvp_internal.h | 2 +- fs/lustre/llite/vvp_io.c | 2 +- fs/lustre/llite/vvp_object.c | 2 +- fs/lustre/llite/vvp_page.c | 2 +- fs/lustre/llite/xattr.c | 2 +- fs/lustre/llite/xattr_cache.c | 2 +- fs/lustre/lmv/lmv_intent.c | 2 +- fs/lustre/lmv/lmv_internal.h | 2 +- fs/lustre/lmv/lmv_obd.c | 2 +- fs/lustre/lmv/lproc_lmv.c | 2 +- fs/lustre/lov/lov_cl_internal.h | 2 +- fs/lustre/lov/lov_dev.c | 2 +- fs/lustre/lov/lov_ea.c | 2 +- fs/lustre/lov/lov_internal.h | 2 +- fs/lustre/lov/lov_io.c | 2 +- fs/lustre/lov/lov_lock.c | 2 +- fs/lustre/lov/lov_merge.c | 2 +- fs/lustre/lov/lov_obd.c | 2 +- fs/lustre/lov/lov_object.c | 2 +- fs/lustre/lov/lov_offset.c | 2 +- fs/lustre/lov/lov_pack.c | 2 +- fs/lustre/lov/lov_page.c | 2 +- fs/lustre/lov/lov_pool.c | 2 +- fs/lustre/lov/lproc_lov.c | 2 +- fs/lustre/mdc/lproc_mdc.c | 2 +- fs/lustre/mdc/mdc_changelog.c | 2 ++ fs/lustre/mdc/mdc_dev.c | 2 +- fs/lustre/mdc/mdc_internal.h | 2 +- fs/lustre/mdc/mdc_lib.c | 2 +- fs/lustre/mdc/mdc_locks.c | 2 +- fs/lustre/mdc/mdc_reint.c | 2 +- fs/lustre/mdc/mdc_request.c | 2 +- fs/lustre/mgc/lproc_mgc.c | 2 +- fs/lustre/mgc/mgc_internal.h | 2 +- fs/lustre/mgc/mgc_request.c | 2 +- fs/lustre/obdclass/cl_internal.h | 2 +- fs/lustre/obdclass/cl_io.c | 2 +- fs/lustre/obdclass/cl_lock.c | 2 +- fs/lustre/obdclass/cl_object.c | 2 +- fs/lustre/obdclass/cl_page.c | 2 +- fs/lustre/obdclass/class_obd.c | 2 +- fs/lustre/obdclass/genops.c | 2 +- fs/lustre/obdclass/kernelcomm.c | 2 +- fs/lustre/obdclass/linkea.c | 2 +- fs/lustre/obdclass/llog.c | 2 +- fs/lustre/obdclass/llog_cat.c | 2 +- fs/lustre/obdclass/llog_internal.h | 2 +- fs/lustre/obdclass/llog_swab.c | 2 +- fs/lustre/obdclass/lprocfs_status.c | 2 +- fs/lustre/obdclass/lu_object.c | 2 +- fs/lustre/obdclass/lu_ref.c | 2 +- fs/lustre/obdclass/lustre_handles.c | 2 +- fs/lustre/obdclass/lustre_peer.c | 2 +- fs/lustre/obdclass/obd_config.c | 2 +- fs/lustre/obdclass/obd_mount.c | 2 +- fs/lustre/obdclass/obd_sysfs.c | 2 +- fs/lustre/obdclass/obdo.c | 2 +- fs/lustre/obdecho/debug.c | 2 +- fs/lustre/obdecho/echo_client.c | 2 +- fs/lustre/osc/lproc_osc.c | 2 +- fs/lustre/osc/osc_cache.c | 2 +- fs/lustre/osc/osc_dev.c | 2 +- fs/lustre/osc/osc_internal.h | 2 +- fs/lustre/osc/osc_io.c | 2 +- fs/lustre/osc/osc_lock.c | 2 +- fs/lustre/osc/osc_object.c | 2 +- fs/lustre/osc/osc_page.c | 2 +- fs/lustre/osc/osc_quota.c | 2 +- fs/lustre/osc/osc_request.c | 2 +- fs/lustre/ptlrpc/client.c | 2 +- fs/lustre/ptlrpc/events.c | 2 +- fs/lustre/ptlrpc/import.c | 2 +- fs/lustre/ptlrpc/layout.c | 2 +- fs/lustre/ptlrpc/llog_net.c | 2 +- fs/lustre/ptlrpc/lproc_ptlrpc.c | 2 +- fs/lustre/ptlrpc/niobuf.c | 2 +- fs/lustre/ptlrpc/pack_generic.c | 2 +- fs/lustre/ptlrpc/ptlrpc_internal.h | 2 +- fs/lustre/ptlrpc/ptlrpc_module.c | 2 +- fs/lustre/ptlrpc/ptlrpcd.c | 2 +- fs/lustre/ptlrpc/recover.c | 2 +- fs/lustre/ptlrpc/sec.c | 2 +- fs/lustre/ptlrpc/sec_bulk.c | 2 +- fs/lustre/ptlrpc/sec_config.c | 2 +- fs/lustre/ptlrpc/sec_gc.c | 2 +- fs/lustre/ptlrpc/sec_null.c | 2 +- fs/lustre/ptlrpc/sec_plain.c | 2 +- fs/lustre/ptlrpc/service.c | 2 +- include/linux/libcfs/libcfs.h | 2 +- include/linux/libcfs/libcfs_cpu.h | 2 +- include/linux/libcfs/libcfs_debug.h | 2 +- include/linux/libcfs/libcfs_hash.h | 2 +- include/linux/libcfs/libcfs_private.h | 2 +- include/linux/libcfs/libcfs_string.h | 2 +- include/linux/lnet/api.h | 2 +- include/linux/lnet/lib-lnet.h | 2 +- include/linux/lnet/lib-types.h | 2 +- include/uapi/linux/lnet/libcfs_debug.h | 2 +- include/uapi/linux/lnet/libcfs_ioctl.h | 2 ++ include/uapi/linux/lnet/lnet-dlc.h | 2 +- include/uapi/linux/lnet/lnet-types.h | 2 +- include/uapi/linux/lnet/lnetctl.h | 3 +++ include/uapi/linux/lnet/nidstr.h | 2 +- include/uapi/linux/lustre/lustre_fid.h | 2 +- include/uapi/linux/lustre/lustre_idl.h | 2 +- include/uapi/linux/lustre/lustre_ioctl.h | 2 +- include/uapi/linux/lustre/lustre_ostid.h | 2 +- include/uapi/linux/lustre/lustre_user.h | 2 +- net/lnet/klnds/o2iblnd/o2iblnd.c | 2 +- net/lnet/klnds/o2iblnd/o2iblnd.h | 2 +- net/lnet/klnds/o2iblnd/o2iblnd_cb.c | 2 +- net/lnet/klnds/o2iblnd/o2iblnd_modparams.c | 2 +- net/lnet/klnds/socklnd/socklnd.c | 2 +- net/lnet/klnds/socklnd/socklnd.h | 2 +- net/lnet/klnds/socklnd/socklnd_cb.c | 2 +- net/lnet/klnds/socklnd/socklnd_lib.c | 2 +- net/lnet/klnds/socklnd/socklnd_proto.c | 2 +- net/lnet/libcfs/debug.c | 2 +- net/lnet/libcfs/libcfs_cpu.c | 2 +- net/lnet/libcfs/libcfs_string.c | 2 +- net/lnet/libcfs/module.c | 2 +- net/lnet/libcfs/tracefile.c | 2 +- net/lnet/libcfs/tracefile.h | 2 +- net/lnet/lnet/acceptor.c | 2 +- net/lnet/lnet/api-ni.c | 2 +- net/lnet/lnet/config.c | 2 +- net/lnet/lnet/lib-md.c | 2 +- net/lnet/lnet/lib-move.c | 2 +- net/lnet/lnet/lib-msg.c | 2 +- net/lnet/lnet/lib-ptl.c | 2 +- net/lnet/lnet/lib-socket.c | 2 +- net/lnet/lnet/lo.c | 2 ++ net/lnet/lnet/module.c | 2 +- net/lnet/lnet/net_fault.c | 2 +- net/lnet/lnet/nidstrings.c | 2 +- net/lnet/lnet/peer.c | 2 +- net/lnet/lnet/router.c | 2 +- net/lnet/lnet/router_proc.c | 2 +- net/lnet/selftest/brw_test.c | 2 +- net/lnet/selftest/framework.c | 2 +- net/lnet/selftest/rpc.c | 2 +- net/lnet/selftest/selftest.h | 2 +- 194 files changed, 200 insertions(+), 189 deletions(-) diff --git a/fs/lustre/fid/fid_internal.h b/fs/lustre/fid/fid_internal.h index 14569e9..7bb1283 100644 --- a/fs/lustre/fid/fid_internal.h +++ b/fs/lustre/fid/fid_internal.h @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, Intel Corporation. + * Copyright (c) 2012, 2014 Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/fid/fid_request.c b/fs/lustre/fid/fid_request.c index cdf5402..861ffbe 100644 --- a/fs/lustre/fid/fid_request.c +++ b/fs/lustre/fid/fid_request.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/fid/lproc_fid.c b/fs/lustre/fid/lproc_fid.c index e2e47df..19c05ad 100644 --- a/fs/lustre/fid/lproc_fid.c +++ b/fs/lustre/fid/lproc_fid.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/include/cl_object.h b/fs/lustre/include/cl_object.h index 946e6e1..1088fde 100644 --- a/fs/lustre/include/cl_object.h +++ b/fs/lustre/include/cl_object.h @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/include/lprocfs_status.h b/fs/lustre/include/lprocfs_status.h index 54e4eda..10d541f 100644 --- a/fs/lustre/include/lprocfs_status.h +++ b/fs/lustre/include/lprocfs_status.h @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/include/lu_object.h b/fs/lustre/include/lu_object.h index 9427ad4..a938e64 100644 --- a/fs/lustre/include/lu_object.h +++ b/fs/lustre/include/lu_object.h @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/include/lu_ref.h b/fs/lustre/include/lu_ref.h index ad0c24d..493df95 100644 --- a/fs/lustre/include/lu_ref.h +++ b/fs/lustre/include/lu_ref.h @@ -3,7 +3,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, Intel Corporation. + * Copyright (c) 2012, 2013, Intel Corporation. * * Author: Nikita Danilov * diff --git a/fs/lustre/include/lustre_acl.h b/fs/lustre/include/lustre_acl.h index aff77b0..067ed3f 100644 --- a/fs/lustre/include/lustre_acl.h +++ b/fs/lustre/include/lustre_acl.h @@ -23,6 +23,8 @@ /* * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. + * + * Copyright (c) 2014, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/include/lustre_disk.h b/fs/lustre/include/lustre_disk.h index 07c074e..8b4a243 100644 --- a/fs/lustre/include/lustre_disk.h +++ b/fs/lustre/include/lustre_disk.h @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/include/lustre_dlm.h b/fs/lustre/include/lustre_dlm.h index fbcd981..b1a37f0 100644 --- a/fs/lustre/include/lustre_dlm.h +++ b/fs/lustre/include/lustre_dlm.h @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2010, 2015, Intel Corporation. + * Copyright (c) 2010, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/include/lustre_export.h b/fs/lustre/include/lustre_export.h index 9e2878e..544166f 100644 --- a/fs/lustre/include/lustre_export.h +++ b/fs/lustre/include/lustre_export.h @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/include/lustre_fid.h b/fs/lustre/include/lustre_fid.h index b469295..e257ad5 100644 --- a/fs/lustre/include/lustre_fid.h +++ b/fs/lustre/include/lustre_fid.h @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/include/lustre_handles.h b/fs/lustre/include/lustre_handles.h index 6836808..d4ec795 100644 --- a/fs/lustre/include/lustre_handles.h +++ b/fs/lustre/include/lustre_handles.h @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2012, 2014, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/include/lustre_import.h b/fs/lustre/include/lustre_import.h index d9a8d9a..6a2d402 100644 --- a/fs/lustre/include/lustre_import.h +++ b/fs/lustre/include/lustre_import.h @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2016, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/include/lustre_kernelcomm.h b/fs/lustre/include/lustre_kernelcomm.h index 2c8ce19..bd5376b 100644 --- a/fs/lustre/include/lustre_kernelcomm.h +++ b/fs/lustre/include/lustre_kernelcomm.h @@ -24,7 +24,7 @@ * Copyright (c) 2009, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2013 Intel Corporation. + * Copyright (c) 2013, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/include/lustre_lib.h b/fs/lustre/include/lustre_lib.h index 64d2e91..fc4b4c5 100644 --- a/fs/lustre/include/lustre_lib.h +++ b/fs/lustre/include/lustre_lib.h @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/include/lustre_linkea.h b/fs/lustre/include/lustre_linkea.h index a028d91..cd0512b 100644 --- a/fs/lustre/include/lustre_linkea.h +++ b/fs/lustre/include/lustre_linkea.h @@ -21,7 +21,7 @@ * GPL HEADER END */ /* - * Copyright (c) 2013, 2014, Intel Corporation. + * Copyright (c) 2013, 2017, Intel Corporation. * Use is subject to license terms. * * Author: di wang diff --git a/fs/lustre/include/lustre_log.h b/fs/lustre/include/lustre_log.h index ab0262c..99c6305 100644 --- a/fs/lustre/include/lustre_log.h +++ b/fs/lustre/include/lustre_log.h @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/include/lustre_mdc.h b/fs/lustre/include/lustre_mdc.h index 63a7413..208989f 100644 --- a/fs/lustre/include/lustre_mdc.h +++ b/fs/lustre/include/lustre_mdc.h @@ -25,7 +25,7 @@ * Use is subject to license terms. */ /* - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2014, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/include/lustre_mds.h b/fs/lustre/include/lustre_mds.h index df178cc..18fe0e3 100644 --- a/fs/lustre/include/lustre_mds.h +++ b/fs/lustre/include/lustre_mds.h @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2014, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/include/lustre_net.h b/fs/lustre/include/lustre_net.h index 200422f..baa1eb8 100644 --- a/fs/lustre/include/lustre_net.h +++ b/fs/lustre/include/lustre_net.h @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2010, 2015, Intel Corporation. + * Copyright (c) 2010, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/include/lustre_osc.h b/fs/lustre/include/lustre_osc.h index a81e802..cba9d43 100644 --- a/fs/lustre/include/lustre_osc.h +++ b/fs/lustre/include/lustre_osc.h @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/include/lustre_req_layout.h b/fs/lustre/include/lustre_req_layout.h index 48f454c..2348569 100644 --- a/fs/lustre/include/lustre_req_layout.h +++ b/fs/lustre/include/lustre_req_layout.h @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/include/lustre_sec.h b/fs/lustre/include/lustre_sec.h index dabc663..99702fd 100644 --- a/fs/lustre/include/lustre_sec.h +++ b/fs/lustre/include/lustre_sec.h @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/include/lustre_swab.h b/fs/lustre/include/lustre_swab.h index 79cacf4..7e96640 100644 --- a/fs/lustre/include/lustre_swab.h +++ b/fs/lustre/include/lustre_swab.h @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2014, Intel Corporation. + * Copyright (c) 2017, Intel Corporation. * * Copyright 2015 Cray Inc, all rights reserved. * Author: Ben Evans. diff --git a/fs/lustre/include/obd.h b/fs/lustre/include/obd.h index f252ff0..309a168 100644 --- a/fs/lustre/include/obd.h +++ b/fs/lustre/include/obd.h @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/include/obd_class.h b/fs/lustre/include/obd_class.h index 38b69e8..f977b2e 100644 --- a/fs/lustre/include/obd_class.h +++ b/fs/lustre/include/obd_class.h @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/include/obd_support.h b/fs/lustre/include/obd_support.h index 6a01af4..3e273f8 100644 --- a/fs/lustre/include/obd_support.h +++ b/fs/lustre/include/obd_support.h @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ldlm/ldlm_extent.c b/fs/lustre/ldlm/ldlm_extent.c index 66616b0..77d3e79 100644 --- a/fs/lustre/ldlm/ldlm_extent.c +++ b/fs/lustre/ldlm/ldlm_extent.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2010, 2012, Intel Corporation. + * Copyright (c) 2010, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ldlm/ldlm_flock.c b/fs/lustre/ldlm/ldlm_flock.c index d2b4f0d..720362f 100644 --- a/fs/lustre/ldlm/ldlm_flock.c +++ b/fs/lustre/ldlm/ldlm_flock.c @@ -28,7 +28,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2010, 2012, Intel Corporation. + * Copyright (c) 2010, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ldlm/ldlm_internal.h b/fs/lustre/ldlm/ldlm_internal.h index b9d2e9a..6e54521 100644 --- a/fs/lustre/ldlm/ldlm_internal.h +++ b/fs/lustre/ldlm/ldlm_internal.h @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ldlm/ldlm_lib.c b/fs/lustre/ldlm/ldlm_lib.c index 3a7b3fb..45dc70d 100644 --- a/fs/lustre/ldlm/ldlm_lib.c +++ b/fs/lustre/ldlm/ldlm_lib.c @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2010, 2015, Intel Corporation. + * Copyright (c) 2010, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ldlm/ldlm_lock.c b/fs/lustre/ldlm/ldlm_lock.c index ef6c5e8..f0657d9 100644 --- a/fs/lustre/ldlm/ldlm_lock.c +++ b/fs/lustre/ldlm/ldlm_lock.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2010, 2015, Intel Corporation. + * Copyright (c) 2010, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ldlm/ldlm_lockd.c b/fs/lustre/ldlm/ldlm_lockd.c index 2c21e7d..7e1ae0c 100644 --- a/fs/lustre/ldlm/ldlm_lockd.c +++ b/fs/lustre/ldlm/ldlm_lockd.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2010, 2015, Intel Corporation. + * Copyright (c) 2010, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ldlm/ldlm_plain.c b/fs/lustre/ldlm/ldlm_plain.c index 33b5a3f..e7357ee 100644 --- a/fs/lustre/ldlm/ldlm_plain.c +++ b/fs/lustre/ldlm/ldlm_plain.c @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ldlm/ldlm_pool.c b/fs/lustre/ldlm/ldlm_pool.c index 3dfbde5..25b1ada 100644 --- a/fs/lustre/ldlm/ldlm_pool.c +++ b/fs/lustre/ldlm/ldlm_pool.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2010, 2015, Intel Corporation. + * Copyright (c) 2010, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ldlm/ldlm_request.c b/fs/lustre/ldlm/ldlm_request.c index eb99279..4c6db33 100644 --- a/fs/lustre/ldlm/ldlm_request.c +++ b/fs/lustre/ldlm/ldlm_request.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2010, 2015, Intel Corporation. + * Copyright (c) 2010, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ldlm/ldlm_resource.c b/fs/lustre/ldlm/ldlm_resource.c index de0ae1a..4e3c6e7 100644 --- a/fs/lustre/ldlm/ldlm_resource.c +++ b/fs/lustre/ldlm/ldlm_resource.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2010, 2015, Intel Corporation. + * Copyright (c) 2010, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/llite/dcache.c b/fs/lustre/llite/dcache.c index 11b82c63..2dfe12a 100644 --- a/fs/lustre/llite/dcache.c +++ b/fs/lustre/llite/dcache.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/llite/dir.c b/fs/lustre/llite/dir.c index 8556fa2..d3ef669 100644 --- a/fs/lustre/llite/dir.c +++ b/fs/lustre/llite/dir.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 47e270d..f1db600 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/llite/glimpse.c b/fs/lustre/llite/glimpse.c index 13a2621..de1a31f 100644 --- a/fs/lustre/llite/glimpse.c +++ b/fs/lustre/llite/glimpse.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/llite/lcommon_cl.c b/fs/lustre/llite/lcommon_cl.c index 24e58c2..988855b 100644 --- a/fs/lustre/llite/lcommon_cl.c +++ b/fs/lustre/llite/lcommon_cl.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/llite/lcommon_misc.c b/fs/lustre/llite/lcommon_misc.c index cafcb4d..29daf5b 100644 --- a/fs/lustre/llite/lcommon_misc.c +++ b/fs/lustre/llite/lcommon_misc.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/llite/llite_internal.h b/fs/lustre/llite/llite_internal.h index bdc90bb..ded310b 100644 --- a/fs/lustre/llite/llite_internal.h +++ b/fs/lustre/llite/llite_internal.h @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index 9e4c797..d4ff2bf 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/llite/llite_nfs.c b/fs/lustre/llite/llite_nfs.c index 64bff0a..d6643d0 100644 --- a/fs/lustre/llite/llite_nfs.c +++ b/fs/lustre/llite/llite_nfs.c @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/llite/lproc_llite.c b/fs/lustre/llite/lproc_llite.c index 69f4f25..81c8fe6 100644 --- a/fs/lustre/llite/lproc_llite.c +++ b/fs/lustre/llite/lproc_llite.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/llite/namei.c b/fs/lustre/llite/namei.c index 355a500..1b5e270 100644 --- a/fs/lustre/llite/namei.c +++ b/fs/lustre/llite/namei.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/llite/rw.c b/fs/lustre/llite/rw.c index e2e47174..02b9a0c 100644 --- a/fs/lustre/llite/rw.c +++ b/fs/lustre/llite/rw.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/llite/rw26.c b/fs/lustre/llite/rw26.c index 35d39fe..4d86c4b 100644 --- a/fs/lustre/llite/rw26.c +++ b/fs/lustre/llite/rw26.c @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/llite/statahead.c b/fs/lustre/llite/statahead.c index 2b7015d..99b3fee 100644 --- a/fs/lustre/llite/statahead.c +++ b/fs/lustre/llite/statahead.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/llite/vvp_internal.h b/fs/lustre/llite/vvp_internal.h index 923f866..96f10d2 100644 --- a/fs/lustre/llite/vvp_internal.h +++ b/fs/lustre/llite/vvp_internal.h @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2013, 2015, Intel Corporation. + * Copyright (c) 2013, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/llite/vvp_io.c b/fs/lustre/llite/vvp_io.c index c8a1ff1..6145064 100644 --- a/fs/lustre/llite/vvp_io.c +++ b/fs/lustre/llite/vvp_io.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/llite/vvp_object.c b/fs/lustre/llite/vvp_object.c index 6606459..c750a80 100644 --- a/fs/lustre/llite/vvp_object.c +++ b/fs/lustre/llite/vvp_object.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/llite/vvp_page.c b/fs/lustre/llite/vvp_page.c index 7b07eae..beebea8 100644 --- a/fs/lustre/llite/vvp_page.c +++ b/fs/lustre/llite/vvp_page.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/llite/xattr.c b/fs/lustre/llite/xattr.c index d604c5b..378c190 100644 --- a/fs/lustre/llite/xattr.c +++ b/fs/lustre/llite/xattr.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/llite/xattr_cache.c b/fs/lustre/llite/xattr_cache.c index 001bdba..c642444 100644 --- a/fs/lustre/llite/xattr_cache.c +++ b/fs/lustre/llite/xattr_cache.c @@ -2,7 +2,7 @@ /* * Copyright 2012 Xyratex Technology Limited * - * Copyright (c) 2013, 2015, Intel Corporation. + * Copyright (c) 2013, 2017, Intel Corporation. * * Author: Andrew Perepechko * diff --git a/fs/lustre/lmv/lmv_intent.c b/fs/lustre/lmv/lmv_intent.c index 818bb72..355a2af 100644 --- a/fs/lustre/lmv/lmv_intent.c +++ b/fs/lustre/lmv/lmv_intent.c @@ -24,7 +24,7 @@ * Copyright (c) 2004, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2016, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/lmv/lmv_internal.h b/fs/lustre/lmv/lmv_internal.h index c0881ff..6794f11 100644 --- a/fs/lustre/lmv/lmv_internal.h +++ b/fs/lustre/lmv/lmv_internal.h @@ -24,7 +24,7 @@ * Copyright (c) 2004, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2016, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/lmv/lmv_obd.c b/fs/lustre/lmv/lmv_obd.c index b6459f5..9a1064b 100644 --- a/fs/lustre/lmv/lmv_obd.c +++ b/fs/lustre/lmv/lmv_obd.c @@ -24,7 +24,7 @@ * Copyright (c) 2004, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/lmv/lproc_lmv.c b/fs/lustre/lmv/lproc_lmv.c index e40473c..170ed564 100644 --- a/fs/lustre/lmv/lproc_lmv.c +++ b/fs/lustre/lmv/lproc_lmv.c @@ -24,7 +24,7 @@ * Copyright (c) 2004, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/lov/lov_cl_internal.h b/fs/lustre/lov/lov_cl_internal.h index 4472d6f..4ab8fba 100644 --- a/fs/lustre/lov/lov_cl_internal.h +++ b/fs/lustre/lov/lov_cl_internal.h @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015 Intel Corporation. + * Copyright (c) 2012, 2017 Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/lov/lov_dev.c b/fs/lustre/lov/lov_dev.c index 46d2e1e..a4e6b91 100644 --- a/fs/lustre/lov/lov_dev.c +++ b/fs/lustre/lov/lov_dev.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/lov/lov_ea.c b/fs/lustre/lov/lov_ea.c index 081a04f..41308d3 100644 --- a/fs/lustre/lov/lov_ea.c +++ b/fs/lustre/lov/lov_ea.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/lov/lov_internal.h b/fs/lustre/lov/lov_internal.h index b3712d2..5dba8d3 100644 --- a/fs/lustre/lov/lov_internal.h +++ b/fs/lustre/lov/lov_internal.h @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/lov/lov_io.c b/fs/lustre/lov/lov_io.c index 75fa646..54e2d7d 100644 --- a/fs/lustre/lov/lov_io.c +++ b/fs/lustre/lov/lov_io.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/lov/lov_lock.c b/fs/lustre/lov/lov_lock.c index 0c10c3a..7dae13f 100644 --- a/fs/lustre/lov/lov_lock.c +++ b/fs/lustre/lov/lov_lock.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/lov/lov_merge.c b/fs/lustre/lov/lov_merge.c index a8c23fe..7728630 100644 --- a/fs/lustre/lov/lov_merge.c +++ b/fs/lustre/lov/lov_merge.c @@ -24,7 +24,7 @@ * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/lov/lov_obd.c b/fs/lustre/lov/lov_obd.c index 4ced5f7..acfee93 100644 --- a/fs/lustre/lov/lov_obd.c +++ b/fs/lustre/lov/lov_obd.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/lov/lov_object.c b/fs/lustre/lov/lov_object.c index 49facf3..a73f2ac 100644 --- a/fs/lustre/lov/lov_object.c +++ b/fs/lustre/lov/lov_object.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/lov/lov_offset.c b/fs/lustre/lov/lov_offset.c index 56a2d7b..bb67d82 100644 --- a/fs/lustre/lov/lov_offset.c +++ b/fs/lustre/lov/lov_offset.c @@ -24,7 +24,7 @@ * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/lov/lov_pack.c b/fs/lustre/lov/lov_pack.c index aac5048..3dbc6aa 100644 --- a/fs/lustre/lov/lov_pack.c +++ b/fs/lustre/lov/lov_pack.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/lov/lov_page.c b/fs/lustre/lov/lov_page.c index 22548f4..3f08da7 100644 --- a/fs/lustre/lov/lov_page.c +++ b/fs/lustre/lov/lov_page.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/lov/lov_pool.c b/fs/lustre/lov/lov_pool.c index 78a7dd2..60565b9 100644 --- a/fs/lustre/lov/lov_pool.c +++ b/fs/lustre/lov/lov_pool.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/lov/lproc_lov.c b/fs/lustre/lov/lproc_lov.c index 4fecd4e..c528a8b 100644 --- a/fs/lustre/lov/lproc_lov.c +++ b/fs/lustre/lov/lproc_lov.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015 Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/mdc/lproc_mdc.c b/fs/lustre/mdc/lproc_mdc.c index 9092a97..2af2e57 100644 --- a/fs/lustre/mdc/lproc_mdc.c +++ b/fs/lustre/mdc/lproc_mdc.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/mdc/mdc_changelog.c b/fs/lustre/mdc/mdc_changelog.c index b6a26db..d9e861e 100644 --- a/fs/lustre/mdc/mdc_changelog.c +++ b/fs/lustre/mdc/mdc_changelog.c @@ -24,6 +24,8 @@ * Copyright (c) 2017, Commissariat a l'Energie Atomique et aux Energies * Alternatives. * + * Copyright (c) 2017, Intel Corporation. + * * Author: Henri Doreau */ diff --git a/fs/lustre/mdc/mdc_dev.c b/fs/lustre/mdc/mdc_dev.c index de21d7d..4ff5cc6 100644 --- a/fs/lustre/mdc/mdc_dev.c +++ b/fs/lustre/mdc/mdc_dev.c @@ -21,7 +21,7 @@ * GPL HEADER END */ /* - * Copyright (c) 2017 Intel Corporation. + * Copyright (c) 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/mdc/mdc_internal.h b/fs/lustre/mdc/mdc_internal.h index 669cb1c..079539d 100644 --- a/fs/lustre/mdc/mdc_internal.h +++ b/fs/lustre/mdc/mdc_internal.h @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015 Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/mdc/mdc_lib.c b/fs/lustre/mdc/mdc_lib.c index df66ed1..dc5dac4 100644 --- a/fs/lustre/mdc/mdc_lib.c +++ b/fs/lustre/mdc/mdc_lib.c @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/mdc/mdc_locks.c b/fs/lustre/mdc/mdc_locks.c index 9c4dcad..f297a98 100644 --- a/fs/lustre/mdc/mdc_locks.c +++ b/fs/lustre/mdc/mdc_locks.c @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/mdc/mdc_reint.c b/fs/lustre/mdc/mdc_reint.c index 8e0bd0a..d6216d6 100644 --- a/fs/lustre/mdc/mdc_reint.c +++ b/fs/lustre/mdc/mdc_reint.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/mdc/mdc_request.c b/fs/lustre/mdc/mdc_request.c index 29a4886..95698c3 100644 --- a/fs/lustre/mdc/mdc_request.c +++ b/fs/lustre/mdc/mdc_request.c @@ -24,7 +24,7 @@ * Copyright (c) 2001, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/mgc/lproc_mgc.c b/fs/lustre/mgc/lproc_mgc.c index 574233e..d977d51 100644 --- a/fs/lustre/mgc/lproc_mgc.c +++ b/fs/lustre/mgc/lproc_mgc.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/mgc/mgc_internal.h b/fs/lustre/mgc/mgc_internal.h index 0a91624..7a6e77b 100644 --- a/fs/lustre/mgc/mgc_internal.h +++ b/fs/lustre/mgc/mgc_internal.h @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/mgc/mgc_request.c b/fs/lustre/mgc/mgc_request.c index ca7f47a..ca4b8a9 100644 --- a/fs/lustre/mgc/mgc_request.c +++ b/fs/lustre/mgc/mgc_request.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/obdclass/cl_internal.h b/fs/lustre/obdclass/cl_internal.h index dc6bf10..4fec0f4 100644 --- a/fs/lustre/obdclass/cl_internal.h +++ b/fs/lustre/obdclass/cl_internal.h @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/obdclass/cl_io.c b/fs/lustre/obdclass/cl_io.c index ca94eb3..a98be15 100644 --- a/fs/lustre/obdclass/cl_io.c +++ b/fs/lustre/obdclass/cl_io.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/obdclass/cl_lock.c b/fs/lustre/obdclass/cl_lock.c index 07d0dc8..0c02546 100644 --- a/fs/lustre/obdclass/cl_lock.c +++ b/fs/lustre/obdclass/cl_lock.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2014, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/obdclass/cl_object.c b/fs/lustre/obdclass/cl_object.c index 4a40121..f0ae34f 100644 --- a/fs/lustre/obdclass/cl_object.c +++ b/fs/lustre/obdclass/cl_object.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/obdclass/cl_page.c b/fs/lustre/obdclass/cl_page.c index 163f4de..8ea63f7 100644 --- a/fs/lustre/obdclass/cl_page.c +++ b/fs/lustre/obdclass/cl_page.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/obdclass/class_obd.c b/fs/lustre/obdclass/class_obd.c index 8b6032a..7e436af 100644 --- a/fs/lustre/obdclass/class_obd.c +++ b/fs/lustre/obdclass/class_obd.c @@ -24,7 +24,7 @@ * Copyright (c) 1999, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/obdclass/genops.c b/fs/lustre/obdclass/genops.c index 1ecceeb..6491032 100644 --- a/fs/lustre/obdclass/genops.c +++ b/fs/lustre/obdclass/genops.c @@ -24,7 +24,7 @@ * Copyright (c) 1999, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/obdclass/kernelcomm.c b/fs/lustre/obdclass/kernelcomm.c index e92a52c..bc9b0f2 100644 --- a/fs/lustre/obdclass/kernelcomm.c +++ b/fs/lustre/obdclass/kernelcomm.c @@ -24,7 +24,7 @@ * Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/obdclass/linkea.c b/fs/lustre/obdclass/linkea.c index b1aa80a..11902e1 100644 --- a/fs/lustre/obdclass/linkea.c +++ b/fs/lustre/obdclass/linkea.c @@ -21,7 +21,7 @@ * GPL HEADER END */ /* - * Copyright (c) 2013, 2014, Intel Corporation. + * Copyright (c) 2013, 2017, Intel Corporation. * Use is subject to license terms. * * Author: Di Wang diff --git a/fs/lustre/obdclass/llog.c b/fs/lustre/obdclass/llog.c index e9a6514..3187a9c 100644 --- a/fs/lustre/obdclass/llog.c +++ b/fs/lustre/obdclass/llog.c @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/obdclass/llog_cat.c b/fs/lustre/obdclass/llog_cat.c index 4536f54..970f2e9 100644 --- a/fs/lustre/obdclass/llog_cat.c +++ b/fs/lustre/obdclass/llog_cat.c @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/obdclass/llog_internal.h b/fs/lustre/obdclass/llog_internal.h index 545119e..365bac9 100644 --- a/fs/lustre/obdclass/llog_internal.h +++ b/fs/lustre/obdclass/llog_internal.h @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/obdclass/llog_swab.c b/fs/lustre/obdclass/llog_swab.c index 49024fd..c18ca95 100644 --- a/fs/lustre/obdclass/llog_swab.c +++ b/fs/lustre/obdclass/llog_swab.c @@ -24,7 +24,7 @@ * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/obdclass/lprocfs_status.c b/fs/lustre/obdclass/lprocfs_status.c index a9617e5..dbbaa92 100644 --- a/fs/lustre/obdclass/lprocfs_status.c +++ b/fs/lustre/obdclass/lprocfs_status.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/obdclass/lu_object.c b/fs/lustre/obdclass/lu_object.c index 4302482..d8dfc721 100644 --- a/fs/lustre/obdclass/lu_object.c +++ b/fs/lustre/obdclass/lu_object.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/obdclass/lu_ref.c b/fs/lustre/obdclass/lu_ref.c index f67cb89..9b32c3c 100644 --- a/fs/lustre/obdclass/lu_ref.c +++ b/fs/lustre/obdclass/lu_ref.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/obdclass/lustre_handles.c b/fs/lustre/obdclass/lustre_handles.c index 0674afb..7c63d9c 100644 --- a/fs/lustre/obdclass/lustre_handles.c +++ b/fs/lustre/obdclass/lustre_handles.c @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/obdclass/lustre_peer.c b/fs/lustre/obdclass/lustre_peer.c index 69a3b32..58b6e670 100644 --- a/fs/lustre/obdclass/lustre_peer.c +++ b/fs/lustre/obdclass/lustre_peer.c @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/obdclass/obd_config.c b/fs/lustre/obdclass/obd_config.c index 1485cc7..550cee0 100644 --- a/fs/lustre/obdclass/obd_config.c +++ b/fs/lustre/obdclass/obd_config.c @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/obdclass/obd_mount.c b/fs/lustre/obdclass/obd_mount.c index de81baa..5cf404c 100644 --- a/fs/lustre/obdclass/obd_mount.c +++ b/fs/lustre/obdclass/obd_mount.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/obdclass/obd_sysfs.c b/fs/lustre/obdclass/obd_sysfs.c index e9e0bcf..bac8e7c5 100644 --- a/fs/lustre/obdclass/obd_sysfs.c +++ b/fs/lustre/obdclass/obd_sysfs.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/obdclass/obdo.c b/fs/lustre/obdclass/obdo.c index 7c65d0d..1926896 100644 --- a/fs/lustre/obdclass/obdo.c +++ b/fs/lustre/obdclass/obdo.c @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/obdecho/debug.c b/fs/lustre/obdecho/debug.c index a8f773a..130d3bf 100644 --- a/fs/lustre/obdecho/debug.c +++ b/fs/lustre/obdecho/debug.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2014, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/obdecho/echo_client.c b/fs/lustre/obdecho/echo_client.c index baf34c8..4ae21fb 100644 --- a/fs/lustre/obdecho/echo_client.c +++ b/fs/lustre/obdecho/echo_client.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/osc/lproc_osc.c b/fs/lustre/osc/lproc_osc.c index 2f122a2..6661092 100644 --- a/fs/lustre/osc/lproc_osc.c +++ b/fs/lustre/osc/lproc_osc.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/osc/osc_cache.c b/fs/lustre/osc/osc_cache.c index e387b7a..68498af 100644 --- a/fs/lustre/osc/osc_cache.c +++ b/fs/lustre/osc/osc_cache.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. * */ /* diff --git a/fs/lustre/osc/osc_dev.c b/fs/lustre/osc/osc_dev.c index fb1d31c..3d0687a 100644 --- a/fs/lustre/osc/osc_dev.c +++ b/fs/lustre/osc/osc_dev.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/osc/osc_internal.h b/fs/lustre/osc/osc_internal.h index 2e0c521..1194033 100644 --- a/fs/lustre/osc/osc_internal.h +++ b/fs/lustre/osc/osc_internal.h @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/osc/osc_io.c b/fs/lustre/osc/osc_io.c index 0545e23..d8fa8cc 100644 --- a/fs/lustre/osc/osc_io.c +++ b/fs/lustre/osc/osc_io.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/osc/osc_lock.c b/fs/lustre/osc/osc_lock.c index 1ede75c..b7b33fb 100644 --- a/fs/lustre/osc/osc_lock.c +++ b/fs/lustre/osc/osc_lock.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/osc/osc_object.c b/fs/lustre/osc/osc_object.c index 76e7e33..4cd6e15 100644 --- a/fs/lustre/osc/osc_object.c +++ b/fs/lustre/osc/osc_object.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/osc/osc_page.c b/fs/lustre/osc/osc_page.c index 6e49cf4..f5b6f73 100644 --- a/fs/lustre/osc/osc_page.c +++ b/fs/lustre/osc/osc_page.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/osc/osc_quota.c b/fs/lustre/osc/osc_quota.c index 3909e11..cb5ddef 100644 --- a/fs/lustre/osc/osc_quota.c +++ b/fs/lustre/osc/osc_quota.c @@ -19,7 +19,7 @@ /* * Copyright (c) 2009, 2010, Oracle and/or its affiliates. All rights reserved. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. * * Code originally extracted from quota directory */ diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index 2dce087..0f7ac2d 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ptlrpc/client.c b/fs/lustre/ptlrpc/client.c index 0499368..e3aee32 100644 --- a/fs/lustre/ptlrpc/client.c +++ b/fs/lustre/ptlrpc/client.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ptlrpc/events.c b/fs/lustre/ptlrpc/events.c index fc3c7f0..93a59b8 100644 --- a/fs/lustre/ptlrpc/events.c +++ b/fs/lustre/ptlrpc/events.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015 Intel Corporation. + * Copyright (c) 2012, 2017 Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ptlrpc/import.c b/fs/lustre/ptlrpc/import.c index 6e2af7f..3c56798 100644 --- a/fs/lustre/ptlrpc/import.c +++ b/fs/lustre/ptlrpc/import.c @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ptlrpc/layout.c b/fs/lustre/ptlrpc/layout.c index 0978329..4909b30 100644 --- a/fs/lustre/ptlrpc/layout.c +++ b/fs/lustre/ptlrpc/layout.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ptlrpc/llog_net.c b/fs/lustre/ptlrpc/llog_net.c index b871d9e..1d86fb3 100644 --- a/fs/lustre/ptlrpc/llog_net.c +++ b/fs/lustre/ptlrpc/llog_net.c @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, Intel Corporation. + * Copyright (c) 2012, 2013, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ptlrpc/lproc_ptlrpc.c b/fs/lustre/ptlrpc/lproc_ptlrpc.c index 02713076..163ee54 100644 --- a/fs/lustre/ptlrpc/lproc_ptlrpc.c +++ b/fs/lustre/ptlrpc/lproc_ptlrpc.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ptlrpc/niobuf.c b/fs/lustre/ptlrpc/niobuf.c index 7bf78ee..47d4159 100644 --- a/fs/lustre/ptlrpc/niobuf.c +++ b/fs/lustre/ptlrpc/niobuf.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ptlrpc/pack_generic.c b/fs/lustre/ptlrpc/pack_generic.c index b9dd3abe..1ba2772 100644 --- a/fs/lustre/ptlrpc/pack_generic.c +++ b/fs/lustre/ptlrpc/pack_generic.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ptlrpc/ptlrpc_internal.h b/fs/lustre/ptlrpc/ptlrpc_internal.h index 40cf7b7..f84d278 100644 --- a/fs/lustre/ptlrpc/ptlrpc_internal.h +++ b/fs/lustre/ptlrpc/ptlrpc_internal.h @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ptlrpc/ptlrpc_module.c b/fs/lustre/ptlrpc/ptlrpc_module.c index 6888df9..0c988ae 100644 --- a/fs/lustre/ptlrpc/ptlrpc_module.c +++ b/fs/lustre/ptlrpc/ptlrpc_module.c @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2014, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ptlrpc/ptlrpcd.c b/fs/lustre/ptlrpc/ptlrpcd.c index 0b73d58..c0b091c 100644 --- a/fs/lustre/ptlrpc/ptlrpcd.c +++ b/fs/lustre/ptlrpc/ptlrpcd.c @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ptlrpc/recover.c b/fs/lustre/ptlrpc/recover.c index 5548049..7c09c4e 100644 --- a/fs/lustre/ptlrpc/recover.c +++ b/fs/lustre/ptlrpc/recover.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ptlrpc/sec.c b/fs/lustre/ptlrpc/sec.c index a0d6732..21055a3 100644 --- a/fs/lustre/ptlrpc/sec.c +++ b/fs/lustre/ptlrpc/sec.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ptlrpc/sec_bulk.c b/fs/lustre/ptlrpc/sec_bulk.c index 755781c..e170da1 100644 --- a/fs/lustre/ptlrpc/sec_bulk.c +++ b/fs/lustre/ptlrpc/sec_bulk.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ptlrpc/sec_config.c b/fs/lustre/ptlrpc/sec_config.c index d3c949d..135ce99 100644 --- a/fs/lustre/ptlrpc/sec_config.c +++ b/fs/lustre/ptlrpc/sec_config.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ptlrpc/sec_gc.c b/fs/lustre/ptlrpc/sec_gc.c index c4dba67..d5edcec 100644 --- a/fs/lustre/ptlrpc/sec_gc.c +++ b/fs/lustre/ptlrpc/sec_gc.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2016, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ptlrpc/sec_null.c b/fs/lustre/ptlrpc/sec_null.c index df6ef4f..726db6e 100644 --- a/fs/lustre/ptlrpc/sec_null.c +++ b/fs/lustre/ptlrpc/sec_null.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ptlrpc/sec_plain.c b/fs/lustre/ptlrpc/sec_plain.c index 021bf7f..84ff1fd 100644 --- a/fs/lustre/ptlrpc/sec_plain.c +++ b/fs/lustre/ptlrpc/sec_plain.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/fs/lustre/ptlrpc/service.c b/fs/lustre/ptlrpc/service.c index b2a1b4a..90c7529 100644 --- a/fs/lustre/ptlrpc/service.c +++ b/fs/lustre/ptlrpc/service.c @@ -24,7 +24,7 @@ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2010, 2015, Intel Corporation. + * Copyright (c) 2010, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/include/linux/libcfs/libcfs.h b/include/linux/libcfs/libcfs.h index cc64039..33f7477 100644 --- a/include/linux/libcfs/libcfs.h +++ b/include/linux/libcfs/libcfs.h @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/include/linux/libcfs/libcfs_cpu.h b/include/linux/libcfs/libcfs_cpu.h index de9ae63..c354dbd 100644 --- a/include/linux/libcfs/libcfs_cpu.h +++ b/include/linux/libcfs/libcfs_cpu.h @@ -19,7 +19,7 @@ /* * Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. * - * Copyright (c) 2012, 2015 Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/include/linux/libcfs/libcfs_debug.h b/include/linux/libcfs/libcfs_debug.h index 911fd35..89deab4 100644 --- a/include/linux/libcfs/libcfs_debug.h +++ b/include/linux/libcfs/libcfs_debug.h @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/include/linux/libcfs/libcfs_hash.h b/include/linux/libcfs/libcfs_hash.h index 89d25ca..7f62379 100644 --- a/include/linux/libcfs/libcfs_hash.h +++ b/include/linux/libcfs/libcfs_hash.h @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015 Intel Corporation. + * Copyright (c) 2012, 2015, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/include/linux/libcfs/libcfs_private.h b/include/linux/libcfs/libcfs_private.h index 2e8cff6..cf13fe0 100644 --- a/include/linux/libcfs/libcfs_private.h +++ b/include/linux/libcfs/libcfs_private.h @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2014, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/include/linux/libcfs/libcfs_string.h b/include/linux/libcfs/libcfs_string.h index f2ac9dc..34b5521 100644 --- a/include/linux/libcfs/libcfs_string.h +++ b/include/linux/libcfs/libcfs_string.h @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/include/linux/lnet/api.h b/include/linux/lnet/api.h index 8d37509..7cc1d04 100644 --- a/include/linux/lnet/api.h +++ b/include/linux/lnet/api.h @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011 - 2015, Intel Corporation. + * Copyright (c) 2016, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/include/linux/lnet/lib-lnet.h b/include/linux/lnet/lib-lnet.h index 538d457..3d7867f 100644 --- a/include/linux/lnet/lib-lnet.h +++ b/include/linux/lnet/lib-lnet.h @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/include/linux/lnet/lib-types.h b/include/linux/lnet/lib-types.h index a5cbf07..15968ba 100644 --- a/include/linux/lnet/lib-types.h +++ b/include/linux/lnet/lib-types.h @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/include/uapi/linux/lnet/libcfs_debug.h b/include/uapi/linux/lnet/libcfs_debug.h index 91d80ab..1a68667 100644 --- a/include/uapi/linux/lnet/libcfs_debug.h +++ b/include/uapi/linux/lnet/libcfs_debug.h @@ -23,7 +23,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2014, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/include/uapi/linux/lnet/libcfs_ioctl.h b/include/uapi/linux/lnet/libcfs_ioctl.h index 765466f..4396d26 100644 --- a/include/uapi/linux/lnet/libcfs_ioctl.h +++ b/include/uapi/linux/lnet/libcfs_ioctl.h @@ -22,6 +22,8 @@ /* * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. + * + * Copyright (c) 2013, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/include/uapi/linux/lnet/lnet-dlc.h b/include/uapi/linux/lnet/lnet-dlc.h index 706892c..484435d 100644 --- a/include/uapi/linux/lnet/lnet-dlc.h +++ b/include/uapi/linux/lnet/lnet-dlc.h @@ -20,7 +20,7 @@ * */ /* - * Copyright (c) 2014, Intel Corporation. + * Copyright (c) 2014, 2017, Intel Corporation. */ /* * Author: Amir Shehata diff --git a/include/uapi/linux/lnet/lnet-types.h b/include/uapi/linux/lnet/lnet-types.h index 956af3c5..2afdd83 100644 --- a/include/uapi/linux/lnet/lnet-types.h +++ b/include/uapi/linux/lnet/lnet-types.h @@ -23,7 +23,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012 - 2015, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/include/uapi/linux/lnet/lnetctl.h b/include/uapi/linux/lnet/lnetctl.h index e9fc57c..191689c 100644 --- a/include/uapi/linux/lnet/lnetctl.h +++ b/include/uapi/linux/lnet/lnetctl.h @@ -12,6 +12,9 @@ * * header for lnet ioctl */ +/* + * Copyright (c) 2014, 2017, Intel Corporation. + */ #ifndef _LNETCTL_H_ #define _LNETCTL_H_ diff --git a/include/uapi/linux/lnet/nidstr.h b/include/uapi/linux/lnet/nidstr.h index 6c49d59..8bff1fd 100644 --- a/include/uapi/linux/lnet/nidstr.h +++ b/include/uapi/linux/lnet/nidstr.h @@ -23,7 +23,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2015, 2017, Intel Corporation. */ #ifndef _LNET_NIDSTRINGS_H #define _LNET_NIDSTRINGS_H diff --git a/include/uapi/linux/lustre/lustre_fid.h b/include/uapi/linux/lustre/lustre_fid.h index 9f7959c..79574c0 100644 --- a/include/uapi/linux/lustre/lustre_fid.h +++ b/include/uapi/linux/lustre/lustre_fid.h @@ -23,7 +23,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2014, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. * * Copyright 2016 Cray Inc, all rights reserved. * Author: Ben Evans. diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index 41d0abb..cd4e0e9 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -23,7 +23,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/include/uapi/linux/lustre/lustre_ioctl.h b/include/uapi/linux/lustre/lustre_ioctl.h index 098b6451..8289d43 100644 --- a/include/uapi/linux/lustre/lustre_ioctl.h +++ b/include/uapi/linux/lustre/lustre_ioctl.h @@ -23,7 +23,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ #ifndef _UAPI_LUSTRE_IOCTL_H_ #define _UAPI_LUSTRE_IOCTL_H_ diff --git a/include/uapi/linux/lustre/lustre_ostid.h b/include/uapi/linux/lustre/lustre_ostid.h index 4b5f110..0f1a0a9 100644 --- a/include/uapi/linux/lustre/lustre_ostid.h +++ b/include/uapi/linux/lustre/lustre_ostid.h @@ -23,7 +23,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2014, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. * * Copyright 2015 Cray Inc, all rights reserved. * Author: Ben Evans. diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index 7a44038..3f29093 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -23,7 +23,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2010, 2015, Intel Corporation. + * Copyright (c) 2010, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/klnds/o2iblnd/o2iblnd.c b/net/lnet/klnds/o2iblnd/o2iblnd.c index d780108..4b3018e 100644 --- a/net/lnet/klnds/o2iblnd/o2iblnd.c +++ b/net/lnet/klnds/o2iblnd/o2iblnd.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/klnds/o2iblnd/o2iblnd.h b/net/lnet/klnds/o2iblnd/o2iblnd.h index 3f28c5b..541f95a 100644 --- a/net/lnet/klnds/o2iblnd/o2iblnd.h +++ b/net/lnet/klnds/o2iblnd/o2iblnd.h @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/klnds/o2iblnd/o2iblnd_cb.c b/net/lnet/klnds/o2iblnd/o2iblnd_cb.c index 871d4fb..a702654 100644 --- a/net/lnet/klnds/o2iblnd/o2iblnd_cb.c +++ b/net/lnet/klnds/o2iblnd/o2iblnd_cb.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c b/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c index 9fb1357..b5df7fe 100644 --- a/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c +++ b/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/klnds/socklnd/socklnd.c b/net/lnet/klnds/socklnd/socklnd.c index d2c72b0..d55b00f 100644 --- a/net/lnet/klnds/socklnd/socklnd.c +++ b/net/lnet/klnds/socklnd/socklnd.c @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/klnds/socklnd/socklnd.h b/net/lnet/klnds/socklnd/socklnd.h index add2744..04381a0 100644 --- a/net/lnet/klnds/socklnd/socklnd.h +++ b/net/lnet/klnds/socklnd/socklnd.h @@ -2,7 +2,7 @@ /* * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. * * Author: Zach Brown * Author: Peter J. Braam diff --git a/net/lnet/klnds/socklnd/socklnd_cb.c b/net/lnet/klnds/socklnd/socklnd_cb.c index 208b8d3..4861dcf 100644 --- a/net/lnet/klnds/socklnd/socklnd_cb.c +++ b/net/lnet/klnds/socklnd/socklnd_cb.c @@ -2,7 +2,7 @@ /* * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. * * Author: Zach Brown * Author: Peter J. Braam diff --git a/net/lnet/klnds/socklnd/socklnd_lib.c b/net/lnet/klnds/socklnd/socklnd_lib.c index a190869..3ac5f2e 100644 --- a/net/lnet/klnds/socklnd/socklnd_lib.c +++ b/net/lnet/klnds/socklnd/socklnd_lib.c @@ -24,7 +24,7 @@ * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/klnds/socklnd/socklnd_proto.c b/net/lnet/klnds/socklnd/socklnd_proto.c index e8b95af..64c0c74 100644 --- a/net/lnet/klnds/socklnd/socklnd_proto.c +++ b/net/lnet/klnds/socklnd/socklnd_proto.c @@ -2,7 +2,7 @@ /* * Copyright (c) 2009, 2010, Oracle and/or its affiliates. All rights reserved. * - * Copyright (c) 2012, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. * * Author: Zach Brown * Author: Peter J. Braam diff --git a/net/lnet/libcfs/debug.c b/net/lnet/libcfs/debug.c index 552e728..fd8ed81 100644 --- a/net/lnet/libcfs/debug.c +++ b/net/lnet/libcfs/debug.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/libcfs/libcfs_cpu.c b/net/lnet/libcfs/libcfs_cpu.c index 5f0d7a2..3e566ac 100644 --- a/net/lnet/libcfs/libcfs_cpu.c +++ b/net/lnet/libcfs/libcfs_cpu.c @@ -18,7 +18,7 @@ */ /* * Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2012, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/libcfs/libcfs_string.c b/net/lnet/libcfs/libcfs_string.c index 02814d3..cefe6fee 100644 --- a/net/lnet/libcfs/libcfs_string.c +++ b/net/lnet/libcfs/libcfs_string.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015 Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/libcfs/module.c b/net/lnet/libcfs/module.c index 89f0783..49731bd 100644 --- a/net/lnet/libcfs/module.c +++ b/net/lnet/libcfs/module.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015 Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/libcfs/tracefile.c b/net/lnet/libcfs/tracefile.c index d634e22..5f51ec7 100644 --- a/net/lnet/libcfs/tracefile.c +++ b/net/lnet/libcfs/tracefile.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/libcfs/tracefile.h b/net/lnet/libcfs/tracefile.h index 3a00a6c..88ff0d1 100644 --- a/net/lnet/libcfs/tracefile.h +++ b/net/lnet/libcfs/tracefile.h @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/lnet/acceptor.c b/net/lnet/lnet/acceptor.c index 83ab3b1..d91a38c 100644 --- a/net/lnet/lnet/acceptor.c +++ b/net/lnet/lnet/acceptor.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/lnet/api-ni.c b/net/lnet/lnet/api-ni.c index b038010..6e29714 100644 --- a/net/lnet/lnet/api-ni.c +++ b/net/lnet/lnet/api-ni.c @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/lnet/config.c b/net/lnet/lnet/config.c index 6dda6e5..ef1faa0 100644 --- a/net/lnet/lnet/config.c +++ b/net/lnet/lnet/config.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/lnet/lib-md.c b/net/lnet/lnet/lib-md.c index 8fe3564..7ea0f5e 100644 --- a/net/lnet/lnet/lib-md.c +++ b/net/lnet/lnet/lib-md.c @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/lnet/lib-move.c b/net/lnet/lnet/lib-move.c index 6f3e941..202372d 100644 --- a/net/lnet/lnet/lib-move.c +++ b/net/lnet/lnet/lib-move.c @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/lnet/lib-msg.c b/net/lnet/lnet/lib-msg.c index 02620fe..a7062f6 100644 --- a/net/lnet/lnet/lib-msg.c +++ b/net/lnet/lnet/lib-msg.c @@ -24,7 +24,7 @@ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/lnet/lib-ptl.c b/net/lnet/lnet/lib-ptl.c index 83548f5..bb92f37 100644 --- a/net/lnet/lnet/lib-ptl.c +++ b/net/lnet/lnet/lib-ptl.c @@ -17,7 +17,7 @@ * GPL HEADER END */ /* - * Copyright (c) 2012, 2015, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/lnet/lib-socket.c b/net/lnet/lnet/lib-socket.c index 6ea8f3f..d430d6f 100644 --- a/net/lnet/lnet/lib-socket.c +++ b/net/lnet/lnet/lib-socket.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/lnet/lo.c b/net/lnet/lnet/lo.c index c8a1eb62..2563243 100644 --- a/net/lnet/lnet/lo.c +++ b/net/lnet/lnet/lo.c @@ -23,6 +23,8 @@ /* * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. + * + * Copyright (c) 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/lnet/module.c b/net/lnet/lnet/module.c index 78fc96e..95e1bae 100644 --- a/net/lnet/lnet/module.c +++ b/net/lnet/lnet/module.c @@ -24,7 +24,7 @@ * Copyright (c) 2004, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/lnet/net_fault.c b/net/lnet/lnet/net_fault.c index 76fb61d4..4589b17 100644 --- a/net/lnet/lnet/net_fault.c +++ b/net/lnet/lnet/net_fault.c @@ -21,7 +21,7 @@ * GPL HEADER END */ /* - * Copyright (c) 2014, Intel Corporation. + * Copyright (c) 2014, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/lnet/nidstrings.c b/net/lnet/lnet/nidstrings.c index 0a63b0c..b84708d 100644 --- a/net/lnet/lnet/nidstrings.c +++ b/net/lnet/lnet/nidstrings.c @@ -24,7 +24,7 @@ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/lnet/peer.c b/net/lnet/lnet/peer.c index 68c6eb7..52d4ec0 100644 --- a/net/lnet/lnet/peer.c +++ b/net/lnet/lnet/peer.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/lnet/router.c b/net/lnet/lnet/router.c index bcde61d..4b0c57a 100644 --- a/net/lnet/lnet/router.c +++ b/net/lnet/lnet/router.c @@ -2,7 +2,7 @@ /* * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. * * This file is part of Portals * http://sourceforge.net/projects/sandiaportals/ diff --git a/net/lnet/lnet/router_proc.c b/net/lnet/lnet/router_proc.c index 94ef441..ebe7993 100644 --- a/net/lnet/lnet/router_proc.c +++ b/net/lnet/lnet/router_proc.c @@ -2,7 +2,7 @@ /* * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * - * Copyright (c) 2011, 2012, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. * * This file is part of Portals * http://sourceforge.net/projects/sandiaportals/ diff --git a/net/lnet/selftest/brw_test.c b/net/lnet/selftest/brw_test.c index eb8b1e9..560bcc7 100644 --- a/net/lnet/selftest/brw_test.c +++ b/net/lnet/selftest/brw_test.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2011, 2015, Intel Corporation. + * Copyright (c) 2011, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/selftest/framework.c b/net/lnet/selftest/framework.c index 72892c1..714ceec 100644 --- a/net/lnet/selftest/framework.c +++ b/net/lnet/selftest/framework.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/selftest/rpc.c b/net/lnet/selftest/rpc.c index abb6f8f..295d704 100644 --- a/net/lnet/selftest/rpc.c +++ b/net/lnet/selftest/rpc.c @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, 2015, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ diff --git a/net/lnet/selftest/selftest.h b/net/lnet/selftest/selftest.h index edf783a..fb16143 100644 --- a/net/lnet/selftest/selftest.h +++ b/net/lnet/selftest/selftest.h @@ -24,7 +24,7 @@ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. * Use is subject to license terms. * - * Copyright (c) 2012, Intel Corporation. + * Copyright (c) 2012, 2017, Intel Corporation. */ /* * This file is part of Lustre, http://www.lustre.org/ From patchwork Mon Sep 30 18:55:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167251 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2976B13B1 for ; Mon, 30 Sep 2019 19:05:11 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 11FE5224EF for ; Mon, 30 Sep 2019 19:05:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 11FE5224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id DB5975E498A; Mon, 30 Sep 2019 12:00:27 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 4C89F5C3B32 for ; Mon, 30 Sep 2019 11:57:29 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 5D39F1005C93; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 5B914BB; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:55 -0400 Message-Id: <1569869810-23848-97-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 096/151] lustre: fid: improve seq allocation error messages X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Emoly Liu When MDTs are waiting for MDT0000 to start the master sequence server and be granted a meta sequence for the first time, in case of "-EINPROGRESS", the limited console messages would be clearer and better than many error messages. WC-bug-id: https://jira.whamcloud.com/browse/LU-10392 Lustre-commit: 6f86519b3483 ("LU-10392 fid: improve seq allocation error messages") Signed-off-by: Emoly Liu Reviewed-on: https://review.whamcloud.com/30623 Reviewed-by: Andreas Dilger Reviewed-by: Yang Sheng Reviewed-by: Giuseppe Di Natale Signed-off-by: James Simmons --- fs/lustre/fid/fid_request.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/fs/lustre/fid/fid_request.c b/fs/lustre/fid/fid_request.c index 861ffbe..a54d1e5 100644 --- a/fs/lustre/fid/fid_request.c +++ b/fs/lustre/fid/fid_request.c @@ -171,8 +171,9 @@ static int seq_client_alloc_seq(const struct lu_env *env, if (lu_seq_range_is_exhausted(&seq->lcs_space)) { rc = seq_client_alloc_meta(env, seq); if (rc) { - CERROR("%s: Can't allocate new meta-sequence, rc %d\n", - seq->lcs_name, rc); + if (rc != -EINPROGRESS) + CERROR("%s: Can't allocate new meta-sequence, rc = %d\n", + seq->lcs_name, rc); *seqnr = U64_MAX; return rc; } @@ -232,8 +233,9 @@ int seq_client_alloc_fid(const struct lu_env *env, wake_up(&seq->lcs_waitq); if (rc) { - CERROR("%s: Can't allocate new sequence, rc %d\n", - seq->lcs_name, rc); + if (rc != -EINPROGRESS) + CERROR("%s: Can't allocate new sequence, rc %d\n", + seq->lcs_name, rc); spin_unlock(&seq->lcs_lock); return rc; } From patchwork Mon Sep 30 18:55:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167301 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B450F1599 for ; Mon, 30 Sep 2019 19:07:08 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9C7C4224EF for ; Mon, 30 Sep 2019 19:07:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9C7C4224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 45C735E4E12; Mon, 30 Sep 2019 12:01:13 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8D4EE5C3B32 for ; Mon, 30 Sep 2019 11:57:29 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 60AEF1005C94; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 5EC72B5; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:56 -0400 Message-Id: <1569869810-23848-98-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 097/151] lustre: mdc: interruptable during RPC retry for EINPROGRESS X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fan Yong , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Fan Yong Sometimes, some system resource may be inaccessible temporarily, for example, related OI mapping is crashed and has yet not been rebuilt. Under such case, the server will reply the client with "-EINPROGRESS", then client will retry the RPC some time later. Currently, the client will retry infinitely until related RPC succeed or get other failure. But we do not know how long it will be before related resource becoming available. It may be very long time as to the RPC sponsor - the application or the user does not want to retry any more, then we need to make the logic to be interruptable. This patch is for such purpose. WC-bug-id: https://jira.whamcloud.com/browse/LU-10237 Lustre-commit: 9c596a4996ee ("LU-10237 mdc: interruptable during RPC retry for EINPROGRESS") Signed-off-by: Fan Yong Reviewed-on: https://review.whamcloud.com/30166 Reviewed-by: Andreas Dilger Reviewed-by: Mike Pershin Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/mdc/mdc_locks.c | 14 ++++++++------ fs/lustre/mdc/mdc_reint.c | 19 +++++++++++-------- 2 files changed, 19 insertions(+), 14 deletions(-) diff --git a/fs/lustre/mdc/mdc_locks.c b/fs/lustre/mdc/mdc_locks.c index f297a98..0b358b6 100644 --- a/fs/lustre/mdc/mdc_locks.c +++ b/fs/lustre/mdc/mdc_locks.c @@ -904,13 +904,15 @@ int mdc_enqueue_base(struct obd_export *exp, struct ldlm_enqueue_info *einfo, if (it->it_op && (int)lockrep->lock_policy_res2 == -EINPROGRESS) { mdc_clear_replay_flag(req, rc); ptlrpc_req_finished(req); - resends++; - - CDEBUG(D_HA, "%s: resend:%d op:%d " DFID "/" DFID "\n", - obddev->obd_name, resends, it->it_op, - PFID(&op_data->op_fid1), PFID(&op_data->op_fid2)); - if (generation == obddev->u.cli.cl_import->imp_generation) { + if (signal_pending(current)) + return -EINTR; + + resends++; + CDEBUG(D_HA, "%s: resend:%d op:%d "DFID"/"DFID"\n", + obddev->obd_name, resends, it->it_op, + PFID(&op_data->op_fid1), + PFID(&op_data->op_fid2)); goto resend; } else { CDEBUG(D_HA, "resend cross eviction\n"); diff --git a/fs/lustre/mdc/mdc_reint.c b/fs/lustre/mdc/mdc_reint.c index d6216d6..d326962 100644 --- a/fs/lustre/mdc/mdc_reint.c +++ b/fs/lustre/mdc/mdc_reint.c @@ -231,17 +231,20 @@ int mdc_create(struct obd_export *exp, struct md_op_data *op_data, level = LUSTRE_IMP_RECOVER; goto resend; } else if (rc == -EINPROGRESS) { - /* Retry create infinitely until succeed or get other - * error code. + /* + * Retry create infinitely until succeed or get other + * error code or interrupted. */ ptlrpc_req_finished(req); - resends++; - - CDEBUG(D_HA, "%s: resend:%d create on " DFID "/" DFID "\n", - exp->exp_obd->obd_name, resends, - PFID(&op_data->op_fid1), PFID(&op_data->op_fid2)); - if (generation == import->imp_generation) { + if (signal_pending(current)) + return -EINTR; + + resends++; + CDEBUG(D_HA, "%s: resend:%d create on "DFID"/"DFID"\n", + exp->exp_obd->obd_name, resends, + PFID(&op_data->op_fid1), + PFID(&op_data->op_fid2)); goto rebuild; } else { CDEBUG(D_HA, "resend cross eviction\n"); From patchwork Mon Sep 30 18:55:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167319 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3ACBF13B1 for ; Mon, 30 Sep 2019 19:07:49 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 234F6224EF for ; Mon, 30 Sep 2019 19:07:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 234F6224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 13F3E5E4F69; Mon, 30 Sep 2019 12:01:30 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id E2F3C5C3B32 for ; Mon, 30 Sep 2019 11:57:29 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 636E01005EE1; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 61DE6A9; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:57 -0400 Message-Id: <1569869810-23848-99-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 098/151] lustre: osc: migrate to 64 bit time X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: James Simmons , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" Change od_contention_time from int to time64_t to make it clear this field is in units of seconds. Change the *_contention_time fields from jiffies to ktime_t to make it clear we are dealing with time and ktime_t is consistent on any platform unlike jiffies. WC-bug-id: https://jira.whamcloud.com/browse/LU-9019 Lustre-commit: 3ea899c9a611 ("LU-9019 osc: migrate to 64 bit time") Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/30607 Reviewed-by: Bobi Jam Reviewed-by: Andreas Dilger Reviewed-by: Mike Pershin Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_osc.h | 6 +++--- fs/lustre/mdc/lproc_mdc.c | 2 +- fs/lustre/osc/lproc_osc.c | 2 +- fs/lustre/osc/osc_cache.c | 4 ++-- fs/lustre/osc/osc_object.c | 10 +++++----- fs/lustre/osc/osc_page.c | 10 +++++----- 6 files changed, 17 insertions(+), 17 deletions(-) diff --git a/fs/lustre/include/lustre_osc.h b/fs/lustre/include/lustre_osc.h index cba9d43..38ae83a8 100644 --- a/fs/lustre/include/lustre_osc.h +++ b/fs/lustre/include/lustre_osc.h @@ -120,7 +120,7 @@ struct osc_device { } od_stats; /* configuration item(s) */ - int od_contention_time; + time64_t od_contention_time; int od_lockless_truncate; }; @@ -264,7 +264,7 @@ struct osc_object { * True if locking against this stripe got -EUSERS. */ int oo_contended; - unsigned long oo_contention_time; + ktime_t oo_contention_time; /* * used by the osc to keep track of what objects to build into rpcs. * Protected by client_obd->cli_loi_list_lock. @@ -532,7 +532,7 @@ struct osc_page { /* * Submit time - the time when the page is starting RPC. For debugging. */ - unsigned long ops_submit_time; + ktime_t ops_submit_time; }; struct osc_brw_async_args { diff --git a/fs/lustre/mdc/lproc_mdc.c b/fs/lustre/mdc/lproc_mdc.c index 2af2e57..f09292e 100644 --- a/fs/lustre/mdc/lproc_mdc.c +++ b/fs/lustre/mdc/lproc_mdc.c @@ -154,7 +154,7 @@ static int mdc_contention_seconds_seq_show(struct seq_file *m, void *v) struct obd_device *obd = m->private; struct osc_device *od = obd2osc_dev(obd); - seq_printf(m, "%u\n", od->od_contention_time); + seq_printf(m, "%lld\n", od->od_contention_time); return 0; } diff --git a/fs/lustre/osc/lproc_osc.c b/fs/lustre/osc/lproc_osc.c index 6661092..df48138 100644 --- a/fs/lustre/osc/lproc_osc.c +++ b/fs/lustre/osc/lproc_osc.c @@ -506,7 +506,7 @@ static ssize_t contention_seconds_show(struct kobject *kobj, obd_kset.kobj); struct osc_device *od = obd2osc_dev(obd); - return sprintf(buf, "%u\n", od->od_contention_time); + return sprintf(buf, "%lld\n", od->od_contention_time); } static ssize_t contention_seconds_store(struct kobject *kobj, diff --git a/fs/lustre/osc/osc_cache.c b/fs/lustre/osc/osc_cache.c index 68498af..3189eb3 100644 --- a/fs/lustre/osc/osc_cache.c +++ b/fs/lustre/osc/osc_cache.c @@ -1307,7 +1307,7 @@ static int osc_make_ready(const struct lu_env *env, struct osc_async_page *oap, result = cl_page_make_ready(env, page, CRT_WRITE); if (result == 0) - opg->ops_submit_time = jiffies; + opg->ops_submit_time = ktime_get(); return result; } @@ -1362,7 +1362,7 @@ static int osc_completion(const struct lu_env *env, struct osc_async_page *oap, /* Clear opg->ops_transfer_pinned before VM lock is released. */ opg->ops_transfer_pinned = 0; - opg->ops_submit_time = 0; + opg->ops_submit_time = ktime_set(0, 0); srvlock = oap->oap_brw_flags & OBD_BRW_SRVLOCK; /* statistic */ diff --git a/fs/lustre/osc/osc_object.c b/fs/lustre/osc/osc_object.c index 4cd6e15..fdee8fa 100644 --- a/fs/lustre/osc/osc_object.c +++ b/fs/lustre/osc/osc_object.c @@ -307,9 +307,8 @@ static int osc_object_fiemap(const struct lu_env *env, struct cl_object *obj, int osc_object_is_contended(struct osc_object *obj) { struct osc_device *dev = lu2osc_dev(obj->oo_cl.co_lu.lo_dev); - int osc_contention_time = dev->od_contention_time; - unsigned long cur_time = jiffies; - unsigned long retry_time; + time64_t osc_contention_time = dev->od_contention_time; + ktime_t retry_time; if (OBD_FAIL_CHECK(OBD_FAIL_OSC_OBJECT_CONTENTION)) return 1; @@ -321,8 +320,9 @@ int osc_object_is_contended(struct osc_object *obj) * I like copy-paste. the code is copied from * ll_file_is_contended. */ - retry_time = obj->oo_contention_time + osc_contention_time * HZ; - if (time_after(cur_time, retry_time)) { + retry_time = ktime_add_ns(obj->oo_contention_time, + osc_contention_time * NSEC_PER_SEC); + if (ktime_after(ktime_get(), retry_time)) { osc_object_clear_contended(obj); return 0; } diff --git a/fs/lustre/osc/osc_page.c b/fs/lustre/osc/osc_page.c index f5b6f73..96d1385 100644 --- a/fs/lustre/osc/osc_page.c +++ b/fs/lustre/osc/osc_page.c @@ -122,12 +122,12 @@ static const char *osc_list(struct list_head *head) return list_empty(head) ? "-" : "+"; } -static inline unsigned long osc_submit_duration(struct osc_page *opg) +static inline s64 osc_submit_duration(struct osc_page *opg) { - if (opg->ops_submit_time == 0) + if (ktime_to_ns(opg->ops_submit_time) == 0) return 0; - return (jiffies - opg->ops_submit_time); + return ktime_ms_delta(ktime_get(), opg->ops_submit_time); } static int osc_page_print(const struct lu_env *env, @@ -139,7 +139,7 @@ static int osc_page_print(const struct lu_env *env, struct osc_object *obj = cl2osc(slice->cpl_obj); struct client_obd *cli = &osc_export(obj)->exp_obd->u.cli; - return (*printer)(env, cookie, LUSTRE_OSC_NAME "-page@%p %lu: 1< %#x %d %s %s > 2< %llu %u %u %#x %#x | %p %p %p > 3< %d %lu %d > 4< %d %d %d %lu %s | %s %s %s %s > 5< %s %s %s %s | %d %s | %d %s %s>\n", + return (*printer)(env, cookie, LUSTRE_OSC_NAME "-page@%p %lu: 1< %#x %d %s %s > 2< %llu %u %u %#x %#x | %p %p %p > 3< %d %lld %d > 4< %d %d %d %lu %s | %s %s %s %s > 5< %s %s %s %s | %d %s | %d %s %s>\n", opg, osc_index(opg), /* 1 */ oap->oap_magic, oap->oap_cmd, @@ -303,7 +303,7 @@ void osc_page_submit(const struct lu_env *env, struct osc_page *opg, oap->oap_cmd |= OBD_BRW_NOQUOTA; } - opg->ops_submit_time = jiffies; + opg->ops_submit_time = ktime_get(); osc_page_transfer_get(opg, "transfer\0imm"); osc_page_transfer_add(env, opg, crt); } From patchwork Mon Sep 30 18:55:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167245 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3708B1599 for ; Mon, 30 Sep 2019 19:05:02 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1FAC6224EF for ; Mon, 30 Sep 2019 19:05:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1FAC6224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 0DB5A5E4906; Mon, 30 Sep 2019 12:00:24 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 48C075C3B32 for ; Mon, 30 Sep 2019 11:57:30 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 665DF1005EE8; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 64B68BD; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:58 -0400 Message-Id: <1569869810-23848-100-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 099/151] lustre: vvp: Print discarded page warning on -EIO X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Patrick Farrell On client eviction, the client sometimes has dirty pages outstanding, which are then discarded. The client is supposed to print an error when this happens, from vvp_vmpage_error->ll_dirty_page_discard_warn. However, the client looks for specific errors, and newer Lustre clients will sometimes return -EIO to I/O requests on eviction, instead of -EINTR. Since they can still return -EINTR, we must add -EIO as a new condition and keep -EINTR. WC-bug-id: https://jira.whamcloud.com/browse/LU-8358 Lustre-commit: 5f327f6670d3 ("LU-8358 vvp: Print discarded page warning on -EIO") Signed-off-by: Patrick Farrell Reviewed-on: https://review.whamcloud.com/21111 Reviewed-by: Chris Horn Reviewed-by: Ben Evans Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/vvp_page.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/lustre/llite/vvp_page.c b/fs/lustre/llite/vvp_page.c index beebea8..78a70b5 100644 --- a/fs/lustre/llite/vvp_page.c +++ b/fs/lustre/llite/vvp_page.c @@ -241,8 +241,8 @@ static void vvp_vmpage_error(struct inode *inode, struct page *vmpage, SetPageError(vmpage); mapping_set_error(inode->i_mapping, ioret); - if ((ioret == -ESHUTDOWN || ioret == -EINTR) && - obj->vob_discard_page_warned == 0) { + if ((ioret == -ESHUTDOWN || ioret == -EINTR || + ioret == -EIO) && obj->vob_discard_page_warned == 0) { obj->vob_discard_page_warned = 1; ll_dirty_page_discard_warn(vmpage, ioret); } From patchwork Mon Sep 30 18:55:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167255 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 82D4D16B1 for ; Mon, 30 Sep 2019 19:05:19 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6B4B4224F0 for ; Mon, 30 Sep 2019 19:05:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6B4B4224F0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 339CE5E49EE; Mon, 30 Sep 2019 12:00:31 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 928235C3253 for ; Mon, 30 Sep 2019 11:57:30 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 6AE381005EE9; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 69BA5B4; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:55:59 -0400 Message-Id: <1569869810-23848-101-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 100/151] lustre: clio: Use readahead for partial page write X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Patrick Farrell When writing to a region of a file less than file size (either an existing file or a shared file with multiple writers), writes of less than one page in size must first read in that page. This results in extremely poor performance. For random I/O, there's no easy improvements available, but the sequential case can benefit enormously by using readahead to bring in those pages. This patch connects ll_prepare_partial_page to the readahead infrastructure. This does not affect random I/O or large unaligned writes, where readahead does not detect I/O. Benchmarks are from a small VM system, files are NOT in cache when rewriting. Write numbers are in MB/s. File per process: access = file-per-process ordering in a file = sequential offsets ordering inter file= no tasks offsets clients = 1 (1 per node) repetitions = 1 blocksize = 1000 MiB aggregate filesize = 1000 MiB New file (best case): xfsize ppr write 1KiB n/a 59.44 5KiB n/a 164.5 Rewrite of existing file: xfsize ppr re-write 1KiB off 4.65 1KiB on 48.40 5KiB off 12.95 5KiB on 143.3 Shared file writing: access = single-shared-file ordering in a file = sequential offsets ordering inter file= no tasks offsets clients = 4 (4 per node) repetitions = 1 blocksize = 1000 MiB aggregate filesize = 4000 MiB xfsize ppr write 1KiB off 11.26 1KiB on 58.72 5KiB off 18.7 5KiB on 127.3 WC-bug-id: https://jira.whamcloud.com/browse/LU-9618 Cray-bug-id: LUS-188 Lustre-commit: b7d38ece0013 ("LU-9618 clio: Use readahead for partial page write") Signed-off-by: Patrick Farrell Signed-off-by: Jinshan Xiong Reviewed-on: https://review.whamcloud.com/27544 Reviewed-by: Jinshan Xiong Reviewed-by: Dmitry Eremin Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/llite_internal.h | 2 ++ fs/lustre/llite/rw.c | 34 +++--------------- fs/lustre/llite/rw26.c | 75 +++++++++++++++++++++++++++++----------- 3 files changed, 61 insertions(+), 50 deletions(-) diff --git a/fs/lustre/llite/llite_internal.h b/fs/lustre/llite/llite_internal.h index ded310b..229d417 100644 --- a/fs/lustre/llite/llite_internal.h +++ b/fs/lustre/llite/llite_internal.h @@ -788,6 +788,8 @@ int ll_md_blocking_ast(struct ldlm_lock *lock, struct ldlm_lock_desc *desc, int ll_writepage(struct page *page, struct writeback_control *wbc); int ll_writepages(struct address_space *mapping, struct writeback_control *wbc); int ll_readpage(struct file *file, struct page *page); +int ll_io_read_page(const struct lu_env *env, struct cl_io *io, + struct cl_page *page, struct file *file); void ll_readahead_init(struct inode *inode, struct ll_readahead_state *ras); int vvp_io_write_commit(const struct lu_env *env, struct cl_io *io); diff --git a/fs/lustre/llite/rw.c b/fs/lustre/llite/rw.c index 02b9a0c..c42bbab 100644 --- a/fs/lustre/llite/rw.c +++ b/fs/lustre/llite/rw.c @@ -1094,7 +1094,7 @@ void ll_cl_remove(struct file *file, const struct lu_env *env) write_unlock(&fd->fd_lock); } -static int ll_io_read_page(const struct lu_env *env, struct cl_io *io, +int ll_io_read_page(const struct lu_env *env, struct cl_io *io, struct cl_page *page, struct file *file) { struct inode *inode = vvp_object_inode(page->cp_obj); @@ -1153,6 +1153,7 @@ static int ll_io_read_page(const struct lu_env *env, struct cl_io *io, if (!rc) task_io_account_read(PAGE_SIZE * count); } + if (anchor && !cl_page_is_owned(page, io)) { /* have sent */ rc = cl_sync_io_wait(env, anchor, 0); @@ -1174,10 +1175,9 @@ static int ll_io_read_page(const struct lu_env *env, struct cl_io *io, /* TODO: discard all pages until page reinit route is implemented */ cl_page_list_discard(env, io, &queue->c2_qin); - /* - * Unlock unsent pages in case of error. - */ + /* Unlock unsent read pages in case of error. */ cl_page_list_disown(env, io, &queue->c2_qin); + cl_2queue_fini(env, queue); return rc; @@ -1270,6 +1270,7 @@ int ll_readpage(struct file *file, struct page *vmpage) LASSERT(page->cp_type == CPT_CACHEABLE); if (likely(!PageUptodate(vmpage))) { cl_page_assume(env, io, page); + result = ll_io_read_page(env, io, page, file); } else { /* Page from a non-object file. */ @@ -1283,28 +1284,3 @@ int ll_readpage(struct file *file, struct page *vmpage) } return result; } - -int ll_page_sync_io(const struct lu_env *env, struct cl_io *io, - struct cl_page *page, enum cl_req_type crt) -{ - struct cl_2queue *queue; - int result; - - LASSERT(io->ci_type == CIT_READ || io->ci_type == CIT_WRITE); - - queue = &io->ci_queue; - cl_2queue_init_page(queue, page); - - result = cl_io_submit_sync(env, io, crt, queue, 0); - LASSERT(cl_page_is_owned(page, io)); - - if (crt == CRT_READ) - /* - * in CRT_WRITE case page is left locked even in case of - * error. - */ - cl_page_list_disown(env, io, &queue->c2_qin); - cl_2queue_fini(env, queue); - - return result; -} diff --git a/fs/lustre/llite/rw26.c b/fs/lustre/llite/rw26.c index 4d86c4b..37b6755 100644 --- a/fs/lustre/llite/rw26.c +++ b/fs/lustre/llite/rw26.c @@ -383,9 +383,11 @@ static ssize_t ll_direct_IO(struct kiocb *iocb, struct iov_iter *iter) /** * Prepare partially written-to page for a write. + * @pg is owned when passed in and disowned when it returns non-zero result to + * the caller. */ static int ll_prepare_partial_page(const struct lu_env *env, struct cl_io *io, - struct cl_page *pg) + struct cl_page *pg, struct file *file) { struct cl_attr *attr = vvp_env_thread_attr(env); struct cl_object *obj = io->ci_obj; @@ -396,23 +398,48 @@ static int ll_prepare_partial_page(const struct lu_env *env, struct cl_io *io, cl_object_attr_lock(obj); result = cl_object_attr_get(env, obj, attr); cl_object_attr_unlock(obj); - if (result == 0) { - /* - * If are writing to a new page, no need to read old data. - * The extent locking will have updated the KMS, and for our - * purposes here we can treat it like i_size. - */ - if (attr->cat_kms <= offset) { - char *kaddr = kmap_atomic(vpg->vpg_page); + if (result) { + cl_page_disown(env, io, pg); + goto out; + } - memset(kaddr, 0, cl_page_size(obj)); - kunmap_atomic(kaddr); - } else if (vpg->vpg_defer_uptodate) { - vpg->vpg_ra_used = 1; - } else { - result = ll_page_sync_io(env, io, pg, CRT_READ); + /* + * If are writing to a new page, no need to read old data. + * The extent locking will have updated the KMS, and for our + * purposes here we can treat it like i_size. + */ + if (attr->cat_kms <= offset) { + char *kaddr = kmap_atomic(vpg->vpg_page); + + memset(kaddr, 0, cl_page_size(obj)); + kunmap_atomic(kaddr); + result = 0; + goto out; + } + + if (vpg->vpg_defer_uptodate) { + vpg->vpg_ra_used = 1; + result = 0; + goto out; + } + + result = ll_io_read_page(env, io, pg, file); + if (result) + goto out; + + /* ll_io_read_page() disowns the page */ + result = cl_page_own(env, io, pg); + if (!result) { + if (!PageUptodate(cl_page_vmpage(pg))) { + cl_page_disown(env, io, pg); + result = -EIO; } + } else if (result == -ENOENT) { + /* page was truncated */ + result = -EAGAIN; } + +out: return result; } @@ -452,7 +479,7 @@ static int ll_write_begin(struct file *file, struct address_space *mapping, result = -EBUSY; goto out; } - +again: /* To avoid deadlock, try to lock page first. */ vmpage = grab_cache_page_nowait(mapping, index); if (unlikely(!vmpage || PageDirty(vmpage) || PageWriteback(vmpage))) { @@ -509,13 +536,19 @@ static int ll_write_begin(struct file *file, struct address_space *mapping, * is a lockless IO. In that case, it's not necessary * to read the data. */ - result = ll_prepare_partial_page(env, io, page); - if (result == 0) - SetPageUptodate(vmpage); + result = ll_prepare_partial_page(env, io, page, file); + if (result) { + /* vmpage should have been unlocked */ + put_page(vmpage); + vmpage = NULL; + + if (result == -EAGAIN) + goto again; + + goto out; + } } } - if (result < 0) - cl_page_unassume(env, io, page); out: if (result < 0) { if (vmpage) { From patchwork Mon Sep 30 18:56:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167203 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DADE5912 for ; Mon, 30 Sep 2019 19:03:22 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C2DAB224EF for ; Mon, 30 Sep 2019 19:03:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C2DAB224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 6E2EC21F148; Mon, 30 Sep 2019 11:59:39 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id EE2505C3A5E for ; Mon, 30 Sep 2019 11:57:30 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 6D1C81005EEA; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 6BB55B5; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:00 -0400 Message-Id: <1569869810-23848-102-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 101/151] lustre: flr: comp-flags support when creating mirrors X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Jinshan Xiong This patch will allow flags to be set when creating mirrors. The flags are set to individual components therefore it would be flexible to flags based on the location of components. Also, 'stale' and 'prefer' flags are allowed to set to individual components later on. This patch also revises component flags matching rules to allow flags and inverted flags to be set at the same time in the command lfs-find(1) and lfs-getstripe(1). WC-bug-id: https://jira.whamcloud.com/browse/LU-10282 Lustre-commit: aa1646a8a1c2 ("LU-10282 flr: comp-flags support when creating mirrors") Signed-off-by: Jinshan Xiong Reviewed-on: https://review.whamcloud.com/30360 Reviewed-by: Andreas Dilger Reviewed-by: Bobi Jam Reviewed-by: Jian Yu Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/lov/lov_object.c | 44 +++++++++++---------------------- include/uapi/linux/lustre/lustre_user.h | 19 +++++++------- 2 files changed, 24 insertions(+), 39 deletions(-) diff --git a/fs/lustre/lov/lov_object.c b/fs/lustre/lov/lov_object.c index a73f2ac..d3ed5fc 100644 --- a/fs/lustre/lov/lov_object.c +++ b/fs/lustre/lov/lov_object.c @@ -611,6 +611,7 @@ static int lov_init_composite(const struct lu_env *env, struct lov_device *dev, unsigned int psz = 0; unsigned int mirror_count; int result = 0; + unsigned int seq; int i, j; LASSERT(lsm->lsm_entry_count > 0); @@ -694,8 +695,8 @@ static int lov_init_composite(const struct lu_env *env, struct lov_device *dev, /* entries must be sorted by mirrors */ lre->lre_mirror_id = mirror_id; lre->lre_start = lre->lre_end = i; - lre->lre_preferred = (lle->lle_lsme->lsme_flags & - LCME_FL_PREFERRED); + lre->lre_preferred = !!(lle->lle_lsme->lsme_flags & + LCME_FL_PREF_RD); lre->lre_valid = lle->lle_valid; lre->lre_stale = !lle->lle_valid; } @@ -734,45 +735,30 @@ static int lov_init_composite(const struct lu_env *env, struct lov_device *dev, if (psz > 0) cl_object_header(&lov->lo_cl)->coh_page_bufsize += psz; - /* decide the preferred mirror */ + /* + * Decide the preferred mirror. It uses the hash value of lov_object + * so that different clients would use different mirrors for read. + */ mirror_count = 0; - i = 0; - lov_foreach_mirror_entry(lov, lre) { - i++; + seq = hash_long((unsigned long)lov, 8); + for (i = 0; i < comp->lo_mirror_count; i++) { + unsigned int idx = (i + seq) % comp->lo_mirror_count; + + lre = lov_mirror_entry(lov, idx); if (lre->lre_stale) continue; mirror_count++; /* valid mirror */ if (lre->lre_preferred || comp->lo_preferred_mirror < 0) - comp->lo_preferred_mirror = i - 1; + comp->lo_preferred_mirror = idx; } - if (mirror_count == 0) { + if (!mirror_count) { CDEBUG(D_INODE, DFID " doesn't have any valid mirrors\n", PFID(lu_object_fid(lov2lu(lov)))); - result = -EINVAL; - goto out; - } - - if (OBD_FAIL_CHECK(OBD_FAIL_FLR_RANDOM_PICK_MIRROR)) { - unsigned int seq; - - get_random_bytes(&seq, sizeof(seq)); - seq %= mirror_count; - - i = 0; - lov_foreach_mirror_entry(lov, lre) { - i++; - if (lre->lre_stale) - continue; - - if (!seq--) { - comp->lo_preferred_mirror = i - 1; - break; - } - } + comp->lo_preferred_mirror = 0; } LASSERT(comp->lo_preferred_mirror >= 0); diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index 3f29093..21dc1e9 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -463,17 +463,21 @@ static inline bool lu_extent_is_whole(struct lu_extent *e) } enum lov_comp_md_entry_flags { - LCME_FL_PRIMARY = 0x00000001, /* Not used */ - LCME_FL_STALE = 0x00000002, /* Not used */ - LCME_FL_OFFLINE = 0x00000004, /* Not used */ - LCME_FL_PREFERRED = 0x00000008, /* Not used */ + LCME_FL_STALE = 0x00000001, /* FLR: stale data */ + LCME_FL_PREF_RD = 0x00000002, /* FLR: preferred for reading */ + LCME_FL_PREF_WR = 0x00000004, /* FLR: preferred for writing */ + LCME_FL_PREF_RW = LCME_FL_PREF_RD | LCME_FL_PREF_WR, + LCME_FL_OFFLINE = 0x00000008, /* Not used */ LCME_FL_INIT = 0x00000010, /* instantiated */ LCME_FL_NEG = 0x80000000, /* used to indicate a negative * flag, won't be stored on disk */ }; -#define LCME_KNOWN_FLAGS (LCME_FL_NEG | LCME_FL_INIT) +#define LCME_KNOWN_FLAGS (LCME_FL_NEG | LCME_FL_INIT | LCME_FL_STALE | \ + LCME_FL_PREF_RW) +/* The flags can be set by users at mirror creation time. */ +#define LCME_USER_FLAGS (LCME_FL_PREF_RW) /* the highest bit in obdo::o_layout_version is used to mark if the file is * being resynced. @@ -546,11 +550,6 @@ struct lov_comp_md_v1 { struct lov_comp_md_entry_v1 lcm_entries[0]; } __packed; -/* - * Maximum number of mirrors Lustre can support. - */ -#define LUSTRE_MIRROR_COUNT_MAX 16 - static inline __u32 lov_user_md_size(__u16 stripes, __u32 lmm_magic) { if (stripes == (__u16)-1) From patchwork Mon Sep 30 18:56:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167323 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 42C8313B1 for ; Mon, 30 Sep 2019 19:07:57 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2B163224EF for ; Mon, 30 Sep 2019 19:07:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2B163224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 36C515E4FB4; Mon, 30 Sep 2019 12:01:33 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 6ED0E5C3A5E for ; Mon, 30 Sep 2019 11:57:31 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 6FCB21005EEB; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 6E802A9; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:01 -0400 Message-Id: <1569869810-23848-103-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 102/151] lustre: libcfs: remove cfs_time_XXX_64 wrappers X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: James Simmons , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" In an attempt to support 64 bit time handling before the linux kernel developed time64_t and ktime lustre attempted to use 64 bit jiffies with a libcfs abstraction. Lets remove these wrappers and replace them with modern 64 bit time support. The lustre code that used these wrappers needs time resolution at the seconds level so replace the code with time64_t handling. WC-bug-id: https://jira.whamcloud.com/browse/LU-9019 Lustre-commit: 2728de4db8c0 ("LU-9019 libcfs: remove cfs_time_XXX_64 wrappers") Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/30867 Reviewed-by: Andreas Dilger Reviewed-by: Dmitry Eremin Signed-off-by: James Simmons --- fs/lustre/include/lustre_import.h | 10 ++++---- fs/lustre/include/obd.h | 6 ++--- fs/lustre/include/obd_class.h | 14 +++++------ fs/lustre/llite/llite_internal.h | 2 +- fs/lustre/llite/llite_lib.c | 6 ++--- fs/lustre/llite/lproc_llite.c | 12 +++++----- fs/lustre/lmv/lmv_obd.c | 4 ++-- fs/lustre/lov/lov_obd.c | 4 ++-- fs/lustre/lov/lov_request.c | 4 ++-- fs/lustre/mdc/mdc_request.c | 4 ++-- fs/lustre/obdclass/genops.c | 2 +- fs/lustre/obdclass/lprocfs_status.c | 48 +++++++++++++++++++++++-------------- fs/lustre/osc/osc_request.c | 4 ++-- fs/lustre/ptlrpc/import.c | 10 ++++---- 14 files changed, 70 insertions(+), 60 deletions(-) diff --git a/fs/lustre/include/lustre_import.h b/fs/lustre/include/lustre_import.h index 6a2d402..2d8f6be 100644 --- a/fs/lustre/include/lustre_import.h +++ b/fs/lustre/include/lustre_import.h @@ -134,10 +134,10 @@ struct obd_import_conn { struct ptlrpc_connection *oic_conn; /** uuid of remote side */ struct obd_uuid oic_uuid; - /** - * Time (64 bit jiffies) of last connection attempt on this connection + /* + * Time (64 bit seconds) of last connection attempt on this connection */ - u64 oic_last_attempt; + time64_t oic_last_attempt; }; /* state history */ @@ -251,8 +251,8 @@ struct obd_import { struct lustre_handle imp_remote_handle; /** When to perform next ping. time in jiffies. */ time64_t imp_next_ping; - /** When we last successfully connected. time in 64bit jiffies */ - u64 imp_last_success_conn; + /** When we last successfully connected. time in 64bit seconds */ + time64_t imp_last_success_conn; /** List of all possible connection for import. */ struct list_head imp_conn_list; diff --git a/fs/lustre/include/obd.h b/fs/lustre/include/obd.h index 309a168..1cb37a0 100644 --- a/fs/lustre/include/obd.h +++ b/fs/lustre/include/obd.h @@ -587,7 +587,7 @@ struct obd_device { spinlock_t obd_dev_lock; /* protect OBD bitfield above */ spinlock_t obd_osfs_lock; struct obd_statfs obd_osfs; /* locked by obd_osfs_lock */ - u64 obd_osfs_age; + time64_t obd_osfs_age; u64 obd_last_committed; struct mutex obd_dev_mutex; struct lvfs_run_ctxt obd_lvfs_ctxt; @@ -850,9 +850,9 @@ struct obd_ops { * about this. */ int (*statfs)(const struct lu_env *, struct obd_export *exp, - struct obd_statfs *osfs, u64 max_age, u32 flags); + struct obd_statfs *osfs, time64_t max_age, u32 flags); int (*statfs_async)(struct obd_export *exp, struct obd_info *oinfo, - u64 max_age, struct ptlrpc_request_set *set); + time64_t max_age, struct ptlrpc_request_set *set); int (*create)(const struct lu_env *env, struct obd_export *exp, struct obdo *oa); int (*destroy)(const struct lu_env *env, struct obd_export *exp, diff --git a/fs/lustre/include/obd_class.h b/fs/lustre/include/obd_class.h index f977b2e..dd07ec5 100644 --- a/fs/lustre/include/obd_class.h +++ b/fs/lustre/include/obd_class.h @@ -874,7 +874,7 @@ static inline int obd_destroy_export(struct obd_export *exp) */ static inline int obd_statfs_async(struct obd_export *exp, struct obd_info *oinfo, - u64 max_age, + time64_t max_age, struct ptlrpc_request_set *rqset) { int rc = 0; @@ -889,9 +889,9 @@ static inline int obd_statfs_async(struct obd_export *exp, return -EOPNOTSUPP; } - CDEBUG(D_SUPER, "%s: osfs %p age %llu, max_age %llu\n", + CDEBUG(D_SUPER, "%s: osfs %p age %lld, max_age %lld\n", obd->obd_name, &obd->obd_osfs, obd->obd_osfs_age, max_age); - if (time_before64(obd->obd_osfs_age, max_age)) { + if (obd->obd_osfs_age < max_age) { rc = OBP(obd, statfs_async)(exp, oinfo, max_age, rqset); } else { CDEBUG(D_SUPER, @@ -915,7 +915,7 @@ static inline int obd_statfs_async(struct obd_export *exp, * target. Use a value of "jiffies + HZ" to guarantee freshness. */ static inline int obd_statfs(const struct lu_env *env, struct obd_export *exp, - struct obd_statfs *osfs, u64 max_age, + struct obd_statfs *osfs, time64_t max_age, u32 flags) { struct obd_device *obd = exp->exp_obd; @@ -929,14 +929,14 @@ static inline int obd_statfs(const struct lu_env *env, struct obd_export *exp, return -EOPNOTSUPP; } - CDEBUG(D_SUPER, "osfs %llu, max_age %llu\n", + CDEBUG(D_SUPER, "osfs %lld, max_age %lld\n", obd->obd_osfs_age, max_age); - if (time_before64(obd->obd_osfs_age, max_age)) { + if (obd->obd_osfs_age < max_age) { rc = OBP(obd, statfs)(env, exp, osfs, max_age, flags); if (rc == 0) { spin_lock(&obd->obd_osfs_lock); memcpy(&obd->obd_osfs, osfs, sizeof(obd->obd_osfs)); - obd->obd_osfs_age = get_jiffies_64(); + obd->obd_osfs_age = ktime_get_seconds(); spin_unlock(&obd->obd_osfs_lock); } } else { diff --git a/fs/lustre/llite/llite_internal.h b/fs/lustre/llite/llite_internal.h index 229d417..fc4c4db 100644 --- a/fs/lustre/llite/llite_internal.h +++ b/fs/lustre/llite/llite_internal.h @@ -878,7 +878,7 @@ int ll_setattr_raw(struct dentry *dentry, struct iattr *attr, int ll_setattr(struct dentry *de, struct iattr *attr); int ll_statfs(struct dentry *de, struct kstatfs *sfs); int ll_statfs_internal(struct ll_sb_info *sbi, struct obd_statfs *osfs, - u64 max_age, u32 flags); + time64_t max_age, __u32 flags); int ll_update_inode(struct inode *inode, struct lustre_md *md); void ll_update_inode_flags(struct inode *inode, int ext_flags); int ll_read_inode2(struct inode *inode, void *opaque); diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index d4ff2bf..4f4234d 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -288,7 +288,7 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt) * available */ err = obd_statfs(NULL, sbi->ll_md_exp, osfs, - get_jiffies_64() - OBD_STATFS_CACHE_SECONDS * HZ, + ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, OBD_STATFS_FOR_MDT0); if (err) goto out_md_fid; @@ -1716,7 +1716,7 @@ int ll_setattr(struct dentry *de, struct iattr *attr) } int ll_statfs_internal(struct ll_sb_info *sbi, struct obd_statfs *osfs, - u64 max_age, u32 flags) + time64_t max_age, u32 flags) { struct obd_statfs obd_osfs; int rc; @@ -1775,7 +1775,7 @@ int ll_statfs(struct dentry *de, struct kstatfs *sfs) /* Some amount of caching on the client is allowed */ rc = ll_statfs_internal(ll_s2sbi(sb), &osfs, - get_jiffies_64() - OBD_STATFS_CACHE_SECONDS * HZ, + ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, 0); if (rc) return rc; diff --git a/fs/lustre/llite/lproc_llite.c b/fs/lustre/llite/lproc_llite.c index 81c8fe6..9173d94 100644 --- a/fs/lustre/llite/lproc_llite.c +++ b/fs/lustre/llite/lproc_llite.c @@ -141,7 +141,7 @@ static ssize_t blocksize_show(struct kobject *kobj, struct attribute *attr, int rc; rc = ll_statfs_internal(sbi, &osfs, - get_jiffies_64() - OBD_STATFS_CACHE_SECONDS * HZ, + ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, OBD_STATFS_NODELAY); if (rc) return rc; @@ -193,7 +193,7 @@ static ssize_t kbytestotal_show(struct kobject *kobj, struct attribute *attr, int rc; rc = ll_statfs_internal(sbi, &osfs, - get_jiffies_64() - OBD_STATFS_CACHE_SECONDS * HZ, + ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, OBD_STATFS_NODELAY); if (rc) return rc; @@ -219,7 +219,7 @@ static ssize_t kbytesfree_show(struct kobject *kobj, struct attribute *attr, int rc; rc = ll_statfs_internal(sbi, &osfs, - get_jiffies_64() - OBD_STATFS_CACHE_SECONDS * HZ, + ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, OBD_STATFS_NODELAY); if (rc) return rc; @@ -245,7 +245,7 @@ static ssize_t kbytesavail_show(struct kobject *kobj, struct attribute *attr, int rc; rc = ll_statfs_internal(sbi, &osfs, - get_jiffies_64() - OBD_STATFS_CACHE_SECONDS * HZ, + ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, OBD_STATFS_NODELAY); if (rc) return rc; @@ -269,7 +269,7 @@ static ssize_t filestotal_show(struct kobject *kobj, struct attribute *attr, int rc; rc = ll_statfs_internal(sbi, &osfs, - get_jiffies_64() - OBD_STATFS_CACHE_SECONDS * HZ, + ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, OBD_STATFS_NODELAY); if (rc) return rc; @@ -287,7 +287,7 @@ static ssize_t filesfree_show(struct kobject *kobj, struct attribute *attr, int rc; rc = ll_statfs_internal(sbi, &osfs, - get_jiffies_64() - OBD_STATFS_CACHE_SECONDS * HZ, + ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, OBD_STATFS_NODELAY); if (rc) return rc; diff --git a/fs/lustre/lmv/lmv_obd.c b/fs/lustre/lmv/lmv_obd.c index 9a1064b..d0f626f 100644 --- a/fs/lustre/lmv/lmv_obd.c +++ b/fs/lustre/lmv/lmv_obd.c @@ -893,7 +893,7 @@ static int lmv_iocontrol(unsigned int cmd, struct obd_export *exp, return -EFAULT; rc = obd_statfs(NULL, tgt->ltd_exp, &stat_buf, - get_jiffies_64() - OBD_STATFS_CACHE_SECONDS * HZ, + ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, 0); if (rc) return rc; @@ -1326,7 +1326,7 @@ static int lmv_process_config(struct obd_device *obd, u32 len, void *buf) } static int lmv_statfs(const struct lu_env *env, struct obd_export *exp, - struct obd_statfs *osfs, u64 max_age, u32 flags) + struct obd_statfs *osfs, time64_t max_age, u32 flags) { struct obd_device *obd = class_exp2obd(exp); struct lmv_obd *lmv = &obd->u.lmv; diff --git a/fs/lustre/lov/lov_obd.c b/fs/lustre/lov/lov_obd.c index acfee93..1708fa9 100644 --- a/fs/lustre/lov/lov_obd.c +++ b/fs/lustre/lov/lov_obd.c @@ -915,7 +915,7 @@ int lov_process_config_base(struct obd_device *obd, struct lustre_cfg *lcfg, } static int lov_statfs(const struct lu_env *env, struct obd_export *exp, - struct obd_statfs *osfs, u64 max_age, u32 flags) + struct obd_statfs *osfs, time64_t max_age, u32 flags) { struct obd_device *obd = class_exp2obd(exp); struct lov_obd *lov = &obd->u.lov; @@ -1001,7 +1001,7 @@ static int lov_iocontrol(unsigned int cmd, struct obd_export *exp, int len, /* got statfs data */ rc = obd_statfs(NULL, lov->lov_tgts[index]->ltd_exp, &stat_buf, - get_jiffies_64() - OBD_STATFS_CACHE_SECONDS * HZ, + ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, flags); if (rc) return rc; diff --git a/fs/lustre/lov/lov_request.c b/fs/lustre/lov/lov_request.c index 057d6ca..864e410 100644 --- a/fs/lustre/lov/lov_request.c +++ b/fs/lustre/lov/lov_request.c @@ -158,7 +158,7 @@ static int lov_fini_statfs(struct obd_device *obd, struct obd_statfs *osfs, spin_lock(&obd->obd_osfs_lock); memcpy(&obd->obd_osfs, osfs, sizeof(*osfs)); - obd->obd_osfs_age = get_jiffies_64(); + obd->obd_osfs_age = ktime_get_seconds(); spin_unlock(&obd->obd_osfs_lock); return 0; } @@ -275,7 +275,7 @@ static int cb_statfs_update(void *cookie, int rc) spin_lock(&tgtobd->obd_osfs_lock); memcpy(&tgtobd->obd_osfs, lov_sfs, sizeof(*lov_sfs)); if ((oinfo->oi_flags & OBD_STATFS_FROM_CACHE) == 0) - tgtobd->obd_osfs_age = get_jiffies_64(); + tgtobd->obd_osfs_age = ktime_get_seconds(); spin_unlock(&tgtobd->obd_osfs_lock); out_update: diff --git a/fs/lustre/mdc/mdc_request.c b/fs/lustre/mdc/mdc_request.c index 95698c3..dd9ab5a 100644 --- a/fs/lustre/mdc/mdc_request.c +++ b/fs/lustre/mdc/mdc_request.c @@ -1439,7 +1439,7 @@ static int mdc_read_page(struct obd_export *exp, struct md_op_data *op_data, static int mdc_statfs(const struct lu_env *env, struct obd_export *exp, struct obd_statfs *osfs, - u64 max_age, u32 flags) + time64_t max_age, u32 flags) { struct obd_device *obd = class_exp2obd(exp); struct ptlrpc_request *req; @@ -2002,7 +2002,7 @@ static int mdc_iocontrol(unsigned int cmd, struct obd_export *exp, int len, } rc = mdc_statfs(NULL, obd->obd_self_export, &stat_buf, - get_jiffies_64() - OBD_STATFS_CACHE_SECONDS * HZ, + ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, 0); if (rc != 0) goto out; diff --git a/fs/lustre/obdclass/genops.c b/fs/lustre/obdclass/genops.c index 6491032..f809d00c 100644 --- a/fs/lustre/obdclass/genops.c +++ b/fs/lustre/obdclass/genops.c @@ -307,7 +307,7 @@ struct obd_device *class_newdev(const char *type_name, const char *name, /* newdev->obd_osfs_age must be set to a value in the distant * past to guarantee a fresh statfs is fetched on mount. */ - newdev->obd_osfs_age = get_jiffies_64() - 1000 * HZ; + newdev->obd_osfs_age = ktime_get_seconds() - 1000; /* XXX belongs in setup not attach */ init_rwsem(&newdev->obd_observer_link_sem); diff --git a/fs/lustre/obdclass/lprocfs_status.c b/fs/lustre/obdclass/lprocfs_status.c index dbbaa92..135926e 100644 --- a/fs/lustre/obdclass/lprocfs_status.c +++ b/fs/lustre/obdclass/lprocfs_status.c @@ -351,9 +351,11 @@ static ssize_t blocksize_show(struct kobject *kobj, struct attribute *attr, struct obd_device *obd = container_of(kobj, struct obd_device, obd_kset.kobj); struct obd_statfs osfs; - int rc = obd_statfs(NULL, obd->obd_self_export, &osfs, - get_jiffies_64() - OBD_STATFS_CACHE_SECONDS * HZ, - OBD_STATFS_NODELAY); + int rc; + + rc = obd_statfs(NULL, obd->obd_self_export, &osfs, + ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, + OBD_STATFS_NODELAY); if (!rc) return sprintf(buf, "%u\n", osfs.os_bsize); @@ -367,9 +369,11 @@ static ssize_t kbytestotal_show(struct kobject *kobj, struct attribute *attr, struct obd_device *obd = container_of(kobj, struct obd_device, obd_kset.kobj); struct obd_statfs osfs; - int rc = obd_statfs(NULL, obd->obd_self_export, &osfs, - get_jiffies_64() - OBD_STATFS_CACHE_SECONDS * HZ, - OBD_STATFS_NODELAY); + int rc; + + rc = obd_statfs(NULL, obd->obd_self_export, &osfs, + ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, + OBD_STATFS_NODELAY); if (!rc) { u32 blk_size = osfs.os_bsize >> 10; u64 result = osfs.os_blocks; @@ -388,9 +392,11 @@ static ssize_t kbytesfree_show(struct kobject *kobj, struct attribute *attr, struct obd_device *obd = container_of(kobj, struct obd_device, obd_kset.kobj); struct obd_statfs osfs; - int rc = obd_statfs(NULL, obd->obd_self_export, &osfs, - get_jiffies_64() - OBD_STATFS_CACHE_SECONDS * HZ, - OBD_STATFS_NODELAY); + int rc; + + rc = obd_statfs(NULL, obd->obd_self_export, &osfs, + ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, + OBD_STATFS_NODELAY); if (!rc) { u32 blk_size = osfs.os_bsize >> 10; u64 result = osfs.os_bfree; @@ -410,9 +416,11 @@ static ssize_t kbytesavail_show(struct kobject *kobj, struct attribute *attr, struct obd_device *obd = container_of(kobj, struct obd_device, obd_kset.kobj); struct obd_statfs osfs; - int rc = obd_statfs(NULL, obd->obd_self_export, &osfs, - get_jiffies_64() - OBD_STATFS_CACHE_SECONDS * HZ, - OBD_STATFS_NODELAY); + int rc; + + rc = obd_statfs(NULL, obd->obd_self_export, &osfs, + ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, + OBD_STATFS_NODELAY); if (!rc) { u32 blk_size = osfs.os_bsize >> 10; u64 result = osfs.os_bavail; @@ -432,9 +440,11 @@ static ssize_t filestotal_show(struct kobject *kobj, struct attribute *attr, struct obd_device *obd = container_of(kobj, struct obd_device, obd_kset.kobj); struct obd_statfs osfs; - int rc = obd_statfs(NULL, obd->obd_self_export, &osfs, - get_jiffies_64() - OBD_STATFS_CACHE_SECONDS * HZ, - OBD_STATFS_NODELAY); + int rc; + + rc = obd_statfs(NULL, obd->obd_self_export, &osfs, + ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, + OBD_STATFS_NODELAY); if (!rc) return sprintf(buf, "%llu\n", osfs.os_files); @@ -448,9 +458,11 @@ static ssize_t filesfree_show(struct kobject *kobj, struct attribute *attr, struct obd_device *obd = container_of(kobj, struct obd_device, obd_kset.kobj); struct obd_statfs osfs; - int rc = obd_statfs(NULL, obd->obd_self_export, &osfs, - get_jiffies_64() - OBD_STATFS_CACHE_SECONDS * HZ, - OBD_STATFS_NODELAY); + int rc; + + rc = obd_statfs(NULL, obd->obd_self_export, &osfs, + ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, + OBD_STATFS_NODELAY); if (!rc) return sprintf(buf, "%llu\n", osfs.os_ffree); diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index 0f7ac2d..5797535 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -2471,7 +2471,7 @@ static int osc_statfs_interpret(const struct lu_env *env, } static int osc_statfs_async(struct obd_export *exp, - struct obd_info *oinfo, u64 max_age, + struct obd_info *oinfo, time64_t max_age, struct ptlrpc_request_set *rqset) { struct obd_device *obd = class_exp2obd(exp); @@ -2515,7 +2515,7 @@ static int osc_statfs_async(struct obd_export *exp, } static int osc_statfs(const struct lu_env *env, struct obd_export *exp, - struct obd_statfs *osfs, u64 max_age, u32 flags) + struct obd_statfs *osfs, time64_t max_age, u32 flags) { struct obd_device *obd = class_exp2obd(exp); struct obd_statfs *msfs; diff --git a/fs/lustre/ptlrpc/import.c b/fs/lustre/ptlrpc/import.c index 3c56798..152a84c 100644 --- a/fs/lustre/ptlrpc/import.c +++ b/fs/lustre/ptlrpc/import.c @@ -462,7 +462,7 @@ static int import_select_connection(struct obd_import *imp) } list_for_each_entry(conn, &imp->imp_conn_list, oic_item) { - CDEBUG(D_HA, "%s: connect to NID %s last attempt %llu\n", + CDEBUG(D_HA, "%s: connect to NID %s last attempt %lld\n", imp->imp_obd->obd_name, libcfs_nid2str(conn->oic_conn->c_peer.nid), conn->oic_last_attempt); @@ -471,8 +471,7 @@ static int import_select_connection(struct obd_import *imp) * the last successful attempt, go with this one */ if ((conn->oic_last_attempt == 0) || - time_before_eq64(conn->oic_last_attempt, - imp->imp_last_success_conn)) { + conn->oic_last_attempt <= imp->imp_last_success_conn) { imp_conn = conn; tried_all = 0; break; @@ -484,8 +483,7 @@ static int import_select_connection(struct obd_import *imp) */ if (!imp_conn) imp_conn = conn; - else if (time_before64(conn->oic_last_attempt, - imp_conn->oic_last_attempt)) + else if (imp_conn->oic_last_attempt > conn->oic_last_attempt) imp_conn = conn; } @@ -517,7 +515,7 @@ static int import_select_connection(struct obd_import *imp) imp->imp_obd->obd_name, at_get(at)); } - imp_conn->oic_last_attempt = get_jiffies_64(); + imp_conn->oic_last_attempt = ktime_get_seconds(); /* switch connection, don't mind if it's same as the current one */ ptlrpc_connection_put(imp->imp_connection); From patchwork Mon Sep 30 18:56:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167191 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D0E35912 for ; Mon, 30 Sep 2019 19:02:28 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B9523224EF for ; Mon, 30 Sep 2019 19:02:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B9523224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 19E675E4479; Mon, 30 Sep 2019 11:59:22 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id C40AB5C3A5E for ; Mon, 30 Sep 2019 11:57:31 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 729AC1005EF4; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 71732BB; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:02 -0400 Message-Id: <1569869810-23848-104-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 103/151] lustre: address issues raised by gcc7 X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: James Simmons , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" Starting with gcc version 7 several platforms have enabled new flags to report potential problems when compling code. For lustre much of the reported problems deal with potential buffer overruns. Address these issues in struct lustre_sb_info and provide info about the fields. WC-bug-id: https://jira.whamcloud.com/browse/LU-9934 Linux-commit: 61743b8867268 ("LU-9934 build: address issues raised by gcc7") Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/30376 Reviewed-by: John L. Hammond Reviewed-by: Dmitry Eremin Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_disk.h | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/fs/lustre/include/lustre_disk.h b/fs/lustre/include/lustre_disk.h index 8b4a243..92618e8 100644 --- a/fs/lustre/include/lustre_disk.h +++ b/fs/lustre/include/lustre_disk.h @@ -125,8 +125,10 @@ struct lustre_sb_info { atomic_t lsi_mounts; /* references to the srv_mnt */ struct kobject *lsi_kobj; char lsi_svname[MTI_NAME_MAXLEN]; - char lsi_osd_obdname[64]; - char lsi_osd_uuid[64]; + /* lsi_osd_obdname format = 'lsi->ls_svname'-osd */ + char lsi_osd_obdname[MTI_NAME_MAXLEN + 4]; + /* lsi_osd_uuid format = 'lsi->ls_osd_obdname'_UUID */ + char lsi_osd_uuid[MTI_NAME_MAXLEN + 9]; struct obd_export *lsi_osd_exp; char lsi_osd_type[16]; char lsi_fstype[16]; From patchwork Mon Sep 30 18:56:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167249 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6E9FF13B1 for ; Mon, 30 Sep 2019 19:05:10 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 56963224EF for ; Mon, 30 Sep 2019 19:05:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 56963224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 6E15D5E4975; Mon, 30 Sep 2019 12:00:27 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 11C485C3B64 for ; Mon, 30 Sep 2019 11:57:32 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 76A5B1005EF5; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 7527EBD; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:03 -0400 Message-Id: <1569869810-23848-105-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 104/151] lustre: lov: fill no-extent fiemap on object with no stripe. X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dominique Martinet , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Dominique Martinet This is useful for cp of large sparse files with no stripe info, as cp relies on fiemap to detect what to read. WC-bug-id: https://jira.whamcloud.com/browse/LU-10405 Lustre-commit: 113fa79b7dc9 ("LU-10405 lov: fill no-extent fiemap on object with no stripe") Signed-off-by: Dominique Martinet Reviewed-on: https://review.whamcloud.com/30591 Reviewed-by: Quentin Bouget Reviewed-by: Andreas Dilger Signed-off-by: James Simmons --- fs/lustre/lov/lov_object.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/fs/lustre/lov/lov_object.c b/fs/lustre/lov/lov_object.c index d3ed5fc..51b95f2 100644 --- a/fs/lustre/lov/lov_object.c +++ b/fs/lustre/lov/lov_object.c @@ -1774,8 +1774,11 @@ static int lov_object_fiemap(const struct lu_env *env, struct cl_object *obj, struct fiemap_state fs = { NULL }; lsm = lov_lsm_addref(cl2lov(obj)); - if (!lsm) - return -ENODATA; + if (!lsm) { + /* no extent: there is no object for mapping */ + fiemap->fm_mapped_extents = 0; + return 0; + } if (!(fiemap->fm_flags & FIEMAP_FLAG_DEVICE_ORDER)) { /** From patchwork Mon Sep 30 18:56:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167257 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8591313B1 for ; Mon, 30 Sep 2019 19:05:27 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6BC92224EF for ; Mon, 30 Sep 2019 19:05:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6BC92224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8610A5E4A42; Mon, 30 Sep 2019 12:00:34 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 571A75C3B35 for ; Mon, 30 Sep 2019 11:57:32 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 7B03B1005EF6; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 78D7FB4; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:04 -0400 Message-Id: <1569869810-23848-106-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 105/151] lustre: ptlrpc: allow to limit number of service's rqbds X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Wang Shilong , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Bruno Faccini This patch provides a way to limit the number of rqbds per service. This should help to avoid OOM during heavy clients requests load, like during target failover/recovery for thousands of Clients. This change has been required, even after first patch for LU-9372 (ptlrpc: drain "ptlrpc_request_buffer_desc" objects) which already allowed to drain unused rqbds previously allocated during heavy load, but was not efficient during too long period of load. WC-bug-id: https://jira.whamcloud.com/browse/LU-9372 Lustre-commit: d9e57a765e73 ("LU-9372 ptlrpc: allow to limit number of service's rqbds") Signed-off-by: Bruno Faccini Reviewed-on: https://review.whamcloud.com/29064 WC-bug-id: https://jira.whamcloud.com/browse/LU-10603 Lustre-commit: 3542f9780bf4 ("LU-10603 ptlrpc: export req_buffers_max via procfs") Signed-off-by: Alex Zhuravlev Reviewed-on: https://review.whamcloud.com/31162 WC-bug-id: https://jira.whamcloud.com/browse/LU-10803 Lustre-commit: c11c61a82d66 ("LU-10803 ptlrpc: fix req_buffers_max and req_history_max setting") Signed-off-by: Wang Shilong Reviewed-on: https://review.whamcloud.com/31622 Reviewed-by: Andreas Dilger Reviewed-by: Dmitry Eremin Reviewed-by: Alex Zhuravlev Reviewed-by: Faccini Bruno Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_net.h | 2 ++ fs/lustre/ptlrpc/lproc_ptlrpc.c | 52 ++++++++++++++++++++++++++++++++++++++--- fs/lustre/ptlrpc/service.c | 14 ++++++++--- 3 files changed, 62 insertions(+), 6 deletions(-) diff --git a/fs/lustre/include/lustre_net.h b/fs/lustre/include/lustre_net.h index baa1eb8..22c9668 100644 --- a/fs/lustre/include/lustre_net.h +++ b/fs/lustre/include/lustre_net.h @@ -1482,6 +1482,8 @@ struct ptlrpc_service { /** under unregister_service */ unsigned srv_is_stopping:1; + /** max # request buffers */ + int srv_nrqbds_max; /** max # request buffers in history per partition */ int srv_hist_nrqbds_cpt_max; /** number of CPTs this service bound on */ diff --git a/fs/lustre/ptlrpc/lproc_ptlrpc.c b/fs/lustre/ptlrpc/lproc_ptlrpc.c index 163ee54..2239680 100644 --- a/fs/lustre/ptlrpc/lproc_ptlrpc.c +++ b/fs/lustre/ptlrpc/lproc_ptlrpc.c @@ -298,10 +298,16 @@ static const char *ll_eopcode2str(u32 opcode) /* This sanity check is more of an insanity check; we can still * hose a kernel by allowing the request history to grow too - * far. + * far. The roundup to the next power of two is an empirical way + * to take care that request buffer is allocated in Slab and thus + * will be upgraded. */ - bufpages = (svc->srv_buf_size + PAGE_SIZE - 1) >> PAGE_SHIFT; - if (val > totalram_pages() / (2 * bufpages)) + bufpages = (roundup_pow_of_two(svc->srv_buf_size) + PAGE_SIZE - 1) >> + PAGE_SHIFT; + /* do not allow history to consume more than half max number of rqbds */ + if ((svc->srv_nrqbds_max == 0 && + val > totalram_pages() / (2 * bufpages)) || + (svc->srv_nrqbds_max != 0 && val > svc->srv_nrqbds_max / 2)) return -ERANGE; spin_lock(&svc->srv_lock); @@ -318,6 +324,43 @@ static const char *ll_eopcode2str(u32 opcode) LPROC_SEQ_FOPS(ptlrpc_lprocfs_req_history_max); +static int +ptlrpc_lprocfs_req_buffers_max_seq_show(struct seq_file *m, void *n) +{ + struct ptlrpc_service *svc = m->private; + + seq_printf(m, "%d\n", svc->srv_nrqbds_max); + return 0; +} + +static ssize_t +ptlrpc_lprocfs_req_buffers_max_seq_write(struct file *file, + const char __user *buffer, + size_t count, loff_t *off) +{ + struct seq_file *m = file->private_data; + struct ptlrpc_service *svc = m->private; + int val; + int rc; + + rc = kstrtoint_from_user(buffer, count, 0, &val); + if (rc < 0) + return rc; + + if (val < svc->srv_nbuf_per_group && val != 0) + return -ERANGE; + + spin_lock(&svc->srv_lock); + + svc->srv_nrqbds_max = (uint)val; + + spin_unlock(&svc->srv_lock); + + return count; +} + +LPROC_SEQ_FOPS(ptlrpc_lprocfs_req_buffers_max); + static ssize_t threads_min_show(struct kobject *kobj, struct attribute *attr, char *buf) { @@ -1090,6 +1133,9 @@ void ptlrpc_ldebugfs_register_service(struct dentry *entry, { .name = "nrs_policies", .fops = &ptlrpc_lprocfs_nrs_fops, .data = svc }, + { .name = "req_buffers_max", + .fops = &ptlrpc_lprocfs_req_buffers_max_fops, + .data = svc }, { NULL } }; static const struct file_operations req_history_fops = { diff --git a/fs/lustre/ptlrpc/service.c b/fs/lustre/ptlrpc/service.c index 90c7529..57c5e28 100644 --- a/fs/lustre/ptlrpc/service.c +++ b/fs/lustre/ptlrpc/service.c @@ -148,7 +148,10 @@ /* NB: another thread might have recycled enough rqbds, we * need to make sure it wouldn't over-allocate, see LU-1212. */ - if (svcpt->scp_nrqbds_posted >= svc->srv_nbuf_per_group) + if (test_req_buffer_pressure || + svcpt->scp_nrqbds_posted >= svc->srv_nbuf_per_group || + (svc->srv_nrqbds_max != 0 && + svcpt->scp_nrqbds_total > svc->srv_nrqbds_max)) break; rqbd = ptlrpc_alloc_rqbd(svcpt); @@ -622,6 +625,9 @@ struct ptlrpc_service * /* buffer configuration */ service->srv_nbuf_per_group = test_req_buffer_pressure ? 1 : conf->psc_buf.bc_nbufs; + /* do not limit max number of rqbds by default */ + service->srv_nrqbds_max = 0; + service->srv_max_req_size = conf->psc_buf.bc_req_max_size + SPTLRPC_MAX_PAYLOAD; service->srv_buf_size = conf->psc_buf.bc_buf_size; @@ -807,8 +813,10 @@ static void ptlrpc_server_drop_request(struct ptlrpc_request *req) */ LASSERT(atomic_read(&rqbd->rqbd_req.rq_refcount) == 0); - if (svcpt->scp_nrqbds_posted >= svc->srv_nbuf_per_group && - !test_req_buffer_pressure) { + if (svcpt->scp_nrqbds_posted >= svc->srv_nbuf_per_group || + (svc->srv_nrqbds_max != 0 && + svcpt->scp_nrqbds_total > svc->srv_nrqbds_max) || + test_req_buffer_pressure) { /* like in ptlrpc_free_rqbd() */ svcpt->scp_nrqbds_total--; kvfree(rqbd->rqbd_buffer); From patchwork Mon Sep 30 18:56:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167253 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7AC9B1599 for ; Mon, 30 Sep 2019 19:05:19 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6294F224EF for ; Mon, 30 Sep 2019 19:05:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6294F224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 344135E49F0; Mon, 30 Sep 2019 12:00:31 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id ACEC85C3B41 for ; Mon, 30 Sep 2019 11:57:32 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 7D0011005EF7; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 7BC86A9; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:05 -0400 Message-Id: <1569869810-23848-107-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 106/151] lnet: ensure peer put back on dc request queue X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Bruno Faccini Upon async PUT request received from peer already in discovery process, lnet_peer_push_event() was not handling the case where peer could be on working/ln_dc_working queue. This could lead for peer not to be re-dsicovered as expected, but left on working queue and to be finally timed-out. Also ensure that peer will not be put back on request queue by event handler if discovery is already completed. WC-bug-id: https://jira.whamcloud.com/browse/LU-10123 Lustre-commit: d0185dd43394 ("LU-10123 lnet: ensure peer put back on dc request queue") Signed-off-by: Bruno Faccini Reviewed-on: https://review.whamcloud.com/30147 Reviewed-by: Amir Shehata Reviewed-by: Dmitry Eremin Reviewed-by: Olaf Weber Reviewed-by: Doug Oucharek Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/lnet/peer.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/net/lnet/lnet/peer.c b/net/lnet/lnet/peer.c index 52d4ec0..e2f8c28 100644 --- a/net/lnet/lnet/peer.c +++ b/net/lnet/lnet/peer.c @@ -1983,13 +1983,16 @@ void lnet_peer_push_event(struct lnet_event *ev) out: /* - * Queue the peer for discovery, and wake the discovery thread - * if the peer was already queued, because its status changed. + * Queue the peer for discovery if not done, force it on the request + * queue and wake the discovery thread if the peer was already queued, + * because its status changed. */ spin_unlock(&lp->lp_lock); lnet_net_lock(LNET_LOCK_EX); - if (lnet_peer_queue_for_discovery(lp)) + if (!lnet_peer_is_uptodate(lp) && lnet_peer_queue_for_discovery(lp)) { + list_move(&lp->lp_dc_list, &the_lnet.ln_dc_request); wake_up(&the_lnet.ln_dc_waitq); + } /* Drop refcount from lookup */ lnet_peer_decref_locked(lp); lnet_net_unlock(LNET_LOCK_EX); @@ -2348,7 +2351,11 @@ static void lnet_discovery_event_handler(struct lnet_event *event) lnet_ping_buffer_decref(pbuf); lnet_peer_decref_locked(lp); } - if (rc == LNET_REDISCOVER_PEER) { + + /* Put peer back at end of request queue, if discovery not already + * done + */ + if (rc == LNET_REDISCOVER_PEER && !lnet_peer_is_uptodate(lp)) { list_move_tail(&lp->lp_dc_list, &the_lnet.ln_dc_request); wake_up(&the_lnet.ln_dc_waitq); } From patchwork Mon Sep 30 18:56:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167259 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E1E9716B1 for ; Mon, 30 Sep 2019 19:05:27 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C93D7224EF for ; Mon, 30 Sep 2019 19:05:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C93D7224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id DA5995E4A4A; Mon, 30 Sep 2019 12:00:34 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id EDFB05C39D2 for ; Mon, 30 Sep 2019 11:57:32 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 8043F1005EF8; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 7EA63B5; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:06 -0400 Message-Id: <1569869810-23848-108-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 107/151] lustre: recovery: support setstripe replay X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lai Siyao , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Lai Siyao Regular file open will always reserve space for LOV ea, which is used to store user specified lov_user_md, or lov_mds_md for replay, but if this open is the first open in 'lfs setstripe', it doesn't have lov_user_md specified, or lov_mds_md for replay because O_LOV_DELAY_CREATE is set, but MDT will treat the EA field in the request as valid one, so fails in magic check in this open replay. This patch contains the fix for the client side. 1. client doesn't reserve space for LOV ea in open(O_LOV_DELAY_CREATE), this change is not necessary, but to make clean of the code. WC-bug-id: https://jira.whamcloud.com/browse/LU-10155 Lustre-commit: 3a0a50f44f68 ("LU-10155 recovery: support setstripe replay") Signed-off-by: Lai Siyao Reviewed-on: https://review.whamcloud.com/30704 Reviewed-by: Mike Pershin Reviewed-by: Jinshan Xiong Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/mdc/mdc_locks.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/fs/lustre/mdc/mdc_locks.c b/fs/lustre/mdc/mdc_locks.c index 0b358b6..2eb6e8a 100644 --- a/fs/lustre/mdc/mdc_locks.c +++ b/fs/lustre/mdc/mdc_locks.c @@ -297,8 +297,14 @@ static int mdc_save_lovea(struct ptlrpc_request *req, req_capsule_set_size(&req->rq_pill, &RMF_NAME, RCL_CLIENT, op_data->op_namelen + 1); - req_capsule_set_size(&req->rq_pill, &RMF_EADATA, RCL_CLIENT, - max(lmmsize, obddev->u.cli.cl_default_mds_easize)); + if (cl_is_lov_delay_create(it->it_flags)) { + /* open(O_LOV_DELAY_CREATE) won't pack lmm */ + LASSERT(lmmsize == 0); + req_capsule_set_size(&req->rq_pill, &RMF_EADATA, RCL_CLIENT, 0); + } else { + req_capsule_set_size(&req->rq_pill, &RMF_EADATA, RCL_CLIENT, + max(lmmsize, obddev->u.cli.cl_default_mds_easize)); + } req_capsule_set_size(&req->rq_pill, &RMF_FILE_SECCTX_NAME, RCL_CLIENT, op_data->op_file_secctx_name ? From patchwork Mon Sep 30 18:56:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167263 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4B37816B1 for ; Mon, 30 Sep 2019 19:05:37 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 33CFE224EF for ; Mon, 30 Sep 2019 19:05:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 33CFE224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 1C30C5E4A9E; Mon, 30 Sep 2019 12:00:38 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 3F3BA5C3B13 for ; Mon, 30 Sep 2019 11:57:33 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 843291005EF9; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 81A9EBB; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:07 -0400 Message-Id: <1569869810-23848-109-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 108/151] lustre: lustre: move LA_* flags to lustre_user.h X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Sebastien Buisson The LA_* flags are written to disk as part of the ChangeLog records in mdd_attr_set_changelog(), which means they are now part of the on-disk and network protocol, and cannot be changed (at least not the first 12 bits that are written). They need to be moved to lustre_user.h. WC-bug-id: https://jira.whamcloud.com/browse/LU-10485 Lustre-commit: 2b14e545001b ("LU-10485 lustre: move LA_* flags to lustre_user.h") Signed-off-by: Sebastien Buisson Reviewed-on: https://review.whamcloud.com/30825 Reviewed-by: Andreas Dilger Reviewed-by: Quentin Bouget Signed-off-by: James Simmons --- fs/lustre/include/lu_object.h | 19 ------------------- include/uapi/linux/lustre/lustre_user.h | 29 +++++++++++++++++++++++++++++ 2 files changed, 29 insertions(+), 19 deletions(-) diff --git a/fs/lustre/include/lu_object.h b/fs/lustre/include/lu_object.h index a938e64..e1979be 100644 --- a/fs/lustre/include/lu_object.h +++ b/fs/lustre/include/lu_object.h @@ -425,25 +425,6 @@ struct lu_attr { u64 la_valid; }; -/** Bit-mask of valid attributes */ -enum la_valid { - LA_ATIME = 1 << 0, - LA_MTIME = 1 << 1, - LA_CTIME = 1 << 2, - LA_SIZE = 1 << 3, - LA_MODE = 1 << 4, - LA_UID = 1 << 5, - LA_GID = 1 << 6, - LA_BLOCKS = 1 << 7, - LA_TYPE = 1 << 8, - LA_FLAGS = 1 << 9, - LA_NLINK = 1 << 10, - LA_RDEV = 1 << 11, - LA_BLKSIZE = 1 << 12, - LA_KILL_SUID = 1 << 13, - LA_KILL_SGID = 1 << 14, -}; - /** * Layer in the layered object. */ diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index 21dc1e9..d53e795 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -837,6 +837,35 @@ struct lustre_swap_layouts { __u64 sl_dv2; }; +/** Bit-mask of valid attributes */ +/* The LA_* flags are written to disk as part of the ChangeLog records + * so they are part of the on-disk and network protocol, and cannot be changed. + * Only the first 12 bits are currently saved. + */ +enum la_valid { + LA_ATIME = 1 << 0, + LA_MTIME = 1 << 1, + LA_CTIME = 1 << 2, + LA_SIZE = 1 << 3, + LA_MODE = 1 << 4, + LA_UID = 1 << 5, + LA_GID = 1 << 6, + LA_BLOCKS = 1 << 7, + LA_TYPE = 1 << 8, + LA_FLAGS = 1 << 9, + LA_NLINK = 1 << 10, + LA_RDEV = 1 << 11, + LA_BLKSIZE = 1 << 12, + LA_KILL_SUID = 1 << 13, + LA_KILL_SGID = 1 << 14, + LA_PROJID = 1 << 15, + LA_LAYOUT_VERSION = 1 << 16, + /** + * Attributes must be transmitted to OST objects + */ + LA_REMOTE_ATTR_SET = (LA_UID | LA_GID | LA_PROJID | LA_LAYOUT_VERSION) +}; + /********* Changelogs **********/ /** Changelog record types */ enum changelog_rec_type { From patchwork Mon Sep 30 18:56:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167209 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 027C6912 for ; Mon, 30 Sep 2019 19:03:34 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DE771224EF for ; Mon, 30 Sep 2019 19:03:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DE771224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 90D3821F6B7; Mon, 30 Sep 2019 11:59:43 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8207B5C3B20 for ; Mon, 30 Sep 2019 11:57:33 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 86DFC1005EFA; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 84903BD; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:08 -0400 Message-Id: <1569869810-23848-110-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 109/151] lustre: flr: revise lease API X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Jinshan Xiong Introduce new user land API LL_LEASE_LAYOUT_MERGE which allows direct control of the bias. Remove special MERGE_LAYOUTS_CLOSE. Instead always use SWAP_LAYOUTS_CLOSE which simplifies the code handling the layout swap on file close. WC-bug-id: https://jira.whamcloud.com/browse/LU-10418 Lustre-commit: 8f1c7c1e44a4 ("LU-10418 flr: revise lease API") Signed-off-by: Jinshan Xiong Reviewed-on: https://review.whamcloud.com/30363 Reviewed-by: Bobi Jam Reviewed-by: Andreas Dilger Signed-off-by: James Simmons --- fs/lustre/llite/file.c | 74 +++++++++++++++++++++++---------- include/uapi/linux/lustre/lustre_user.h | 3 +- 2 files changed, 53 insertions(+), 24 deletions(-) diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index f1db600..9d60587 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -970,12 +970,10 @@ static int ll_check_swap_layouts_validity(struct inode *inode1, } static int ll_swap_layouts_close(struct obd_client_handle *och, - struct inode *inode, struct inode *inode2, - int intent) + struct inode *inode, struct inode *inode2) { const struct lu_fid *fid1 = ll_inode2fid(inode); const struct lu_fid *fid2; - enum mds_op_bias bias; int rc; CDEBUG(D_INODE, "%s: biased close of file " DFID "\n", @@ -994,24 +992,13 @@ static int ll_swap_layouts_close(struct obd_client_handle *och, goto out_free_och; } - switch (intent) { - case SWAP_LAYOUTS_CLOSE: - bias = MDS_CLOSE_LAYOUT_SWAP; - break; - case MERGE_LAYOUTS_CLOSE: - bias = MDS_CLOSE_LAYOUT_MERGE; - break; - default: - rc = -EOPNOTSUPP; - goto out_free_och; - } - /* * Close the file and {swap,merge} layouts between inode & inode2. * NB: lease lock handle is released in mdc_close_layout_swap_pack() * because we still need it to pack l_remote_handle to MDT. */ - rc = ll_close_inode_openhandle(inode, och, bias, inode2); + rc = ll_close_inode_openhandle(inode, och, MDS_CLOSE_LAYOUT_SWAP, + inode2); och = NULL; /* freed in ll_close_inode_openhandle() */ @@ -2675,6 +2662,7 @@ static long ll_file_unlock_lease(struct file *file, struct ll_ioc_lease *ioc, bool lease_broken; fmode_t fmode = 0; enum mds_op_bias bias = 0; + struct file *layout_file = NULL; void *data = NULL; size_t data_size = 0; long rc; @@ -2693,7 +2681,8 @@ static long ll_file_unlock_lease(struct file *file, struct ll_ioc_lease *ioc, fmode = och->och_flags; - if (ioc->lil_flags & LL_LEASE_RESYNC_DONE) { + switch (ioc->lil_flags) { + case LL_LEASE_RESYNC_DONE: if (ioc->lil_count > IOC_IDS_MAX) { rc = -EINVAL; goto out; @@ -2712,6 +2701,40 @@ static long ll_file_unlock_lease(struct file *file, struct ll_ioc_lease *ioc, } bias = MDS_CLOSE_RESYNC_DONE; + break; + case LL_LEASE_LAYOUT_MERGE: { + int fd; + + if (ioc->lil_count != 1) { + rc = -EINVAL; + goto out; + } + + arg += sizeof(*ioc); + if (copy_from_user(&fd, (void __user *)arg, sizeof(u32))) { + rc = -EFAULT; + goto out; + } + + layout_file = fget(fd); + if (!layout_file) { + rc = -EBADF; + goto out; + } + + if ((file->f_flags & O_ACCMODE) == O_RDONLY || + (layout_file->f_flags & O_ACCMODE) == O_RDONLY) { + rc = -EPERM; + goto out; + } + + data = file_inode(layout_file); + bias = MDS_CLOSE_LAYOUT_MERGE; + break; + } + default: + /* without close intent */ + break; } rc = ll_lease_close_intent(och, inode, &lease_broken, bias, data); @@ -2726,7 +2749,16 @@ static long ll_file_unlock_lease(struct file *file, struct ll_ioc_lease *ioc, fmode = 0; out: - kfree(data); + switch (ioc->lil_flags) { + case LL_LEASE_RESYNC_DONE: + kfree(data); + break; + case LL_LEASE_LAYOUT_MERGE: + if (layout_file) + fput(layout_file); + break; + } + if (!rc) rc = ll_lease_type_from_fmode(fmode); return rc; @@ -2847,7 +2879,6 @@ static long ll_file_set_lease(struct file *file, struct ll_ioc_lease *ioc, case LL_IOC_LOV_SWAP_LAYOUTS: { struct file *file2; struct lustre_swap_layouts lsl; - u64 intent; if (copy_from_user(&lsl, (char __user *)arg, sizeof(struct lustre_swap_layouts))) @@ -2866,8 +2897,7 @@ static long ll_file_set_lease(struct file *file, struct ll_ioc_lease *ioc, goto out; } - intent = lsl.sl_flags & INTENT_LAYOUTS_CLOSE; - if (intent) { + if (lsl.sl_flags & SWAP_LAYOUTS_CLOSE) { struct obd_client_handle *och = NULL; struct ll_inode_info *lli; struct inode *inode2; @@ -2884,7 +2914,7 @@ static long ll_file_set_lease(struct file *file, struct ll_ioc_lease *ioc, goto out; } inode2 = file_inode(file2); - rc = ll_swap_layouts_close(och, inode, inode2, intent); + rc = ll_swap_layouts_close(och, inode, inode2); } else { rc = ll_swap_layouts(file, file2, &lsl); } diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index d53e795..36f78a6 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -253,6 +253,7 @@ enum ll_lease_mode { enum ll_lease_flags { LL_LEASE_RESYNC = 0x1, LL_LEASE_RESYNC_DONE = 0x2, + LL_LEASE_LAYOUT_MERGE = 0x4, }; #define IOC_IDS_MAX 4096 @@ -824,8 +825,6 @@ struct if_quotactl { #define SWAP_LAYOUTS_KEEP_MTIME (1 << 2) #define SWAP_LAYOUTS_KEEP_ATIME (1 << 3) #define SWAP_LAYOUTS_CLOSE (1 << 4) -#define MERGE_LAYOUTS_CLOSE (1 << 5) -#define INTENT_LAYOUTS_CLOSE (SWAP_LAYOUTS_CLOSE | MERGE_LAYOUTS_CLOSE) /* Swap XATTR_NAME_HSM as well, only on the MDT so far */ #define SWAP_LAYOUTS_MDS_HSM (1 << 31) From patchwork Mon Sep 30 18:56:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167267 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A2DFD16B1 for ; Mon, 30 Sep 2019 19:05:45 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8B50C224F0 for ; Mon, 30 Sep 2019 19:05:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8B50C224F0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id D6FC65E4AF3; Mon, 30 Sep 2019 12:00:41 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id DA34C5C3C6D for ; Mon, 30 Sep 2019 11:57:33 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 890B31005EFB; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 88006A9; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:09 -0400 Message-Id: <1569869810-23848-111-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 110/151] lustre: idl: add PTLRPC definitions to enum X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Andreas Dilger PTLRPC definitions to enums for cleanliness, and use them: enum lustre_msg_version for LUSTRE_*_VERSION enum lustre_msg_magic for LUSTRE_MSG_MAGIC_* enum lustre_msghdr for MSGHDR_* WC-bug-id: https://jira.whamcloud.com/browse/LU-6349 Lustre-commit: 0c9bb26fc4d9 ("LU-6349 idl: add PTLRPC definitions to enum") Signed-off-by: Andreas Dilger Reviewed-on: https://review.whamcloud.com/30958 Reviewed-by: Bob Glossman Reviewed-by: Emoly Liu Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_import.h | 7 ++++-- fs/lustre/ptlrpc/pack_generic.c | 4 ++-- include/uapi/linux/lustre/lustre_idl.h | 44 +++++++++++++++++++--------------- 3 files changed, 32 insertions(+), 23 deletions(-) diff --git a/fs/lustre/include/lustre_import.h b/fs/lustre/include/lustre_import.h index 2d8f6be..ad06f30 100644 --- a/fs/lustre/include/lustre_import.h +++ b/fs/lustre/include/lustre_import.h @@ -308,8 +308,11 @@ struct obd_import { u64 imp_connect_flags2_orig; int imp_connect_error; - u32 imp_msg_magic; - u32 imp_msghdr_flags; /* adjusted based on server capability */ + enum lustre_msg_magic imp_msg_magic; + /* adjusted based on server + * capability + */ + enum lustre_msghdr imp_msghdr_flags; struct imp_at imp_at; /* adaptive timeout data */ time64_t imp_last_reply_time; /* for health check */ diff --git a/fs/lustre/ptlrpc/pack_generic.c b/fs/lustre/ptlrpc/pack_generic.c index 1ba2772..cac4b8d 100644 --- a/fs/lustre/ptlrpc/pack_generic.c +++ b/fs/lustre/ptlrpc/pack_generic.c @@ -749,7 +749,7 @@ static inline struct ptlrpc_body *lustre_msg_ptlrpc_body(struct lustre_msg *msg) sizeof(struct ptlrpc_body_v2)); } -u32 lustre_msghdr_get_flags(struct lustre_msg *msg) +enum lustre_msghdr lustre_msghdr_get_flags(struct lustre_msg *msg) { switch (msg->lm_magic) { case LUSTRE_MSG_MAGIC_V2: @@ -762,7 +762,7 @@ u32 lustre_msghdr_get_flags(struct lustre_msg *msg) } EXPORT_SYMBOL(lustre_msghdr_get_flags); -void lustre_msghdr_set_flags(struct lustre_msg *msg, u32 flags) +void lustre_msghdr_set_flags(struct lustre_msg *msg, enum lustre_msghdr flags) { switch (msg->lm_magic) { case LUSTRE_MSG_MAGIC_V2: diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index cd4e0e9..cf35813 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -530,18 +530,22 @@ static inline void lustre_handle_copy(struct lustre_handle *tgt, } /* lustre_msg struct magic. DON'T use swabbed values of MAGIC as magic! */ -#define LUSTRE_MSG_MAGIC_V2 0x0BD00BD3 -#define LUSTRE_MSG_MAGIC_V2_SWABBED 0xD30BD00B -#define LUSTRE_MSG_MAGIC LUSTRE_MSG_MAGIC_V2 +enum lustre_msg_magic { + LUSTRE_MSG_MAGIC_V2 = 0x0BD00BD3, + LUSTRE_MSG_MAGIC_V2_SWABBED = 0xD30BD00B, + LUSTRE_MSG_MAGIC = LUSTRE_MSG_MAGIC_V2, +}; /* flags for lm_flags */ -#define MSGHDR_AT_SUPPORT 0x1 /* adaptive timeouts, lm_cksum valid - * in early reply messages - */ -#define MSGHDR_CKSUM_INCOMPAT18 0x2 /* compat for 1.8, needs to be set - * well beyond 2.8.0 for - * compatibility - */ +enum lustre_msghdr { + MSGHDR_AT_SUPPORT = 0x1,/* adaptive timeouts, lm_cksum valid + * in early reply messages + */ + MSGHDR_CKSUM_INCOMPAT18 = 0x2,/* compat for 1.8, needs to be set + * well beyond 2.8.0 for + * compatibility + */ +}; #define lustre_msg lustre_msg_v2 /* we depend on this structure to be 8-byte aligned */ @@ -552,7 +556,7 @@ struct lustre_msg_v2 { __u32 lm_magic; /* RPC version magic = LUSTRE_MSG_MAGIC_V2 */ __u32 lm_repsize; /* size of preallocated reply buffer */ __u32 lm_cksum; /* CRC32 of ptlrpc_body early reply messages */ - __u32 lm_flags; /* MSGHDR_* flags */ + __u32 lm_flags; /* enum lustre_msghdr MSGHDR_* flags */ __u32 lm_padding_2; /* unused */ __u32 lm_padding_3; /* unused */ __u32 lm_buflens[0]; /* length of additional buffers in bytes, @@ -570,14 +574,16 @@ struct lustre_msg_v2 { #define PTL_RPC_MSG_REPLY 4713 /* normal RPC reply message */ /* ptlrpc_body pb_version ((target_version << 16) | rpc_version) */ -#define PTLRPC_MSG_VERSION 0x00000003 -#define LUSTRE_VERSION_MASK 0xffff0000 -#define LUSTRE_OBD_VERSION 0x00010000 -#define LUSTRE_MDS_VERSION 0x00020000 -#define LUSTRE_OST_VERSION 0x00030000 -#define LUSTRE_DLM_VERSION 0x00040000 -#define LUSTRE_LOG_VERSION 0x00050000 -#define LUSTRE_MGS_VERSION 0x00060000 +enum lustre_msg_version { + PTLRPC_MSG_VERSION = 0x00000003, + LUSTRE_VERSION_MASK = 0xffff0000, + LUSTRE_OBD_VERSION = 0x00010000, + LUSTRE_MDS_VERSION = 0x00020000, + LUSTRE_OST_VERSION = 0x00030000, + LUSTRE_DLM_VERSION = 0x00040000, + LUSTRE_LOG_VERSION = 0x00050000, + LUSTRE_MGS_VERSION = 0x00060000, +}; /* pb_flags that apply to all request messages */ /* #define MSG_LAST_REPLAY 0x0001 obsolete 2.0 => {REQ,LOCK}_REPLAY_DONE */ From patchwork Mon Sep 30 18:56:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167213 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DEE1816C1 for ; Mon, 30 Sep 2019 19:03:46 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C71CD224EF for ; Mon, 30 Sep 2019 19:03:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C71CD224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 6585A21F7EB; Mon, 30 Sep 2019 11:59:47 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 44F055C36DD for ; Mon, 30 Sep 2019 11:57:34 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 8DB1C1005EFC; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 8B1DDB4; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:10 -0400 Message-Id: <1569869810-23848-112-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 111/151] lustre: obd: remove s2dhms time function X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: James Simmons , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" Lustre created the s2dhms function to report a time lapse in a days:hour:minute:seconds format. This is over kill so remove s2dhms and just report the number of seconds that has lapse. WC-bug-id: https://jira.whamcloud.com/browse/LU-9019 Lustre-commit: 5ed38cca120f ("LU-9019 obd: remove s2dhms time function") Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/23269 Reviewed-by: John L. Hammond Reviewed-by: Andreas Dilger Reviewed-by: Dmitry Eremin Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lprocfs_status.h | 19 ------------------- fs/lustre/obdclass/lprocfs_status.c | 19 ++++++++----------- fs/lustre/ptlrpc/lproc_ptlrpc.c | 8 +++----- 3 files changed, 11 insertions(+), 35 deletions(-) diff --git a/fs/lustre/include/lprocfs_status.h b/fs/lustre/include/lprocfs_status.h index 10d541f..c3644f9 100644 --- a/fs/lustre/include/lprocfs_status.h +++ b/fs/lustre/include/lprocfs_status.h @@ -348,25 +348,6 @@ enum { struct obd_device; struct obd_histogram; -/* Days / hours / mins / seconds format */ -struct dhms { - int d, h, m, s; -}; - -static inline void s2dhms(struct dhms *ts, time64_t secs64) -{ - unsigned int secs; - - ts->d = div_u64_rem(secs64, 86400, &secs); - ts->h = secs / 3600; - secs = secs % 3600; - ts->m = secs / 60; - ts->s = secs % 60; -} - -#define DHMS_FMT "%dd%dh%02dm%02ds" -#define DHMS_VARS(x) (x)->d, (x)->h, (x)->m, (x)->s - #define JOBSTATS_JOBID_VAR_MAX_LEN 20 #define JOBSTATS_DISABLE "disable" #define JOBSTATS_PROCNAME_UID "procname_uid" diff --git a/fs/lustre/obdclass/lprocfs_status.c b/fs/lustre/obdclass/lprocfs_status.c index 135926e..0e7fc70 100644 --- a/fs/lustre/obdclass/lprocfs_status.c +++ b/fs/lustre/obdclass/lprocfs_status.c @@ -943,7 +943,6 @@ int lprocfs_rd_timeouts(struct seq_file *m, void *data) struct obd_import *imp; unsigned int cur, worst; time64_t now, worstt; - struct dhms ts; int i, rc; LASSERT(obd); @@ -956,16 +955,15 @@ int lprocfs_rd_timeouts(struct seq_file *m, void *data) now = ktime_get_real_seconds(); /* Some network health info for kicks */ - s2dhms(&ts, now - imp->imp_last_reply_time); - seq_printf(m, "%-10s : %lld, " DHMS_FMT " ago\n", - "last reply", (s64)imp->imp_last_reply_time, DHMS_VARS(&ts)); + seq_printf(m, "%-10s : %lld, %llds ago\n", + "last reply", (s64)imp->imp_last_reply_time, + (s64)(now - imp->imp_last_reply_time)); cur = at_get(&imp->imp_at.iat_net_latency); worst = imp->imp_at.iat_net_latency.at_worst_ever; worstt = imp->imp_at.iat_net_latency.at_worst_time; - s2dhms(&ts, now - worstt); - seq_printf(m, "%-10s : cur %3u worst %3u (at %lld, " DHMS_FMT " ago) ", - "network", cur, worst, (s64)worstt, DHMS_VARS(&ts)); + seq_printf(m, "%-10s : cur %3u worst %3u (at %lld, %llds ago) ", + "network", cur, worst, (s64)worstt, (s64)(now - worstt)); lprocfs_at_hist_helper(m, &imp->imp_at.iat_net_latency); for (i = 0; i < IMP_AT_MAX_PORTALS; i++) { @@ -974,10 +972,9 @@ int lprocfs_rd_timeouts(struct seq_file *m, void *data) cur = at_get(&imp->imp_at.iat_service_estimate[i]); worst = imp->imp_at.iat_service_estimate[i].at_worst_ever; worstt = imp->imp_at.iat_service_estimate[i].at_worst_time; - s2dhms(&ts, now - worstt); - seq_printf(m, "portal %-2d : cur %3u worst %3u (at %lld, " - DHMS_FMT " ago) ", imp->imp_at.iat_portal[i], - cur, worst, (s64)worstt, DHMS_VARS(&ts)); + seq_printf(m, "portal %-2d : cur %3u worst %3u (at %lld, %llds ago) ", + imp->imp_at.iat_portal[i], cur, worst, (s64)worstt, + (s64)(now - worstt)); lprocfs_at_hist_helper(m, &imp->imp_at.iat_service_estimate[i]); } diff --git a/fs/lustre/ptlrpc/lproc_ptlrpc.c b/fs/lustre/ptlrpc/lproc_ptlrpc.c index 2239680..3dc99d4 100644 --- a/fs/lustre/ptlrpc/lproc_ptlrpc.c +++ b/fs/lustre/ptlrpc/lproc_ptlrpc.c @@ -1009,7 +1009,6 @@ static int ptlrpc_lprocfs_timeouts_seq_show(struct seq_file *m, void *n) { struct ptlrpc_service *svc = m->private; struct ptlrpc_service_part *svcpt; - struct dhms ts; time64_t worstt; unsigned int cur; unsigned int worst; @@ -1025,11 +1024,10 @@ static int ptlrpc_lprocfs_timeouts_seq_show(struct seq_file *m, void *n) cur = at_get(&svcpt->scp_at_estimate); worst = svcpt->scp_at_estimate.at_worst_ever; worstt = svcpt->scp_at_estimate.at_worst_time; - s2dhms(&ts, ktime_get_real_seconds() - worstt); - seq_printf(m, "%10s : cur %3u worst %3u (at %lld, " - DHMS_FMT " ago) ", "service", - cur, worst, (s64)worstt, DHMS_VARS(&ts)); + seq_printf(m, "%10s : cur %3u worst %3u (at %lld, %llds ago) ", + "service", cur, worst, (s64)worstt, + (s64)(ktime_get_real_seconds() - worstt)); lprocfs_at_hist_helper(m, &svcpt->scp_at_estimate); } From patchwork Mon Sep 30 18:56:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167305 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6360213B1 for ; Mon, 30 Sep 2019 19:07:19 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4B7D0224EF for ; Mon, 30 Sep 2019 19:07:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4B7D0224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 3D04C5E4E58; Mon, 30 Sep 2019 12:01:16 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 9F9FC5C3C6D for ; Mon, 30 Sep 2019 11:57:34 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 900671005F04; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 8E021B5; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:11 -0400 Message-Id: <1569869810-23848-113-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 112/151] lustre: mdc: add client NID to Changelogs entries X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Sebastien Buisson Add a new changelog extension named changelog_ext_nid to hold client's NID information. NID info is added to every Changelog entry type except MARK, in the form 'nid=': 1 01CREAT 15:50:20.834838318 2017.07.24 0x0 t=[0x200000401:0x2:0x0] ef=0x3 u=500:500 nid=10.128.11.159@tcp p=[0x200000007:0x1:0x0] fileA WC-bug-id: https://jira.whamcloud.com/browse/LU-9727 Lustre-commit: d2629cfcabfa ("LU-9727 mdc: add client NID to Changelogs entries") Signed-off-by: Sebastien Buisson Reviewed-on: https://review.whamcloud.com/28213 Reviewed-by: Andreas Dilger Reviewed-by: Quentin Bouget Reviewed-by: Li Xi Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/mdc/mdc_changelog.c | 3 ++- include/uapi/linux/lustre/lustre_idl.h | 3 ++- include/uapi/linux/lustre/lustre_user.h | 47 +++++++++++++++++++++++++++++++-- 3 files changed, 49 insertions(+), 4 deletions(-) diff --git a/fs/lustre/mdc/mdc_changelog.c b/fs/lustre/mdc/mdc_changelog.c index d9e861e..118e605 100644 --- a/fs/lustre/mdc/mdc_changelog.c +++ b/fs/lustre/mdc/mdc_changelog.c @@ -237,7 +237,8 @@ static int chlg_load(void *args) LLOG_F_IS_CAT | LLOG_F_EXT_JOBID | LLOG_F_EXT_EXTRA_FLAGS | - LLOG_F_EXT_X_UIDGID, + LLOG_F_EXT_X_UIDGID | + LLOG_F_EXT_X_NID, NULL); if (rc) { CERROR("%s: fail to init llog handle: rc = %d\n", diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index cf35813..c2fa26f 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -2584,6 +2584,7 @@ enum llog_flag { LLOG_F_IS_FIXSIZE = 0x10, LLOG_F_EXT_EXTRA_FLAGS = 0x20, LLOG_F_EXT_X_UIDGID = 0x40, + LLOG_F_EXT_X_NID = 0x80, /* * Note: Flags covered by LLOG_F_EXT_MASK will be inherited from @@ -2592,7 +2593,7 @@ enum llog_flag { * log record can be variable */ LLOG_F_EXT_MASK = LLOG_F_EXT_JOBID | LLOG_F_EXT_EXTRA_FLAGS | - LLOG_F_EXT_X_UIDGID, + LLOG_F_EXT_X_UIDGID | LLOG_F_EXT_X_NID, }; /* On-disk header structure of each log object, stored in little endian order */ diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index 36f78a6..771a2cf 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -1005,7 +1005,8 @@ static inline void hsm_set_cl_error(int *flags, int error) enum changelog_rec_extra_flags { CLFE_INVALID = 0, CLFE_UIDGID = 0x0001, - CLFE_SUPPORTED = CLFE_UIDGID + CLFE_NID = 0x0002, + CLFE_SUPPORTED = CLFE_UIDGID | CLFE_NID }; enum changelog_send_flag { @@ -1026,6 +1027,8 @@ enum changelog_send_flag { enum changelog_send_extra_flag { /* Pack uid/gid into the changelog record */ CHANGELOG_EXTRA_FLAG_UIDGID = 0x01, + /* Pack nid into the changelog record */ + CHANGELOG_EXTRA_FLAG_NID = 0x02, }; #define CR_MAXSIZE __ALIGN_KERNEL(2 * NAME_MAX + 2 + \ @@ -1068,6 +1071,15 @@ struct changelog_ext_jobid { char cr_jobid[LUSTRE_JOBID_SIZE]; /**< zero-terminated string. */ }; +/* Changelog extra extension to include NID. */ +struct changelog_ext_nid { + /* have __u64 instead of lnet_nid_t type for use by client api */ + __u64 cr_nid; + /* for use when IPv6 support is added */ + __u64 extra; + __u32 padding; +}; + /* Changelog extension to include additional flags. */ struct changelog_ext_extra_flags { __u64 cr_extra_flags; /* Additional CLFE_* flags */ @@ -1097,6 +1109,8 @@ static inline size_t changelog_rec_offset(enum changelog_rec_flags crf, size += sizeof(struct changelog_ext_extra_flags); if (cref & CLFE_UIDGID) size += sizeof(struct changelog_ext_uidgid); + if (cref & CLFE_NID) + size += sizeof(struct changelog_ext_nid); } return size; @@ -1165,6 +1179,22 @@ struct changelog_ext_uidgid *changelog_rec_uidgid( CLFE_INVALID)); } +/* The nid is the second extra extension */ +static inline +struct changelog_ext_nid *changelog_rec_nid(const struct changelog_rec *rec) +{ + enum changelog_rec_flags crf = rec->cr_flags & + (CLF_VERSION | CLF_RENAME | CLF_JOBID | CLF_EXTRA_FLAGS); + enum changelog_rec_extra_flags cref = CLFE_INVALID; + + if (rec->cr_flags & CLF_EXTRA_FLAGS) + cref = changelog_rec_extra_flags(rec)->cr_extra_flags & + CLFE_UIDGID; + + return (struct changelog_ext_nid *)((char *)rec + + changelog_rec_offset(crf, cref)); +} + /* The name follows the rename, jobid and extra flags extns, if present */ static inline char *changelog_rec_name(struct changelog_rec *rec) { @@ -1214,6 +1244,7 @@ static inline void changelog_remap_rec(struct changelog_rec *rec, enum changelog_rec_flags crf_wanted, enum changelog_rec_extra_flags cref_want) { + char *nid_mov = NULL; char *uidgid_mov = NULL; char *ef_mov; char *jid_mov, *rnm_mov; @@ -1237,9 +1268,13 @@ static inline void changelog_remap_rec(struct changelog_rec *rec, /* Locations of extensions in the remapped record */ if (rec->cr_flags & CLF_EXTRA_FLAGS) { + nid_mov = (char *)rec + + changelog_rec_offset(crf_wanted & CLF_SUPPORTED, + cref_want & ~CLFE_NID); uidgid_mov = (char *)rec + changelog_rec_offset(crf_wanted & CLF_SUPPORTED, - CLFE_INVALID); + cref_want & ~(CLFE_UIDGID | + CLFE_NID)); cref = changelog_rec_extra_flags(rec)->cr_extra_flags; } @@ -1260,6 +1295,10 @@ static inline void changelog_remap_rec(struct changelog_rec *rec, /* Move the extension fields to the desired positions */ if ((crf_wanted & CLF_EXTRA_FLAGS) && (rec->cr_flags & CLF_EXTRA_FLAGS)) { + if ((cref_want & CLFE_NID) && (cref & CLFE_NID)) + memmove(nid_mov, changelog_rec_nid(rec), + sizeof(struct changelog_ext_nid)); + if ((cref_want & CLFE_UIDGID) && (cref & CLFE_UIDGID)) memmove(uidgid_mov, changelog_rec_uidgid(rec), sizeof(struct changelog_ext_uidgid)); @@ -1277,6 +1316,10 @@ static inline void changelog_remap_rec(struct changelog_rec *rec, sizeof(struct changelog_ext_rename)); /* Clear newly added fields */ + if (nid_mov && (cref_want & CLFE_NID) && + !(cref & CLFE_NID)) + memset(nid_mov, 0, sizeof(struct changelog_ext_nid)); + if (uidgid_mov && (cref_want & CLFE_UIDGID) && !(cref & CLFE_UIDGID)) memset(uidgid_mov, 0, sizeof(struct changelog_ext_uidgid)); From patchwork Mon Sep 30 18:56:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167309 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A46541599 for ; Mon, 30 Sep 2019 19:07:28 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8C3E0224EF for ; Mon, 30 Sep 2019 19:07:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8C3E0224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id B96235E4EA0; Mon, 30 Sep 2019 12:01:19 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 07C7F5C3B3B for ; Mon, 30 Sep 2019 11:57:35 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 92C061005F05; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 9147ABB; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:12 -0400 Message-Id: <1569869810-23848-114-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 113/151] lustre: mdc: implement CL_OPEN for Changelogs X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Sebastien Buisson Record OPEN events in Changelogs, and add a new changelog extension named changelog_ext_openmode to hold open mode. An OPEN changlog entry is in the form: 7 10OPEN 13:38:51.510728296 2017.07.25 0x242 t=[0x200000401:0x2:0x0] ef=0x7 u=500:500 nid=10.128.11.159@tcp m=-w- By default, disable recording of OPEN events in Changelogs. Note that CREAT are still recorded even if OPEN are disabled. WC-bug-id: https://jira.whamcloud.com/browse/LU-9727 Lustre-commit: 21fb4d93a94e ("LU-9727 mdc: implement CL_OPEN for Changelogs") Signed-off-by: Sebastien Buisson Reviewed-on: https://review.whamcloud.com/28214 Reviewed-by: Andreas Dilger Reviewed-by: Li Xi Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/mdc/mdc_changelog.c | 3 +- include/uapi/linux/lustre/lustre_idl.h | 52 ++---------------- include/uapi/linux/lustre/lustre_user.h | 95 +++++++++++++++++++++++++++++++-- 3 files changed, 97 insertions(+), 53 deletions(-) diff --git a/fs/lustre/mdc/mdc_changelog.c b/fs/lustre/mdc/mdc_changelog.c index 118e605..2aa4a8c 100644 --- a/fs/lustre/mdc/mdc_changelog.c +++ b/fs/lustre/mdc/mdc_changelog.c @@ -238,7 +238,8 @@ static int chlg_load(void *args) LLOG_F_EXT_JOBID | LLOG_F_EXT_EXTRA_FLAGS | LLOG_F_EXT_X_UIDGID | - LLOG_F_EXT_X_NID, + LLOG_F_EXT_X_NID | + LLOG_F_EXT_X_OMODE, NULL); if (rc) { CERROR("%s: fail to init llog handle: rc = %d\n", diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index c2fa26f..4b33104 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -1636,54 +1636,6 @@ struct mdt_rec_setattr { #define MDS_ATTR_BLOCKS 0x8000ULL /* = 32768 */ #define MDS_ATTR_PROJID 0x10000ULL /* = 65536 */ -#define MDS_FMODE_CLOSED 00000000 -#define MDS_FMODE_EXEC 00000004 -/* MDS_FMODE_EPOCH 01000000 obsolete since 2.8.0 */ -/* MDS_FMODE_TRUNC 02000000 obsolete since 2.8.0 */ -/* MDS_FMODE_SOM 04000000 obsolete since 2.8.0 */ - -#define MDS_OPEN_CREATED 00000010 -#define MDS_OPEN_CROSS 00000020 - -#define MDS_OPEN_CREAT 00000100 -#define MDS_OPEN_EXCL 00000200 -#define MDS_OPEN_TRUNC 00001000 -#define MDS_OPEN_APPEND 00002000 -#define MDS_OPEN_SYNC 00010000 -#define MDS_OPEN_DIRECTORY 00200000 - -#define MDS_OPEN_BY_FID 040000000 /* open_by_fid for known object */ -#define MDS_OPEN_DELAY_CREATE 0100000000 /* delay initial object create */ -#define MDS_OPEN_OWNEROVERRIDE 0200000000 /* NFSD rw-reopen ro file for owner */ -#define MDS_OPEN_JOIN_FILE 0400000000 /* open for join file. - * We do not support JOIN FILE - * anymore, reserve this flags - * just for preventing such bit - * to be reused. - */ - -#define MDS_OPEN_LOCK 04000000000 /* This open requires open lock */ -#define MDS_OPEN_HAS_EA 010000000000 /* specify object create pattern */ -#define MDS_OPEN_HAS_OBJS 020000000000 /* Just set the EA the obj exist */ -#define MDS_OPEN_NORESTORE 0100000000000ULL /* Do not restore file at open */ -#define MDS_OPEN_NEWSTRIPE 0200000000000ULL /* New stripe needed (restripe or - * hsm restore) - */ -#define MDS_OPEN_VOLATILE 0400000000000ULL /* File is volatile = created - * unlinked - */ -#define MDS_OPEN_LEASE 01000000000000ULL /* Open the file and grant lease - * delegation, succeed if it's not - * being opened with conflict mode. - */ -#define MDS_OPEN_RELEASE 02000000000000ULL /* Open the file for HSM release */ -#define MDS_OPEN_RESYNC 04000000000000ULL /* FLR: file resync */ - -#define MDS_OPEN_FL_INTERNAL (MDS_OPEN_HAS_EA | MDS_OPEN_HAS_OBJS | \ - MDS_OPEN_OWNEROVERRIDE | MDS_OPEN_LOCK | \ - MDS_OPEN_BY_FID | MDS_OPEN_LEASE | \ - MDS_OPEN_RELEASE | MDS_OPEN_RESYNC) - enum mds_op_bias { MDS_CHECK_SPLIT = 1 << 0, MDS_CROSS_REF = 1 << 1, @@ -2585,6 +2537,7 @@ enum llog_flag { LLOG_F_EXT_EXTRA_FLAGS = 0x20, LLOG_F_EXT_X_UIDGID = 0x40, LLOG_F_EXT_X_NID = 0x80, + LLOG_F_EXT_X_OMODE = 0x100, /* * Note: Flags covered by LLOG_F_EXT_MASK will be inherited from @@ -2593,7 +2546,8 @@ enum llog_flag { * log record can be variable */ LLOG_F_EXT_MASK = LLOG_F_EXT_JOBID | LLOG_F_EXT_EXTRA_FLAGS | - LLOG_F_EXT_X_UIDGID | LLOG_F_EXT_X_NID, + LLOG_F_EXT_X_UIDGID | LLOG_F_EXT_X_NID | + LLOG_F_EXT_X_OMODE, }; /* On-disk header structure of each log object, stored in little endian order */ diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index 771a2cf..f3fa24b 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -865,6 +865,54 @@ enum la_valid { LA_REMOTE_ATTR_SET = (LA_UID | LA_GID | LA_PROJID | LA_LAYOUT_VERSION) }; +#define MDS_FMODE_CLOSED 00000000 +#define MDS_FMODE_EXEC 00000004 +/* MDS_FMODE_EPOCH 01000000 obsolete since 2.8.0 */ +/* MDS_FMODE_TRUNC 02000000 obsolete since 2.8.0 */ +/* MDS_FMODE_SOM 04000000 obsolete since 2.8.0 */ + +#define MDS_OPEN_CREATED 00000010 +#define MDS_OPEN_CROSS 00000020 + +#define MDS_OPEN_CREAT 00000100 +#define MDS_OPEN_EXCL 00000200 +#define MDS_OPEN_TRUNC 00001000 +#define MDS_OPEN_APPEND 00002000 +#define MDS_OPEN_SYNC 00010000 +#define MDS_OPEN_DIRECTORY 00200000 + +#define MDS_OPEN_BY_FID 040000000 /* open_by_fid for known object */ +#define MDS_OPEN_DELAY_CREATE 0100000000 /* delay initial object create */ +#define MDS_OPEN_OWNEROVERRIDE 0200000000 /* NFSD rw-reopen ro file for owner */ +#define MDS_OPEN_JOIN_FILE 0400000000 /* open for join file. + * We do not support JOIN FILE + * anymore, reserve this flags + * just for preventing such bit + * to be reused. + */ + +#define MDS_OPEN_LOCK 04000000000 /* This open requires open lock */ +#define MDS_OPEN_HAS_EA 010000000000 /* specify object create pattern */ +#define MDS_OPEN_HAS_OBJS 020000000000 /* Just set the EA the obj exist */ +#define MDS_OPEN_NORESTORE 0100000000000ULL /* Do not restore file at open */ +#define MDS_OPEN_NEWSTRIPE 0200000000000ULL /* New stripe needed (restripe or + * hsm restore) + */ +#define MDS_OPEN_VOLATILE 0400000000000ULL /* File is volatile = created + * unlinked + */ +#define MDS_OPEN_LEASE 01000000000000ULL /* Open the file and grant lease + * delegation, succeed if it's not + * being opened with conflict mode. + */ +#define MDS_OPEN_RELEASE 02000000000000ULL /* Open the file for HSM release */ +#define MDS_OPEN_RESYNC 04000000000000ULL /* FLR: file resync */ + +#define MDS_OPEN_FL_INTERNAL (MDS_OPEN_HAS_EA | MDS_OPEN_HAS_OBJS | \ + MDS_OPEN_OWNEROVERRIDE | MDS_OPEN_LOCK | \ + MDS_OPEN_BY_FID | MDS_OPEN_LEASE | \ + MDS_OPEN_RELEASE | MDS_OPEN_RESYNC) + /********* Changelogs **********/ /** Changelog record types */ enum changelog_rec_type { @@ -1006,7 +1054,8 @@ enum changelog_rec_extra_flags { CLFE_INVALID = 0, CLFE_UIDGID = 0x0001, CLFE_NID = 0x0002, - CLFE_SUPPORTED = CLFE_UIDGID | CLFE_NID + CLFE_OPEN = 0x0004, + CLFE_SUPPORTED = CLFE_UIDGID | CLFE_NID | CLFE_OPEN }; enum changelog_send_flag { @@ -1029,6 +1078,8 @@ enum changelog_send_extra_flag { CHANGELOG_EXTRA_FLAG_UIDGID = 0x01, /* Pack nid into the changelog record */ CHANGELOG_EXTRA_FLAG_NID = 0x02, + /* Pack open mode into the changelog record */ + CHANGELOG_EXTRA_FLAG_OMODE = 0x04, }; #define CR_MAXSIZE __ALIGN_KERNEL(2 * NAME_MAX + 2 + \ @@ -1080,6 +1131,11 @@ struct changelog_ext_nid { __u32 padding; }; +/* Changelog extra extension to include OPEN mode. */ +struct changelog_ext_openmode { + __u32 cr_openflags; +}; + /* Changelog extension to include additional flags. */ struct changelog_ext_extra_flags { __u64 cr_extra_flags; /* Additional CLFE_* flags */ @@ -1111,6 +1167,8 @@ static inline size_t changelog_rec_offset(enum changelog_rec_flags crf, size += sizeof(struct changelog_ext_uidgid); if (cref & CLFE_NID) size += sizeof(struct changelog_ext_nid); + if (cref & CLFE_OPEN) + size += sizeof(struct changelog_ext_openmode); } return size; @@ -1195,6 +1253,23 @@ struct changelog_ext_nid *changelog_rec_nid(const struct changelog_rec *rec) changelog_rec_offset(crf, cref)); } +/* The OPEN mode is the third extra extension */ +static inline +struct changelog_ext_openmode *changelog_rec_openmode( + const struct changelog_rec *rec) +{ + enum changelog_rec_flags crf = rec->cr_flags & + (CLF_VERSION | CLF_RENAME | CLF_JOBID | CLF_EXTRA_FLAGS); + enum changelog_rec_extra_flags cref = CLFE_INVALID; + + if (rec->cr_flags & CLF_EXTRA_FLAGS) + cref = changelog_rec_extra_flags(rec)->cr_extra_flags & + (CLFE_UIDGID | CLFE_NID); + + return (struct changelog_ext_openmode *)((char *)rec + + changelog_rec_offset(crf, cref)); +} + /* The name follows the rename, jobid and extra flags extns, if present */ static inline char *changelog_rec_name(struct changelog_rec *rec) { @@ -1244,6 +1319,7 @@ static inline void changelog_remap_rec(struct changelog_rec *rec, enum changelog_rec_flags crf_wanted, enum changelog_rec_extra_flags cref_want) { + char *omd_mov = NULL; char *nid_mov = NULL; char *uidgid_mov = NULL; char *ef_mov; @@ -1268,13 +1344,18 @@ static inline void changelog_remap_rec(struct changelog_rec *rec, /* Locations of extensions in the remapped record */ if (rec->cr_flags & CLF_EXTRA_FLAGS) { + omd_mov = (char *)rec + + changelog_rec_offset(crf_wanted & CLF_SUPPORTED, + cref_want & ~CLFE_OPEN); nid_mov = (char *)rec + changelog_rec_offset(crf_wanted & CLF_SUPPORTED, - cref_want & ~CLFE_NID); + cref_want & ~(CLFE_NID | + CLFE_OPEN)); uidgid_mov = (char *)rec + changelog_rec_offset(crf_wanted & CLF_SUPPORTED, cref_want & ~(CLFE_UIDGID | - CLFE_NID)); + CLFE_NID | + CLFE_OPEN)); cref = changelog_rec_extra_flags(rec)->cr_extra_flags; } @@ -1295,6 +1376,10 @@ static inline void changelog_remap_rec(struct changelog_rec *rec, /* Move the extension fields to the desired positions */ if ((crf_wanted & CLF_EXTRA_FLAGS) && (rec->cr_flags & CLF_EXTRA_FLAGS)) { + if ((cref_want & CLFE_OPEN) && (cref & CLFE_OPEN)) + memmove(omd_mov, changelog_rec_openmode(rec), + sizeof(struct changelog_ext_openmode)); + if ((cref_want & CLFE_NID) && (cref & CLFE_NID)) memmove(nid_mov, changelog_rec_nid(rec), sizeof(struct changelog_ext_nid)); @@ -1316,6 +1401,10 @@ static inline void changelog_remap_rec(struct changelog_rec *rec, sizeof(struct changelog_ext_rename)); /* Clear newly added fields */ + if (omd_mov && (cref_want & CLFE_OPEN) && + !(cref & CLFE_OPEN)) + memset(omd_mov, 0, sizeof(struct changelog_ext_openmode)); + if (nid_mov && (cref_want & CLFE_NID) && !(cref & CLFE_NID)) memset(nid_mov, 0, sizeof(struct changelog_ext_nid)); From patchwork Mon Sep 30 18:56:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167269 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 63F6A13B1 for ; Mon, 30 Sep 2019 19:05:53 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4C2E32168B for ; Mon, 30 Sep 2019 19:05:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4C2E32168B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 0DF085E4B36; Mon, 30 Sep 2019 12:00:45 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 5D8975C3B3B for ; Mon, 30 Sep 2019 11:57:35 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 968681005F06; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 94846A9; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:13 -0400 Message-Id: <1569869810-23848-115-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 114/151] lustre: acl: prepare small buffer for ACL RPC reply X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fan Yong , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Fan Yong For most of files, their ACL entries are very limited, under such case, it is unnecessary to prepare very large reply buffer to hold unknown-sized ACL entries for the getattr/open RPCs. Instead, we can prepare some relative small buffer, such as the LUSTRE_POSIX_ACL_MAX_SIZE_OLD (260) bytes, that is equal to the ACL size before patch 64b2fad22a4eb4727315709e014d8f74c5a7f289. If the target file has too many ACL entries and exceeds the prepared reply buffer, then the MDT will reply -ERANGE failure to the client, and then the client can prepare more large buffer and try again. Since the file with large ACL is rare case, such retrying getattr/open RPCs will not affect the real performance too much. The advantage is that it reduces the client side RAM pressure. WC-bug-id: https://jira.whamcloud.com/browse/LU-10513 Lustre-commit: 416e88938f68 ("LU-10513 acl: prepare small buffer for ACL RPC reply") Signed-off-by: Fan Yong Reviewed-on: https://review.whamcloud.com/28116 Reviewed-by: Andreas Dilger Reviewed-by: Lai Siyao Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/mdc/mdc_locks.c | 43 +++++++++++++++++++++-------- fs/lustre/mdc/mdc_request.c | 61 +++++++++++++++++++++++++++++++---------- fs/lustre/ptlrpc/pack_generic.c | 1 + fs/lustre/ptlrpc/sec.c | 1 + 4 files changed, 79 insertions(+), 27 deletions(-) diff --git a/fs/lustre/mdc/mdc_locks.c b/fs/lustre/mdc/mdc_locks.c index 2eb6e8a..0787ba3 100644 --- a/fs/lustre/mdc/mdc_locks.c +++ b/fs/lustre/mdc/mdc_locks.c @@ -44,6 +44,7 @@ #include #include #include +#include #include "mdc_internal.h" @@ -244,7 +245,7 @@ static int mdc_save_lovea(struct ptlrpc_request *req, static struct ptlrpc_request * mdc_intent_open_pack(struct obd_export *exp, struct lookup_intent *it, - struct md_op_data *op_data) + struct md_op_data *op_data, u32 acl_bufsize) { struct ptlrpc_request *req; struct obd_device *obddev = class_exp2obd(exp); @@ -333,8 +334,7 @@ static int mdc_save_lovea(struct ptlrpc_request *req, req_capsule_set_size(&req->rq_pill, &RMF_MDT_MD, RCL_SERVER, obddev->u.cli.cl_max_mds_easize); - req_capsule_set_size(&req->rq_pill, &RMF_ACL, RCL_SERVER, - req->rq_import->imp_connect_data.ocd_max_easize); + req_capsule_set_size(&req->rq_pill, &RMF_ACL, RCL_SERVER, acl_bufsize); ptlrpc_request_set_replen(req); return req; @@ -440,9 +440,9 @@ static struct ptlrpc_request *mdc_intent_unlink_pack(struct obd_export *exp, return req; } -static struct ptlrpc_request *mdc_intent_getattr_pack(struct obd_export *exp, - struct lookup_intent *it, - struct md_op_data *op_data) +static struct ptlrpc_request * +mdc_intent_getattr_pack(struct obd_export *exp, struct lookup_intent *it, + struct md_op_data *op_data, u32 acl_bufsize) { struct ptlrpc_request *req; struct obd_device *obddev = class_exp2obd(exp); @@ -480,8 +480,7 @@ static struct ptlrpc_request *mdc_intent_getattr_pack(struct obd_export *exp, mdc_getattr_pack(req, valid, it->it_flags, op_data, easize); req_capsule_set_size(&req->rq_pill, &RMF_MDT_MD, RCL_SERVER, easize); - req_capsule_set_size(&req->rq_pill, &RMF_ACL, RCL_SERVER, - req->rq_import->imp_connect_data.ocd_max_easize); + req_capsule_set_size(&req->rq_pill, &RMF_ACL, RCL_SERVER, acl_bufsize); ptlrpc_request_set_replen(req); return req; } @@ -782,6 +781,8 @@ int mdc_enqueue_base(struct obd_export *exp, struct ldlm_enqueue_info *einfo, struct ldlm_res_id res_id; int generation, resends = 0; struct ldlm_reply *lockrep; + struct obd_import *imp = class_exp2cliimp(exp); + u32 acl_bufsize; enum lvb_type lvb_type = LVB_T_NONE; int rc; @@ -804,6 +805,11 @@ int mdc_enqueue_base(struct obd_export *exp, struct ldlm_enqueue_info *einfo, } generation = obddev->u.cli.cl_import->imp_generation; + if (!it || (it->it_op & (IT_CREAT | IT_OPEN_CREAT))) + acl_bufsize = imp->imp_connect_data.ocd_max_easize; + else + acl_bufsize = LUSTRE_POSIX_ACL_MAX_SIZE_OLD; + resend: flags = saved_flags; if (!it) { @@ -812,15 +818,15 @@ int mdc_enqueue_base(struct obd_export *exp, struct ldlm_enqueue_info *einfo, einfo->ei_type); res_id.name[3] = LDLM_FLOCK; } else if (it->it_op & IT_OPEN) { - req = mdc_intent_open_pack(exp, it, op_data); + req = mdc_intent_open_pack(exp, it, op_data, acl_bufsize); } else if (it->it_op & IT_UNLINK) { req = mdc_intent_unlink_pack(exp, it, op_data); } else if (it->it_op & (IT_GETATTR | IT_LOOKUP)) { - req = mdc_intent_getattr_pack(exp, it, op_data); + req = mdc_intent_getattr_pack(exp, it, op_data, acl_bufsize); } else if (it->it_op & IT_READDIR) { req = mdc_enqueue_pack(exp, 0); } else if (it->it_op & IT_LAYOUT) { - if (!imp_connect_lvb_type(class_exp2cliimp(exp))) + if (!imp_connect_lvb_type(imp)) return -EOPNOTSUPP; req = mdc_intent_layout_pack(exp, it, op_data); lvb_type = LVB_T_LAYOUT; @@ -926,6 +932,15 @@ int mdc_enqueue_base(struct obd_export *exp, struct ldlm_enqueue_info *einfo, } } + if ((int)lockrep->lock_policy_res2 == -ERANGE && + it->it_op & (IT_OPEN | IT_GETATTR | IT_LOOKUP) && + acl_bufsize != imp->imp_connect_data.ocd_max_easize) { + mdc_clear_replay_flag(req, -ERANGE); + ptlrpc_req_finished(req); + acl_bufsize = imp->imp_connect_data.ocd_max_easize; + goto resend; + } + rc = mdc_finish_enqueue(exp, req, einfo, it, lockh, rc); if (rc < 0) { if (lustre_handle_is_used(lockh)) { @@ -1284,7 +1299,11 @@ int mdc_intent_getattr_async(struct obd_export *exp, PFID(&op_data->op_fid1), ldlm_it2str(it->it_op), it->it_flags); fid_build_reg_res_name(&op_data->op_fid1, &res_id); - req = mdc_intent_getattr_pack(exp, it, op_data); + /* If the MDT return -ERANGE because of large ACL, then the sponsor + * of the async getattr RPC will handle that by itself. + */ + req = mdc_intent_getattr_pack(exp, it, op_data, + LUSTRE_POSIX_ACL_MAX_SIZE_OLD); if (IS_ERR(req)) return PTR_ERR(req); diff --git a/fs/lustre/mdc/mdc_request.c b/fs/lustre/mdc/mdc_request.c index dd9ab5a..e344e78 100644 --- a/fs/lustre/mdc/mdc_request.c +++ b/fs/lustre/mdc/mdc_request.c @@ -188,10 +188,23 @@ static int mdc_getattr_common(struct obd_export *exp, return 0; } +static void mdc_reset_acl_req(struct ptlrpc_request *req) +{ + spin_lock(&req->rq_early_free_lock); + sptlrpc_cli_free_repbuf(req); + req->rq_repbuf = NULL; + req->rq_repbuf_len = 0; + req->rq_repdata = NULL; + req->rq_reqdata_len = 0; + spin_unlock(&req->rq_early_free_lock); +} + static int mdc_getattr(struct obd_export *exp, struct md_op_data *op_data, struct ptlrpc_request **request) { struct ptlrpc_request *req; + struct obd_import *imp = class_exp2cliimp(exp); + u32 acl_bufsize = LUSTRE_POSIX_ACL_MAX_SIZE_OLD; int rc; /* Single MDS without an LMV case */ @@ -199,8 +212,9 @@ static int mdc_getattr(struct obd_export *exp, struct md_op_data *op_data, op_data->op_mds = 0; return 0; } + *request = NULL; - req = ptlrpc_request_alloc(class_exp2cliimp(exp), &RQF_MDS_GETATTR); + req = ptlrpc_request_alloc(imp, &RQF_MDS_GETATTR); if (!req) return -ENOMEM; @@ -210,20 +224,28 @@ static int mdc_getattr(struct obd_export *exp, struct md_op_data *op_data, return rc; } +again: mdc_pack_body(req, &op_data->op_fid1, op_data->op_valid, op_data->op_mode, -1, 0); - - req_capsule_set_size(&req->rq_pill, &RMF_ACL, RCL_SERVER, - req->rq_import->imp_connect_data.ocd_max_easize); + req_capsule_set_size(&req->rq_pill, &RMF_ACL, RCL_SERVER, acl_bufsize); req_capsule_set_size(&req->rq_pill, &RMF_MDT_MD, RCL_SERVER, op_data->op_mode); ptlrpc_request_set_replen(req); rc = mdc_getattr_common(exp, req); - if (rc) + if (rc) { + if (rc == -ERANGE && + acl_bufsize != imp->imp_connect_data.ocd_max_easize) { + acl_bufsize = imp->imp_connect_data.ocd_max_easize; + mdc_reset_acl_req(req); + goto again; + } + ptlrpc_req_finished(req); - else + } else { *request = req; + } + return rc; } @@ -231,11 +253,12 @@ static int mdc_getattr_name(struct obd_export *exp, struct md_op_data *op_data, struct ptlrpc_request **request) { struct ptlrpc_request *req; + struct obd_import *imp = class_exp2cliimp(exp); + u32 acl_bufsize = LUSTRE_POSIX_ACL_MAX_SIZE_OLD; int rc; *request = NULL; - req = ptlrpc_request_alloc(class_exp2cliimp(exp), - &RQF_MDS_GETATTR_NAME); + req = ptlrpc_request_alloc(imp, &RQF_MDS_GETATTR_NAME); if (!req) return -ENOMEM; @@ -248,9 +271,6 @@ static int mdc_getattr_name(struct obd_export *exp, struct md_op_data *op_data, return rc; } - mdc_pack_body(req, &op_data->op_fid1, op_data->op_valid, - op_data->op_mode, op_data->op_suppgids[0], 0); - if (op_data->op_name) { char *name = req_capsule_client_get(&req->rq_pill, &RMF_NAME); @@ -259,17 +279,28 @@ static int mdc_getattr_name(struct obd_export *exp, struct md_op_data *op_data, memcpy(name, op_data->op_name, op_data->op_namelen); } +again: + mdc_pack_body(req, &op_data->op_fid1, op_data->op_valid, + op_data->op_mode, op_data->op_suppgids[0], 0); req_capsule_set_size(&req->rq_pill, &RMF_MDT_MD, RCL_SERVER, op_data->op_mode); - req_capsule_set_size(&req->rq_pill, &RMF_ACL, RCL_SERVER, - req->rq_import->imp_connect_data.ocd_max_easize); + req_capsule_set_size(&req->rq_pill, &RMF_ACL, RCL_SERVER, acl_bufsize); ptlrpc_request_set_replen(req); rc = mdc_getattr_common(exp, req); - if (rc) + if (rc) { + if (rc == -ERANGE && + acl_bufsize != imp->imp_connect_data.ocd_max_easize) { + acl_bufsize = imp->imp_connect_data.ocd_max_easize; + mdc_reset_acl_req(req); + goto again; + } + ptlrpc_req_finished(req); - else + } else { *request = req; + } + return rc; } diff --git a/fs/lustre/ptlrpc/pack_generic.c b/fs/lustre/ptlrpc/pack_generic.c index cac4b8d..1face33 100644 --- a/fs/lustre/ptlrpc/pack_generic.c +++ b/fs/lustre/ptlrpc/pack_generic.c @@ -164,6 +164,7 @@ u32 lustre_packed_msg_size(struct lustre_msg *msg) return 0; } } +EXPORT_SYMBOL(lustre_packed_msg_size); void lustre_init_msg_v2(struct lustre_msg_v2 *msg, int count, u32 *lens, char **bufs) diff --git a/fs/lustre/ptlrpc/sec.c b/fs/lustre/ptlrpc/sec.c index 21055a3..54ca97c 100644 --- a/fs/lustre/ptlrpc/sec.c +++ b/fs/lustre/ptlrpc/sec.c @@ -1680,6 +1680,7 @@ void sptlrpc_cli_free_repbuf(struct ptlrpc_request *req) policy->sp_cops->free_repbuf(ctx->cc_sec, req); req->rq_repmsg = NULL; } +EXPORT_SYMBOL(sptlrpc_cli_free_repbuf); static int sptlrpc_svc_install_rvs_ctx(struct obd_import *imp, struct ptlrpc_svc_ctx *ctx) From patchwork Mon Sep 30 18:56:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167273 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ABED71599 for ; Mon, 30 Sep 2019 19:06:01 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 936D72245A for ; Mon, 30 Sep 2019 19:06:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 936D72245A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id B49245E4B95; Mon, 30 Sep 2019 12:00:48 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id B358E5C3B3B for ; Mon, 30 Sep 2019 11:57:35 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 98A161005F07; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 97685BD; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:14 -0400 Message-Id: <1569869810-23848-116-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 115/151] lnet: safe access in debug print X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Amir Shehata , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Amir Shehata Move debug print within the cpt lock to keep peer access safe. WC-bug-id: https://jira.whamcloud.com/browse/LU-9919 Lustre-commit: 7c9ffeac1b69 ("LU-9919 lnet: safe access in debug print") Signed-off-by: Amir Shehata Reviewed-on: https://review.whamcloud.com/28771 Reviewed-by: John L. Hammond Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/lnet/lib-move.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/net/lnet/lnet/lib-move.c b/net/lnet/lnet/lib-move.c index 202372d..eee4510a 100644 --- a/net/lnet/lnet/lib-move.c +++ b/net/lnet/lnet/lib-move.c @@ -1422,11 +1422,10 @@ void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats, msg->msg_rtr_nid_param = rtr_nid; msg->msg_sending = 0; list_add_tail(&msg->msg_list, &peer->lp_dc_pendq); - lnet_peer_ni_decref_locked(lpni); - lnet_net_unlock(cpt); - CDEBUG(D_NET, "%s pending discovery\n", libcfs_nid2str(peer->lp_primary_nid)); + lnet_peer_ni_decref_locked(lpni); + lnet_net_unlock(cpt); return LNET_DC_WAIT; } @@ -1560,10 +1559,10 @@ void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats, best_lpni->lpni_net->net_id, cpt); /* If there is no best_ni we don't have a route */ if (!best_ni) { - lnet_net_unlock(cpt); CERROR("no path to %s from net %s\n", libcfs_nid2str(best_lpni->lpni_nid), libcfs_net2str(best_lpni->lpni_net->net_id)); + lnet_net_unlock(cpt); return -EHOSTUNREACH; } lpni = list_first_entry(&peer_net->lpn_peer_nis, @@ -1718,9 +1717,9 @@ void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats, * move onto the next peer_ni in the peer */ if (!best_gw) { - lnet_net_unlock(cpt); LCONSOLE_WARN("No route to peer from %s\n", libcfs_nid2str(best_ni->ni_nid)); + lnet_net_unlock(cpt); return -EHOSTUNREACH; } From patchwork Mon Sep 30 18:56:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167327 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2419F13B1 for ; Mon, 30 Sep 2019 19:08:05 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0C62F224EF for ; Mon, 30 Sep 2019 19:08:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0C62F224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 6C7C1602D07; Mon, 30 Sep 2019 12:01:36 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 03B405C3B75 for ; Mon, 30 Sep 2019 11:57:35 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 9C1591005F19; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 9A4B0B4; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:15 -0400 Message-Id: <1569869810-23848-117-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 116/151] lnet: Remove LASSERT on userspace data X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Sonia Sharma If the net information is not provided while adding NI, it results in an LBUG. Remove the LASSERT on userspace input and handle it gracefully. WC-bug-id: https://jira.whamcloud.com/browse/LU-10554 Lustre-commit: 8059dbbe97a6 ("LU-10554 lnet: Remove LASSERT on userspace data") Signed-off-by: Sonia Sharma Reviewed-on: https://review.whamcloud.com/31100 Reviewed-by: Amir Shehata Reviewed-by: Dmitry Eremin Reviewed-by: Andreas Dilger Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/lnet/api-ni.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/net/lnet/lnet/api-ni.c b/net/lnet/lnet/api-ni.c index 6e29714..fc4fe5d 100644 --- a/net/lnet/lnet/api-ni.c +++ b/net/lnet/lnet/api-ni.c @@ -1816,8 +1816,6 @@ static void lnet_push_target_fini(void) if (lnet_net_unique(net->net_id, &the_lnet.ln_nets, &net_l)) { lnd_type = LNET_NETTYP(net->net_id); - LASSERT(libcfs_isknown_lnd(lnd_type)); - mutex_lock(&the_lnet.ln_lnd_mutex); lnd = lnet_find_lnd_by_type(lnd_type); @@ -2715,7 +2713,7 @@ int lnet_dyn_add_ni(struct lnet_ioctl_config_ni *conf) struct lnet_ni *ni; struct lnet_ioctl_config_lnd_tunables *tun = NULL; int rc, i; - u32 net_id; + u32 net_id, lnd_type; /* get the tunables if they are available */ if (conf->lic_cfg_hdr.ioc_len >= @@ -2729,6 +2727,12 @@ int lnet_dyn_add_ni(struct lnet_ioctl_config_ni *conf) tun); net_id = LNET_NIDNET(conf->lic_nid); + lnd_type = LNET_NETTYP(net_id); + + if (!libcfs_isknown_lnd(lnd_type)) { + CERROR("No valid net and lnd information provided\n"); + return -EINVAL; + } net = lnet_net_alloc(net_id, NULL); if (!net) From patchwork Mon Sep 30 18:56:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167277 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 94F7013B1 for ; Mon, 30 Sep 2019 19:06:10 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7CF19224D2 for ; Mon, 30 Sep 2019 19:06:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7CF19224D2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id B78735E4BF7; Mon, 30 Sep 2019 12:00:51 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 451EF5C3B75 for ; Mon, 30 Sep 2019 11:57:36 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 9F3BB1005F1A; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 9D712B5; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:16 -0400 Message-Id: <1569869810-23848-118-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 117/151] lustre: flr: split a mirror from mirrored file X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Bobi Jam , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Bobi Jam Splits a mirror with mirror_id out of a mirrored file. WC-bug-id: https://jira.whamcloud.com/browse/LU-10420 Lustre-commit: 23b2d4781899 ("LU-10420 flr: split a mirror from mirrored file") Signed-off-by: Bobi Jam Reviewed-on: https://review.whamcloud.com/30388 Reviewed-by: Fan Yong Reviewed-by: Jinshan Xiong Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lu_object.h | 1 + fs/lustre/include/lustre_mds.h | 5 +++ fs/lustre/include/obd.h | 2 ++ fs/lustre/llite/file.c | 54 +++++++++++++++++++++++++++++++-- fs/lustre/llite/llite_lib.c | 3 +- fs/lustre/mdc/mdc_lib.c | 4 ++- include/uapi/linux/lustre/lustre_idl.h | 8 +++-- include/uapi/linux/lustre/lustre_user.h | 3 +- 8 files changed, 72 insertions(+), 8 deletions(-) diff --git a/fs/lustre/include/lu_object.h b/fs/lustre/include/lu_object.h index e1979be..e49954c 100644 --- a/fs/lustre/include/lu_object.h +++ b/fs/lustre/include/lu_object.h @@ -847,6 +847,7 @@ enum lu_xattr_flags { LU_XATTR_REPLACE = (1 << 0), LU_XATTR_CREATE = BIT(1), LU_XATTR_MERGE = BIT(2), + LU_XATTR_SPLIT = BIT(3), }; /** @} helpers */ diff --git a/fs/lustre/include/lustre_mds.h b/fs/lustre/include/lustre_mds.h index 18fe0e3..2510fe0 100644 --- a/fs/lustre/include/lustre_mds.h +++ b/fs/lustre/include/lustre_mds.h @@ -54,6 +54,11 @@ struct mds_group_info { int group; }; +struct md_rejig_data { + struct md_object *mrd_obj; + u16 mrd_mirror_id; +}; + #define MDD_OBD_NAME "mdd_obd" #define MDD_OBD_UUID "mdd_obd_uuid" diff --git a/fs/lustre/include/obd.h b/fs/lustre/include/obd.h index 1cb37a0..2db2596 100644 --- a/fs/lustre/include/obd.h +++ b/fs/lustre/include/obd.h @@ -780,6 +780,8 @@ struct md_op_data { u32 op_default_stripe_offset; u32 op_projid; + + u16 op_mirror_id; }; struct md_callback { diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 9d60587..92f4a43 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -50,6 +50,11 @@ #include #include "llite_internal.h" +struct split_param { + struct inode *sp_inode; + u16 sp_mirror_id; +}; + static int ll_put_grouplock(struct inode *inode, struct file *file, unsigned long arg); @@ -153,13 +158,22 @@ static int ll_close_inode_openhandle(struct inode *inode, op_data->op_attr.ia_valid |= ATTR_SIZE; op_data->op_xvalid |= OP_XVALID_BLOCKS; /* fallthrough */ - case MDS_CLOSE_LAYOUT_SWAP: + case MDS_CLOSE_LAYOUT_SPLIT: + case MDS_CLOSE_LAYOUT_SWAP: { + struct split_param *sp = data; + LASSERT(data); op_data->op_bias |= bias; op_data->op_data_version = 0; op_data->op_lease_handle = och->och_lease_handle; - op_data->op_fid2 = *ll_inode2fid(data); + if (bias == MDS_CLOSE_LAYOUT_SPLIT) { + op_data->op_fid2 = *ll_inode2fid(sp->sp_inode); + op_data->op_mirror_id = sp->sp_mirror_id; + } else { + op_data->op_fid2 = *ll_inode2fid(data); + } break; + } case MDS_CLOSE_RESYNC_DONE: { struct ll_ioc_lease *ioc = data; @@ -2659,6 +2673,7 @@ static long ll_file_unlock_lease(struct file *file, struct ll_ioc_lease *ioc, struct ll_file_data *fd = LUSTRE_FPRIVATE(file); struct ll_inode_info *lli = ll_i2info(inode); struct obd_client_handle *och = NULL; + struct split_param sp; bool lease_broken; fmode_t fmode = 0; enum mds_op_bias bias = 0; @@ -2732,6 +2747,40 @@ static long ll_file_unlock_lease(struct file *file, struct ll_ioc_lease *ioc, bias = MDS_CLOSE_LAYOUT_MERGE; break; } + case LL_LEASE_LAYOUT_SPLIT: { + int fdv; + int mirror_id; + + if (ioc->lil_count != 2) { + rc = -EINVAL; + goto out; + } + + arg += sizeof(*ioc); + if (copy_from_user(&fdv, (void __user *)arg, sizeof(u32))) { + rc = -EFAULT; + goto out; + } + + arg += sizeof(u32); + if (copy_from_user(&mirror_id, (void __user *)arg, + sizeof(u32))) { + rc = -EFAULT; + goto out; + } + + layout_file = fget(fdv); + if (!layout_file) { + rc = -EBADF; + goto out; + } + + sp.sp_inode = file_inode(layout_file); + sp.sp_mirror_id = (u16)mirror_id; + data = &sp; + bias = MDS_CLOSE_LAYOUT_SPLIT; + break; + } default: /* without close intent */ break; @@ -2754,6 +2803,7 @@ static long ll_file_unlock_lease(struct file *file, struct ll_ioc_lease *ioc, kfree(data); break; case LL_LEASE_LAYOUT_MERGE: + case LL_LEASE_LAYOUT_SPLIT: if (layout_file) fput(layout_file); break; diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index 4f4234d..2a7ef64 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -2427,11 +2427,10 @@ struct md_op_data *ll_prep_md_op_data(struct md_op_data *op_data, op_data->op_fsuid = from_kuid(&init_user_ns, current_fsuid()); op_data->op_fsgid = from_kgid(&init_user_ns, current_fsgid()); op_data->op_cap = current_cap(); + op_data->op_mds = 0; if ((opc == LUSTRE_OPC_CREATE) && name && filename_is_volatile(name, namelen, &op_data->op_mds)) op_data->op_bias |= MDS_CREATE_VOLATILE; - else - op_data->op_mds = 0; op_data->op_data = data; return op_data; diff --git a/fs/lustre/mdc/mdc_lib.c b/fs/lustre/mdc/mdc_lib.c index dc5dac4..1031a01 100644 --- a/fs/lustre/mdc/mdc_lib.c +++ b/fs/lustre/mdc/mdc_lib.c @@ -451,7 +451,9 @@ static void mdc_close_intent_pack(struct ptlrpc_request *req, data->cd_data_version = op_data->op_data_version; data->cd_fid = op_data->op_fid2; - if (bias & MDS_CLOSE_RESYNC_DONE) { + if (bias & MDS_CLOSE_LAYOUT_SPLIT) { + data->cd_mirror_id = op_data->op_mirror_id; + } else if (bias & MDS_CLOSE_RESYNC_DONE) { struct close_data_resync_done *sync = &data->cd_resync; BUILD_BUG_ON(sizeof(data->cd_resync) > sizeof(data->cd_reserved)); diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index 4b33104..6dce6f1 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -1654,10 +1654,12 @@ enum mds_op_bias { MDS_CLOSE_LAYOUT_SWAP = 1 << 14, MDS_CLOSE_LAYOUT_MERGE = 1 << 15, MDS_CLOSE_RESYNC_DONE = 1 << 16, + MDS_CLOSE_LAYOUT_SPLIT = 1 << 17, }; -#define MDS_CLOSE_INTENT (MDS_HSM_RELEASE | MDS_CLOSE_LAYOUT_SWAP | \ - MDS_CLOSE_LAYOUT_MERGE | MDS_CLOSE_RESYNC_DONE) +#define MDS_CLOSE_INTENT (MDS_HSM_RELEASE | MDS_CLOSE_LAYOUT_SWAP | \ + MDS_CLOSE_LAYOUT_MERGE | MDS_CLOSE_LAYOUT_SPLIT | \ + MDS_CLOSE_RESYNC_DONE) /* instance of mdt_reint_rec */ struct mdt_rec_create { @@ -2863,6 +2865,8 @@ struct close_data { union { __u64 cd_reserved[8]; struct close_data_resync_done cd_resync; + /* split close */ + __u16 cd_mirror_id; }; }; diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index f3fa24b..cea7d13 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -254,6 +254,7 @@ enum ll_lease_flags { LL_LEASE_RESYNC = 0x1, LL_LEASE_RESYNC_DONE = 0x2, LL_LEASE_LAYOUT_MERGE = 0x4, + LL_LEASE_LAYOUT_SPLIT = 0x8, }; #define IOC_IDS_MAX 4096 @@ -766,7 +767,7 @@ struct identity_downcall_data { }; /* lustre volatile file support - * file name header: .^L^S^T^R:volatile" + * file name header: ".^L^S^T^R:volatile" */ #define LUSTRE_VOLATILE_HDR ".\x0c\x13\x14\x12:VOLATILE" #define LUSTRE_VOLATILE_HDR_LEN 14 From patchwork Mon Sep 30 18:56:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167217 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B7FA215AB for ; Mon, 30 Sep 2019 19:03:55 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A0863224EF for ; Mon, 30 Sep 2019 19:03:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A0863224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id ADCAF5E45ED; Mon, 30 Sep 2019 11:59:50 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 9B6B15C3B96 for ; Mon, 30 Sep 2019 11:57:36 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id A15B41005F1B; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id A0345BB; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:17 -0400 Message-Id: <1569869810-23848-119-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 118/151] lustre: llite: deny 2.10 clients to open mirrored files X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Jinshan Xiong 2.10 clients would manipulate mirrored layout as PFL layout, which would damage mirrored files. This patch only allows mirrored files to be opened by clients who understand mirror layout. It also fixes the problem that it should check OBD_CONNECT_FLAGS2 first before checking OBD_CONNECT2_XXX flags. WC-bug-id: https://jira.whamcloud.com/browse/LU-10286 Lustre-commit: 21e39775a0f4 ("LU-10286 llite: deny 2.10 clients to open mirrored files") Signed-off-by: Jinshan Xiong Reviewed-on: https://review.whamcloud.com/30957 Tested-by: Jian Yu Reviewed-by: Jian Yu Reviewed-by: Andreas Dilger Signed-off-by: James Simmons --- fs/lustre/include/lustre_export.h | 9 ++++++++- fs/lustre/llite/llite_lib.c | 2 +- include/uapi/linux/lustre/lustre_idl.h | 1 + 3 files changed, 10 insertions(+), 2 deletions(-) diff --git a/fs/lustre/include/lustre_export.h b/fs/lustre/include/lustre_export.h index 544166f..de3b109 100644 --- a/fs/lustre/include/lustre_export.h +++ b/fs/lustre/include/lustre_export.h @@ -156,7 +156,9 @@ static inline u64 *exp_connect_flags2_ptr(struct obd_export *exp) static inline u64 exp_connect_flags2(struct obd_export *exp) { - return *exp_connect_flags2_ptr(exp); + if (exp_connect_flags(exp) & OBD_CONNECT_FLAGS2) + return *exp_connect_flags2_ptr(exp); + return 0; } static inline int exp_max_brw_size(struct obd_export *exp) @@ -262,6 +264,11 @@ static inline int exp_connect_lockahead(struct obd_export *exp) return !!(exp_connect_flags2(exp) & OBD_CONNECT2_LOCKAHEAD); } +static inline int exp_connect_flr(struct obd_export *exp) +{ + return !!(exp_connect_flags2(exp) & OBD_CONNECT2_FLR); +} + struct obd_export *class_conn2export(struct lustre_handle *conn); #define KKUC_CT_DATA_MAGIC 0x092013cea diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index 2a7ef64..efd1093 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -208,7 +208,7 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt) OBD_CONNECT_GRANT_PARAM | OBD_CONNECT_SHORTIO | OBD_CONNECT_FLAGS2; - data->ocd_connect_flags2 = 0; + data->ocd_connect_flags2 = OBD_CONNECT2_FLR; if (sbi->ll_flags & LL_SBI_LRU_RESIZE) data->ocd_connect_flags |= OBD_CONNECT_LRU_RESIZE; diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index 6dce6f1..6294eaa 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -791,6 +791,7 @@ struct ptlrpc_body_v2 { #define OBD_CONNECT2_LOCKAHEAD 0x2ULL /* ladvise lockahead * v2 */ +#define OBD_CONNECT2_FLR 0x20ULL /* FLR support */ /* XXX README XXX: * Please DO NOT add flag values here before first ensuring that this same From patchwork Mon Sep 30 18:56:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167221 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E685A13B1 for ; Mon, 30 Sep 2019 19:04:04 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CE99D224EF for ; Mon, 30 Sep 2019 19:04:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CE99D224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id C50695E4651; Mon, 30 Sep 2019 11:59:53 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id DC0B15C3B96 for ; Mon, 30 Sep 2019 11:57:36 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id A4C701005F1C; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id A3A0FA9; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:18 -0400 Message-Id: <1569869810-23848-120-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 119/151] lustre: uapi: rename LCM_FL_NOT_FLR to LCM_FL_NONE X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Andreas Dilger Having "lfs getstripe" print out "lcm_flags: not_flr" is not very useful, as the composite files may not relate to FLR (e.g. PFL). Rename "LCM_FL_NOT_FLR" to "LCM_FL_NONE" so it is more clear there are no composite flags on the layout. Print out "0" for flags if no flags are set, to match old behaviour. WC-bug-id: https://jira.whamcloud.com/browse/LU-9771 Lustre-commit: fa15e5347c2e ("LU-9771 util: rename LCM_FL_NOT_FLR to LCM_FL_NONE") Signed-off-by: Andreas Dilger Reviewed-on: https://review.whamcloud.com/31047 Reviewed-by: Bobi Jam Reviewed-by: Jian Yu Reviewed-by: Jinshan Xiong Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/lov/lov_cl_internal.h | 4 ++-- fs/lustre/lov/lov_object.c | 4 ++-- fs/lustre/ptlrpc/wiretest.c | 4 ++-- include/uapi/linux/lustre/lustre_user.h | 2 +- 4 files changed, 7 insertions(+), 7 deletions(-) diff --git a/fs/lustre/lov/lov_cl_internal.h b/fs/lustre/lov/lov_cl_internal.h index 4ab8fba..875af37 100644 --- a/fs/lustre/lov/lov_cl_internal.h +++ b/fs/lustre/lov/lov_cl_internal.h @@ -354,14 +354,14 @@ static inline struct lov_stripe_md_entry *lov_lse(struct lov_object *lov, int i) static inline unsigned int lov_flr_state(const struct lov_object *lov) { if (lov->lo_type != LLT_COMP) - return LCM_FL_NOT_FLR; + return LCM_FL_NONE; return lov->u.composite.lo_flags & LCM_FL_FLR_MASK; } static inline bool lov_is_flr(const struct lov_object *lov) { - return lov_flr_state(lov) != LCM_FL_NOT_FLR; + return lov_flr_state(lov) != LCM_FL_NONE; } static inline struct lov_layout_entry *lov_entry(struct lov_object *lov, int i) diff --git a/fs/lustre/lov/lov_object.c b/fs/lustre/lov/lov_object.c index 51b95f2..6ba821b 100644 --- a/fs/lustre/lov/lov_object.c +++ b/fs/lustre/lov/lov_object.c @@ -629,7 +629,7 @@ static int lov_init_composite(const struct lu_env *env, struct lov_device *dev, comp->lo_entry_count = lsm->lsm_entry_count; comp->lo_preferred_mirror = -1; - if (equi(flr_state == LCM_FL_NOT_FLR, comp->lo_mirror_count > 1)) + if (equi(flr_state == LCM_FL_NONE, comp->lo_mirror_count > 1)) return -EINVAL; comp->lo_mirrors = kcalloc(comp->lo_mirror_count, @@ -669,7 +669,7 @@ static int lov_init_composite(const struct lu_env *env, struct lov_device *dev, lle->lle_extent = &lle->lle_lsme->lsme_extent; lle->lle_valid = !(lle->lle_lsme->lsme_flags & LCME_FL_STALE); - if (flr_state != LCM_FL_NOT_FLR) + if (flr_state != LCM_FL_NONE) mirror_id = mirror_id_of(lle->lle_lsme->lsme_id); lre = &comp->lo_mirrors[j]; diff --git a/fs/lustre/ptlrpc/wiretest.c b/fs/lustre/ptlrpc/wiretest.c index e161fe6..70ed2cd 100644 --- a/fs/lustre/ptlrpc/wiretest.c +++ b/fs/lustre/ptlrpc/wiretest.c @@ -1542,8 +1542,8 @@ void lustre_assert_wire_constants(void) LASSERTF((int)sizeof(((struct lov_comp_md_v1 *)0)->lcm_entries[0]) == 48, "found %lld\n", (long long)(int)sizeof(((struct lov_comp_md_v1 *)0)->lcm_entries[0])); BUILD_BUG_ON(LOV_MAGIC_COMP_V1 != (0x0BD60000 | 0x0BD0)); - LASSERTF(LCM_FL_NOT_FLR == 0, "found %lld\n", - (long long)LCM_FL_NOT_FLR); + LASSERTF(LCM_FL_NONE == 0, "found %lld\n", + (long long)LCM_FL_NONE); LASSERTF(LCM_FL_RDONLY == 1, "found %lld\n", (long long)LCM_FL_RDONLY); LASSERTF(LCM_FL_WRITE_PENDING == 2, "found %lld\n", diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index cea7d13..4e673cd 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -530,7 +530,7 @@ static inline __u16 mirror_id_of(__u32 id) */ enum lov_comp_md_flags { /* the least 2 bits are used by FLR to record file state */ - LCM_FL_NOT_FLR = 0, + LCM_FL_NONE = 0, LCM_FL_RDONLY = 1, LCM_FL_WRITE_PENDING = 2, LCM_FL_SYNC_PENDING = 3, From patchwork Mon Sep 30 18:56:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167311 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E4A321599 for ; Mon, 30 Sep 2019 19:07:34 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CCC92224EF for ; Mon, 30 Sep 2019 19:07:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CCC92224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id EA1785E4ED2; Mon, 30 Sep 2019 12:01:22 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 430AB5C3B96 for ; Mon, 30 Sep 2019 11:57:37 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id A89C51005F80; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id A6DD4BD; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:19 -0400 Message-Id: <1569869810-23848-121-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 120/151] lustre: flr: layout truncate compatibility X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Bobi Jam , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Bobi Jam In PFL design, client issues [0, size) intent to MDS to instantiate objects in this extent. While in FLR design, the intent serves two purposes: 1) make objects across [size, EOF) on other mirrors stale, and 2) instantiate objects in the chosen write mirror. And original FLR chose to use [size, EOF) as the extent of truncate write intent request. This patch reverts the choice, and still uses [0, size) as the truncate write intent extent. WC-bug-id: https://jira.whamcloud.com/browse/LU-10438 Lustre-commit: 795ca68f1360 ("LU-10438 flr: layout truncate compatibility") Signed-off-by: Bobi Jam Reviewed-on: https://review.whamcloud.com/30786 Reviewed-by: Andreas Dilger Reviewed-by: Jinshan Xiong Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/lov/lov_io.c | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/fs/lustre/lov/lov_io.c b/fs/lustre/lov/lov_io.c index 54e2d7d..5b28793 100644 --- a/fs/lustre/lov/lov_io.c +++ b/fs/lustre/lov/lov_io.c @@ -357,6 +357,16 @@ static int lov_io_mirror_init(struct lov_io *lio, struct lov_object *obj, PFID(lu_object_fid(lov2lu(obj))), lio->lis_pos, lio->lis_endpos); + if (cl_io_is_trunc(io)) { + /** + * for truncate, we uses [size, EOF) to judge whether + * a write intent needs to be send, but we need to + * restore the write extent to [0, size). + */ + io->ci_write_intent.e_start = 0; + io->ci_write_intent.e_end = + io->u.ci_setattr.sa_attr.lvb_size; + } /* stop cl_io_init() loop */ return 1; } @@ -441,7 +451,6 @@ static int lov_io_mirror_init(struct lov_io *lio, struct lov_object *obj, static int lov_io_slice_init(struct lov_io *lio, struct lov_object *obj, struct cl_io *io) { - struct lu_extent ext; int index; int result = 0; @@ -538,20 +547,19 @@ static int lov_io_slice_init(struct lov_io *lio, struct lov_object *obj, goto out; } - io->ci_write_intent.e_start = lio->lis_pos; - io->ci_write_intent.e_end = lio->lis_endpos; - - ext = io->ci_write_intent; /* for truncate, it only needs to instantiate the components * before the truncated size. */ if (cl_io_is_trunc(io)) { - ext.e_start = 0; - ext.e_end = io->u.ci_setattr.sa_attr.lvb_size; + io->ci_write_intent.e_start = 0; + io->ci_write_intent.e_end = io->u.ci_setattr.sa_attr.lvb_size; + } else { + io->ci_write_intent.e_start = lio->lis_pos; + io->ci_write_intent.e_end = lio->lis_endpos; } index = 0; - lov_foreach_io_layout(index, lio, &ext) { + lov_foreach_io_layout(index, lio, &io->ci_write_intent) { if (!lsm_entry_inited(obj->lo_lsm, index)) { io->ci_need_write_intent = 1; break; From patchwork Mon Sep 30 18:56:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167281 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5AA051599 for ; Mon, 30 Sep 2019 19:06:21 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 43033224D2 for ; Mon, 30 Sep 2019 19:06:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 43033224D2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id DFA075E4C55; Mon, 30 Sep 2019 12:00:54 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8864A5C3B96 for ; Mon, 30 Sep 2019 11:57:37 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id AAF901005F81; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id AA046B4; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:20 -0400 Message-Id: <1569869810-23848-122-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 121/151] lustre: mdc: high-priority request handling for DOM X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mikhal Pershin , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mikhal Pershin Implement high-priority request handling and lock prolongation for Data-on-MDT BRW requests to avoid incorrect timeouts and client eviction under heavy MDS load. WC-bug-id: https://jira.whamcloud.com/browse/LU-10181 Lustre-commit: 9e338ee25609 ("LU-10181 mdc: high-priority request handling for DOM") Signed-off-by: Mikhal Pershin Reviewed-on: https://review.whamcloud.com/29968 Reviewed-by: Andreas Dilger Reviewed-by: Alex Zhuravlev Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/mdc/mdc_dev.c | 72 +++++++++++++++++++++++++++++++------------------ 1 file changed, 46 insertions(+), 26 deletions(-) diff --git a/fs/lustre/mdc/mdc_dev.c b/fs/lustre/mdc/mdc_dev.c index 4ff5cc6..2de849a 100644 --- a/fs/lustre/mdc/mdc_dev.c +++ b/fs/lustre/mdc/mdc_dev.c @@ -937,6 +937,34 @@ static int mdc_async_upcall(void *a, int rc) return 0; } +static int mdc_get_lock_handle(const struct lu_env *env, struct osc_object *osc, + pgoff_t index, struct lustre_handle *lh) +{ + struct ldlm_lock *lock; + int rc = 0; + + /* find DOM lock protecting object */ + lock = mdc_dlmlock_at_pgoff(env, osc, index, + OSC_DAP_FL_TEST_LOCK | + OSC_DAP_FL_CANCELING); + if (!lock) { + struct ldlm_resource *res; + struct ldlm_res_id *resname; + + resname = &osc_env_info(env)->oti_resname; + fid_build_reg_res_name(lu_object_fid(osc2lu(osc)), resname); + res = ldlm_resource_get(osc_export(osc)->exp_obd->obd_namespace, + NULL, resname, LDLM_IBITS, 0); + ldlm_resource_dump(D_ERROR, res); + libcfs_debug_dumpstack(NULL); + rc = -ENOENT; + } else { + *lh = lock->l_remote_handle; + LDLM_LOCK_PUT(lock); + } + return rc; +} + static int mdc_io_setattr_start(const struct lu_env *env, const struct cl_io_slice *slice) { @@ -1008,6 +1036,11 @@ static int mdc_io_setattr_start(const struct lu_env *env, if (oio->oi_lockless) { oa->o_flags = OBD_FL_SRVLOCK; oa->o_valid |= OBD_MD_FLFLAGS; + } else { + rc = mdc_get_lock_handle(env, cl2osc(obj), CL_PAGE_EOF, + &oa->o_handle); + if (!rc) + oa->o_valid |= OBD_MD_FLHANDLE; } init_completion(&cbargs->opc_sync); @@ -1157,35 +1190,22 @@ static void mdc_req_attr_set(const struct lu_env *env, struct cl_object *obj, attr->cra_oa->o_valid |= OBD_MD_FLID; if (flags & OBD_MD_FLHANDLE) { - struct ldlm_lock *lock; /* _some_ lock protecting @apage */ struct osc_page *opg; opg = osc_cl_page_osc(attr->cra_page, cl2osc(obj)); - lock = mdc_dlmlock_at_pgoff(env, cl2osc(obj), osc_index(opg), - OSC_DAP_FL_TEST_LOCK | OSC_DAP_FL_CANCELING); - if (!lock && !opg->ops_srvlock) { - struct ldlm_resource *res; - struct ldlm_res_id *resname; - - CL_PAGE_DEBUG(D_ERROR, env, attr->cra_page, - "uncovered page!\n"); - - resname = &osc_env_info(env)->oti_resname; - mdc_build_res_name(cl2osc(obj), resname); - res = ldlm_resource_get( - osc_export(cl2osc(obj))->exp_obd->obd_namespace, - NULL, resname, LDLM_IBITS, 0); - ldlm_resource_dump(D_ERROR, res); - - libcfs_debug_dumpstack(NULL); - LBUG(); - } - - /* check for lockless io. */ - if (lock) { - attr->cra_oa->o_handle = lock->l_remote_handle; - attr->cra_oa->o_valid |= OBD_MD_FLHANDLE; - LDLM_LOCK_PUT(lock); + if (!opg->ops_srvlock) { + int rc; + + rc = mdc_get_lock_handle(env, cl2osc(obj), + osc_index(opg), + &attr->cra_oa->o_handle); + if (rc) { + CL_PAGE_DEBUG(D_ERROR, env, attr->cra_page, + "uncovered page!\n"); + LBUG(); + } else { + attr->cra_oa->o_valid |= OBD_MD_FLHANDLE; + } } } } From patchwork Mon Sep 30 18:56:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167287 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 03E5B13B1 for ; Mon, 30 Sep 2019 19:06:30 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E031A224F0 for ; Mon, 30 Sep 2019 19:06:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E031A224F0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 9B0645E4CAE; Mon, 30 Sep 2019 12:00:58 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id E26E25C3A45 for ; Mon, 30 Sep 2019 11:57:37 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id ADB9A1005F82; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id ACBDAB5; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:21 -0400 Message-Id: <1569869810-23848-123-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 122/151] lustre: llite: Add tiny write support X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Patrick Farrell If a page is already dirty in the page cache, we can write to it without a full i/o. This improves performance for writes of < 1 page dramatically. Append writes are a bit tricky, requiring us to take the range lock (which we can normally avoid), but they are still much faster than the normal i/o path. Performance numbers with dd, on a VM with an older Xeon. All numbers in MiB/s. 8 bytes 1KiB Without patch: .75 75 With patch: 6.5 153 WC-bug-id: https://jira.whamcloud.com/browse/LU-9409 Cray-bug-id: LUS-1705 Lustre-commit: 94470f7eeab5 ("LU-9409 llite: Add tiny write support") Signed-off-by: Patrick Farrell Reviewed-on: https://review.whamcloud.com/27903 Reviewed-by: Jinshan Xiong Reviewed-by: Alexey Lyashkov Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/cl_object.h | 9 ++++ fs/lustre/llite/file.c | 122 ++++++++++++++++++++++++++++++++++++++++-- fs/lustre/llite/rw26.c | 71 ++++++++++++++++++++++-- fs/lustre/obdclass/cl_page.c | 13 +++++ fs/lustre/osc/osc_internal.h | 2 + fs/lustre/osc/osc_io.c | 16 ++---- fs/lustre/osc/osc_page.c | 12 ++++- 7 files changed, 224 insertions(+), 21 deletions(-) diff --git a/fs/lustre/include/cl_object.h b/fs/lustre/include/cl_object.h index 1088fde..c96a5b7 100644 --- a/fs/lustre/include/cl_object.h +++ b/fs/lustre/include/cl_object.h @@ -865,6 +865,13 @@ struct cl_page_operations { */ int (*cpo_is_vmlocked)(const struct lu_env *env, const struct cl_page_slice *slice); + + /** + * Update file attributes when all we have is this page. Used for tiny + * writes to update attributes when we don't have a full cl_io. + */ + void (*cpo_page_touch)(const struct lu_env *env, + const struct cl_page_slice *slice, size_t to); /** * Page destruction. */ @@ -2203,6 +2210,8 @@ void cl_page_discard(const struct lu_env *env, struct cl_io *io, struct cl_page *pg); void cl_page_delete(const struct lu_env *env, struct cl_page *pg); int cl_page_is_vmlocked(const struct lu_env *env, const struct cl_page *pg); +void cl_page_touch(const struct lu_env *env, const struct cl_page *pg, + size_t to); void cl_page_export(const struct lu_env *env, struct cl_page *pg, int uptodate); loff_t cl_offset(const struct cl_object *obj, pgoff_t idx); pgoff_t cl_index(const struct cl_object *obj, loff_t offset); diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 92f4a43..da5bf86 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -1475,6 +1475,101 @@ static ssize_t ll_file_read_iter(struct kiocb *iocb, struct iov_iter *to) return result; } +/** + * Similar trick to ll_do_fast_read, this improves write speed for tiny writes. + * If a page is already in the page cache and dirty (and some other things - + * See ll_tiny_write_begin for the instantiation of these rules), then we can + * write to it without doing a full I/O, because Lustre already knows about it + * and will write it out. This saves a lot of processing time. + * + * All writes here are within one page, so exclusion is handled by the page + * lock on the vm page. Exception is appending, which requires locking the + * full file to handle size issues. We do not do tiny writes for writes which + * touch multiple pages because it's very unlikely multiple sequential pages + * are already dirty. + * + * We limit these to < PAGE_SIZE because PAGE_SIZE writes are relatively common + * and are unlikely to be to already dirty pages. + * + * Attribute updates are important here, we do it in ll_tiny_write_end. + */ +static ssize_t ll_do_tiny_write(struct kiocb *iocb, struct iov_iter *iter) +{ + ssize_t count = iov_iter_count(iter); + struct file *file = iocb->ki_filp; + struct inode *inode = file_inode(file); + struct ll_inode_info *lli = ll_i2info(inode); + struct range_lock range; + ssize_t result = 0; + bool append = false; + + /* NB: we can't do direct IO for tiny writes because they use the page + * cache, and we can't do sync writes because tiny writes can't flush + * pages. + */ + if (file->f_flags & (O_DIRECT | O_SYNC)) + return 0; + + /* It is relatively unlikely we will overwrite a full dirty page, so + * limit tiny writes to < PAGE_SIZE + */ + if (count >= PAGE_SIZE) + return 0; + + /* For append writes, we must take the range lock to protect size + * and also move pos to current size before writing. + */ + if (file->f_flags & O_APPEND) { + struct lu_env *env; + u16 refcheck; + + append = true; + range_lock_init(&range, 0, LUSTRE_EOF); + result = range_lock(&lli->lli_write_tree, &range); + if (result) + return result; + env = cl_env_get(&refcheck); + if (IS_ERR(env)) { + result = PTR_ERR(env); + goto out; + } + ll_merge_attr(env, inode); + cl_env_put(env, &refcheck); + iocb->ki_pos = i_size_read(inode); + } + + /* Does this write touch multiple pages? + * + * This partly duplicates the PAGE_SIZE check above, but must come + * after range locking for append writes because it depends on the + * write position (ki_pos). + */ + if ((iocb->ki_pos & (PAGE_SIZE-1)) + count > PAGE_SIZE) + goto out; + + result = __generic_file_write_iter(iocb, iter); + + /* If the page is not already dirty, ll_tiny_write_begin returns + * -ENODATA. We continue on to normal write. + */ + if (result == -ENODATA) + result = 0; + + if (result > 0) { + ll_stats_ops_tally(ll_i2sbi(inode), LPROC_LL_WRITE_BYTES, + result); + set_bit(LLIF_DATA_MODIFIED, &ll_i2info(inode)->lli_flags); + } + +out: + if (append) + range_unlock(&lli->lli_write_tree, &range); + + CDEBUG(D_VFSTRACE, "result: %zu, original count %zu\n", result, count); + + return result; +} + /* * Write to a file (through the page cache). */ @@ -1482,9 +1577,19 @@ static ssize_t ll_file_write_iter(struct kiocb *iocb, struct iov_iter *from) { struct lu_env *env; struct vvp_io_args *args; - ssize_t result; + ssize_t rc_tiny, rc_normal; u16 refcheck; + rc_tiny = ll_do_tiny_write(iocb, from); + + /* In case of error, go on and try normal write - Only stop if tiny + * write completed I/O. + */ + if (iov_iter_count(from) == 0) { + rc_normal = rc_tiny; + goto out; + } + env = cl_env_get(&refcheck); if (IS_ERR(env)) return PTR_ERR(env); @@ -1493,10 +1598,21 @@ static ssize_t ll_file_write_iter(struct kiocb *iocb, struct iov_iter *from) args->u.normal.via_iter = from; args->u.normal.via_iocb = iocb; - result = ll_file_io_generic(env, args, iocb->ki_filp, CIT_WRITE, + rc_normal = ll_file_io_generic(env, args, iocb->ki_filp, CIT_WRITE, &iocb->ki_pos, iov_iter_count(from)); + + /* On success, combine bytes written. */ + if (rc_tiny >= 0 && rc_normal > 0) + rc_normal += rc_tiny; + /* On error, only return error from normal write if tiny write did not + * write any bytes. Otherwise return bytes written by tiny write. + */ + else if (rc_tiny > 0) + rc_normal = rc_tiny; + cl_env_put(env, &refcheck); - return result; +out: + return rc_normal; } int ll_lov_setstripe_ea_info(struct inode *inode, struct dentry *dentry, diff --git a/fs/lustre/llite/rw26.c b/fs/lustre/llite/rw26.c index 37b6755..2baab10 100644 --- a/fs/lustre/llite/rw26.c +++ b/fs/lustre/llite/rw26.c @@ -443,13 +443,23 @@ static int ll_prepare_partial_page(const struct lu_env *env, struct cl_io *io, return result; } +static int ll_tiny_write_begin(struct page *vmpage) +{ + /* Page must be present, up to date, dirty, and not in writeback. */ + if (!vmpage || !PageUptodate(vmpage) || !PageDirty(vmpage) || + PageWriteback(vmpage)) + return -ENODATA; + + return 0; +} + static int ll_write_begin(struct file *file, struct address_space *mapping, loff_t pos, unsigned int len, unsigned int flags, struct page **pagep, void **fsdata) { - struct ll_cl_context *lcc; + struct ll_cl_context *lcc = NULL; const struct lu_env *env = NULL; - struct cl_io *io; + struct cl_io *io = NULL; struct cl_page *page = NULL; struct cl_object *clob = ll_i2info(mapping->host)->lli_clob; pgoff_t index = pos >> PAGE_SHIFT; @@ -462,8 +472,8 @@ static int ll_write_begin(struct file *file, struct address_space *mapping, lcc = ll_cl_find(file); if (!lcc) { - io = NULL; - result = -EIO; + vmpage = grab_cache_page_nowait(mapping, index); + result = ll_tiny_write_begin(vmpage); goto out; } @@ -479,6 +489,7 @@ static int ll_write_begin(struct file *file, struct address_space *mapping, result = -EBUSY; goto out; } + again: /* To avoid deadlock, try to lock page first. */ vmpage = grab_cache_page_nowait(mapping, index); @@ -544,7 +555,6 @@ static int ll_write_begin(struct file *file, struct address_space *mapping, if (result == -EAGAIN) goto again; - goto out; } } @@ -555,6 +565,7 @@ static int ll_write_begin(struct file *file, struct address_space *mapping, unlock_page(vmpage); put_page(vmpage); } + /* On tiny_write failure, page and io are always null. */ if (!IS_ERR_OR_NULL(page)) { lu_ref_del(&page->cp_reference, "cl_io", io); cl_page_put(env, page); @@ -568,6 +579,45 @@ static int ll_write_begin(struct file *file, struct address_space *mapping, return result; } +static int ll_tiny_write_end(struct file *file, struct address_space *mapping, + loff_t pos, unsigned int len, unsigned int copied, + struct page *vmpage) +{ + struct cl_page *clpage = (struct cl_page *) vmpage->private; + loff_t kms = pos+copied; + loff_t to = kms & (PAGE_SIZE-1) ? kms & (PAGE_SIZE-1) : PAGE_SIZE; + u16 refcheck; + struct lu_env *env = cl_env_get(&refcheck); + int rc = 0; + + if (IS_ERR(env)) { + rc = PTR_ERR(env); + goto out; + } + + /* This page is dirty in cache, so it should have a cl_page pointer + * set in vmpage->private. + */ + LASSERT(clpage); + + if (copied == 0) + goto out_env; + + /* Update the underlying size information in the OSC/LOV objects this + * page is part of. + */ + cl_page_touch(env, clpage, to); + +out_env: + cl_env_put(env, &refcheck); + +out: + /* Must return page unlocked. */ + unlock_page(vmpage); + + return rc; +} + static int ll_write_end(struct file *file, struct address_space *mapping, loff_t pos, unsigned int len, unsigned int copied, struct page *vmpage, void *fsdata) @@ -583,6 +633,14 @@ static int ll_write_end(struct file *file, struct address_space *mapping, put_page(vmpage); + CDEBUG(D_VFSTRACE, "pos %llu, len %u, copied %u\n", pos, len, copied); + + if (!lcc) { + result = ll_tiny_write_end(file, mapping, pos, len, copied, + vmpage); + goto out; + } + env = lcc->lcc_env; page = lcc->lcc_page; io = lcc->lcc_io; @@ -632,6 +690,9 @@ static int ll_write_end(struct file *file, struct address_space *mapping, if (result < 0) io->ci_result = result; + + +out: return result >= 0 ? copied : result; } diff --git a/fs/lustre/obdclass/cl_page.c b/fs/lustre/obdclass/cl_page.c index 8ea63f7..8dbd312 100644 --- a/fs/lustre/obdclass/cl_page.c +++ b/fs/lustre/obdclass/cl_page.c @@ -681,6 +681,19 @@ int cl_page_is_vmlocked(const struct lu_env *env, const struct cl_page *pg) } EXPORT_SYMBOL(cl_page_is_vmlocked); +void cl_page_touch(const struct lu_env *env, const struct cl_page *pg, + size_t to) +{ + const struct cl_page_slice *slice; + + list_for_each_entry(slice, &pg->cp_layers, cpl_linkage) { + if (slice->cpl_ops->cpo_page_touch) + (*slice->cpl_ops->cpo_page_touch)(env, slice, to); + } + +} +EXPORT_SYMBOL(cl_page_touch); + static enum cl_page_state cl_req_type_state(enum cl_req_type crt) { return crt == CRT_WRITE ? CPS_PAGEOUT : CPS_PAGEIN; diff --git a/fs/lustre/osc/osc_internal.h b/fs/lustre/osc/osc_internal.h index 1194033..3ba209f 100644 --- a/fs/lustre/osc/osc_internal.h +++ b/fs/lustre/osc/osc_internal.h @@ -143,6 +143,8 @@ int osc_quotactl(struct obd_device *unused, struct obd_export *exp, void osc_inc_unstable_pages(struct ptlrpc_request *req); void osc_dec_unstable_pages(struct ptlrpc_request *req); bool osc_over_unstable_soft_limit(struct client_obd *cli); +void osc_page_touch_at(const struct lu_env *env, struct cl_object *obj, + pgoff_t idx, size_t to); struct ldlm_lock *osc_obj_dlmlock_at_pgoff(const struct lu_env *env, struct osc_object *obj, diff --git a/fs/lustre/osc/osc_io.c b/fs/lustre/osc/osc_io.c index d8fa8cc..98726cd 100644 --- a/fs/lustre/osc/osc_io.c +++ b/fs/lustre/osc/osc_io.c @@ -216,14 +216,13 @@ int osc_io_submit(const struct lu_env *env, const struct cl_io_slice *ios, EXPORT_SYMBOL(osc_io_submit); /** - * This is called when a page is accessed within file in a way that creates - * new page, if one were missing (i.e., if there were a hole at that place in - * the file, or accessed page is beyond the current file size). + * This is called to update the attributes when modifying a specific page, + * both when making new pages and when doing updates to existing cached pages. * * Expand stripe KMS if necessary. */ -static void osc_page_touch_at(const struct lu_env *env, - struct cl_object *obj, pgoff_t idx, size_t to) +void osc_page_touch_at(const struct lu_env *env, struct cl_object *obj, + pgoff_t idx, size_t to) { struct lov_oinfo *loi = cl2osc(obj)->oo_oinfo; struct cl_attr *attr = &osc_env_info(env)->oti_attr; @@ -234,13 +233,6 @@ static void osc_page_touch_at(const struct lu_env *env, kms = cl_offset(obj, idx) + to; cl_object_attr_lock(obj); - /* - * XXX old code used - * - * ll_inode_size_lock(inode, 0); lov_stripe_lock(lsm); - * - * here - */ CDEBUG(D_INODE, "stripe KMS %sincreasing %llu->%llu %llu\n", kms > loi->loi_kms ? "" : "not ", loi->loi_kms, kms, loi->loi_lvb.lvb_size); diff --git a/fs/lustre/osc/osc_page.c b/fs/lustre/osc/osc_page.c index 96d1385..731fd27 100644 --- a/fs/lustre/osc/osc_page.c +++ b/fs/lustre/osc/osc_page.c @@ -228,11 +228,21 @@ static int osc_page_flush(const struct lu_env *env, return rc; } +static void osc_page_touch(const struct lu_env *env, + const struct cl_page_slice *slice, size_t to) +{ + struct osc_page *opg = cl2osc_page(slice); + struct cl_object *obj = opg->ops_cl.cpl_obj; + + osc_page_touch_at(env, obj, osc_index(opg), to); +} + static const struct cl_page_operations osc_page_ops = { .cpo_print = osc_page_print, .cpo_delete = osc_page_delete, .cpo_clip = osc_page_clip, - .cpo_flush = osc_page_flush + .cpo_flush = osc_page_flush, + .cpo_page_touch = osc_page_touch, }; int osc_page_init(const struct lu_env *env, struct cl_object *obj, From patchwork Mon Sep 30 18:56:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167289 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9D83613B1 for ; Mon, 30 Sep 2019 19:06:38 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 85B34224EF for ; Mon, 30 Sep 2019 19:06:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 85B34224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 1566E5E4D0B; Mon, 30 Sep 2019 12:01:02 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 446FC5C3A45 for ; Mon, 30 Sep 2019 11:57:38 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id B1C991005F8F; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id B06B8A9; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:22 -0400 Message-Id: <1569869810-23848-124-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 123/151] lustre: mdc: add CL_GETXATTR for Changelogs X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Sebastien Buisson Record GETXATTR events in Changelogs, and add a new changelog extension named changelog_ext_xattr to hold xattr name. A GETXATTR changlog entry is in the following form: 8 23GXATR 09:22:55.886793012 2017.07.27 0x0 t=[0x200000402:0x1:0x0] ef=0xf u=500:500 nid=10.128.11.159@tcp x=user.name0 Also, rename CL_XATTR type to CL_SETXATTR. By default, disable recording of GETXATTR events in Changelogs. WC-bug-id: https://jira.whamcloud.com/browse/LU-9727 Lustre-commit: c722371c1880 ("LU-9727 mdc: add CL_GETXATTR for Changelogs") Signed-off-by: Sebastien Buisson Reviewed-on: https://review.whamcloud.com/28251 Reviewed-by: Andreas Dilger Reviewed-by: Li Xi Signed-off-by: James Simmons --- fs/lustre/mdc/mdc_changelog.c | 3 +- include/uapi/linux/lustre/lustre_idl.h | 3 +- include/uapi/linux/lustre/lustre_user.h | 56 +++++++++++++++++++++++++++++---- 3 files changed, 54 insertions(+), 8 deletions(-) diff --git a/fs/lustre/mdc/mdc_changelog.c b/fs/lustre/mdc/mdc_changelog.c index 2aa4a8c..fb0de68 100644 --- a/fs/lustre/mdc/mdc_changelog.c +++ b/fs/lustre/mdc/mdc_changelog.c @@ -239,7 +239,8 @@ static int chlg_load(void *args) LLOG_F_EXT_EXTRA_FLAGS | LLOG_F_EXT_X_UIDGID | LLOG_F_EXT_X_NID | - LLOG_F_EXT_X_OMODE, + LLOG_F_EXT_X_OMODE | + LLOG_F_EXT_X_XATTR, NULL); if (rc) { CERROR("%s: fail to init llog handle: rc = %d\n", diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index 6294eaa..1d8c607 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -2541,6 +2541,7 @@ enum llog_flag { LLOG_F_EXT_X_UIDGID = 0x40, LLOG_F_EXT_X_NID = 0x80, LLOG_F_EXT_X_OMODE = 0x100, + LLOG_F_EXT_X_XATTR = 0x200, /* * Note: Flags covered by LLOG_F_EXT_MASK will be inherited from @@ -2550,7 +2551,7 @@ enum llog_flag { */ LLOG_F_EXT_MASK = LLOG_F_EXT_JOBID | LLOG_F_EXT_EXTRA_FLAGS | LLOG_F_EXT_X_UIDGID | LLOG_F_EXT_X_NID | - LLOG_F_EXT_X_OMODE, + LLOG_F_EXT_X_OMODE | LLOG_F_EXT_X_XATTR, }; /* On-disk header structure of each log object, stored in little endian order */ diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index 4e673cd..8a529a1 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -932,13 +932,15 @@ enum changelog_rec_type { CL_LAYOUT = 12, /* file layout/striping modified */ CL_TRUNC = 13, CL_SETATTR = 14, - CL_XATTR = 15, + CL_SETXATTR = 15, + CL_XATTR = CL_SETXATTR, /* Deprecated name */ CL_HSM = 16, /* HSM specific events, see flags */ CL_MTIME = 17, /* Precedence: setattr > mtime > ctime > atime */ CL_CTIME = 18, CL_ATIME = 19, CL_FLRW = 21, /* FLR: file was firstly written */ CL_RESYNC = 22, /* FLR: file was resync-ed */ + CL_GETXATTR = 23, CL_LAST }; @@ -948,7 +950,7 @@ static inline const char *changelog_type2str(int type) "MARK", "CREAT", "MKDIR", "HLINK", "SLINK", "MKNOD", "UNLNK", "RMDIR", "RENME", "RNMTO", "OPEN", "CLOSE", "LYOUT", "TRUNC", "SATTR", "XATTR", "HSM", "MTIME", "CTIME", "ATIME", "", - "FLRW", "RESYNC", + "FLRW", "RESYNC", "GXATTR", }; if (type >= 0 && type < CL_LAST) @@ -1056,7 +1058,8 @@ enum changelog_rec_extra_flags { CLFE_UIDGID = 0x0001, CLFE_NID = 0x0002, CLFE_OPEN = 0x0004, - CLFE_SUPPORTED = CLFE_UIDGID | CLFE_NID | CLFE_OPEN + CLFE_XATTR = 0x0008, + CLFE_SUPPORTED = CLFE_UIDGID | CLFE_NID | CLFE_OPEN | CLFE_XATTR }; enum changelog_send_flag { @@ -1081,6 +1084,8 @@ enum changelog_send_extra_flag { CHANGELOG_EXTRA_FLAG_NID = 0x02, /* Pack open mode into the changelog record */ CHANGELOG_EXTRA_FLAG_OMODE = 0x04, + /* Pack xattr name into the changelog record */ + CHANGELOG_EXTRA_FLAG_XATTR = 0x08, }; #define CR_MAXSIZE __ALIGN_KERNEL(2 * NAME_MAX + 2 + \ @@ -1137,6 +1142,11 @@ struct changelog_ext_openmode { __u32 cr_openflags; }; +/* Changelog extra extension to include xattr */ +struct changelog_ext_xattr { + char cr_xattr[XATTR_NAME_MAX + 1]; /**< zero-terminated string. */ +}; + /* Changelog extension to include additional flags. */ struct changelog_ext_extra_flags { __u64 cr_extra_flags; /* Additional CLFE_* flags */ @@ -1170,6 +1180,8 @@ static inline size_t changelog_rec_offset(enum changelog_rec_flags crf, size += sizeof(struct changelog_ext_nid); if (cref & CLFE_OPEN) size += sizeof(struct changelog_ext_openmode); + if (cref & CLFE_XATTR) + size += sizeof(struct changelog_ext_xattr); } return size; @@ -1271,6 +1283,23 @@ struct changelog_ext_openmode *changelog_rec_openmode( changelog_rec_offset(crf, cref)); } +/* The xattr name is the fourth extra extension */ +static inline +struct changelog_ext_xattr *changelog_rec_xattr( + const struct changelog_rec *rec) +{ + enum changelog_rec_flags crf = rec->cr_flags & + (CLF_VERSION | CLF_RENAME | CLF_JOBID | CLF_EXTRA_FLAGS); + enum changelog_rec_extra_flags cref = CLFE_INVALID; + + if (rec->cr_flags & CLF_EXTRA_FLAGS) + cref = changelog_rec_extra_flags(rec)->cr_extra_flags & + (CLFE_UIDGID | CLFE_NID | CLFE_OPEN); + + return (struct changelog_ext_xattr *)((char *)rec + + changelog_rec_offset(crf, cref)); +} + /* The name follows the rename, jobid and extra flags extns, if present */ static inline char *changelog_rec_name(struct changelog_rec *rec) { @@ -1320,6 +1349,7 @@ static inline void changelog_remap_rec(struct changelog_rec *rec, enum changelog_rec_flags crf_wanted, enum changelog_rec_extra_flags cref_want) { + char *xattr_mov = NULL; char *omd_mov = NULL; char *nid_mov = NULL; char *uidgid_mov = NULL; @@ -1345,18 +1375,24 @@ static inline void changelog_remap_rec(struct changelog_rec *rec, /* Locations of extensions in the remapped record */ if (rec->cr_flags & CLF_EXTRA_FLAGS) { + xattr_mov = (char *)rec + + changelog_rec_offset(crf_wanted & CLF_SUPPORTED, + cref_want & ~CLFE_XATTR); omd_mov = (char *)rec + changelog_rec_offset(crf_wanted & CLF_SUPPORTED, - cref_want & ~CLFE_OPEN); + cref_want & ~(CLFE_OPEN | + CLFE_XATTR)); nid_mov = (char *)rec + changelog_rec_offset(crf_wanted & CLF_SUPPORTED, cref_want & ~(CLFE_NID | - CLFE_OPEN)); + CLFE_OPEN | + CLFE_XATTR)); uidgid_mov = (char *)rec + changelog_rec_offset(crf_wanted & CLF_SUPPORTED, cref_want & ~(CLFE_UIDGID | CLFE_NID | - CLFE_OPEN)); + CLFE_OPEN | + CLFE_XATTR)); cref = changelog_rec_extra_flags(rec)->cr_extra_flags; } @@ -1377,6 +1413,10 @@ static inline void changelog_remap_rec(struct changelog_rec *rec, /* Move the extension fields to the desired positions */ if ((crf_wanted & CLF_EXTRA_FLAGS) && (rec->cr_flags & CLF_EXTRA_FLAGS)) { + if ((cref_want & CLFE_XATTR) && (cref & CLFE_XATTR)) + memmove(xattr_mov, changelog_rec_xattr(rec), + sizeof(struct changelog_ext_xattr)); + if ((cref_want & CLFE_OPEN) && (cref & CLFE_OPEN)) memmove(omd_mov, changelog_rec_openmode(rec), sizeof(struct changelog_ext_openmode)); @@ -1402,6 +1442,10 @@ static inline void changelog_remap_rec(struct changelog_rec *rec, sizeof(struct changelog_ext_rename)); /* Clear newly added fields */ + if (xattr_mov && (cref_want & CLFE_XATTR) && + !(cref & CLFE_XATTR)) + memset(xattr_mov, 0, sizeof(struct changelog_ext_xattr)); + if (omd_mov && (cref_want & CLFE_OPEN) && !(cref & CLFE_OPEN)) memset(omd_mov, 0, sizeof(struct changelog_ext_openmode)); From patchwork Mon Sep 30 18:56:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167291 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BAF9013B1 for ; Mon, 30 Sep 2019 19:06:44 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A3209224EF for ; Mon, 30 Sep 2019 19:06:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A3209224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 49CE95E4D46; Mon, 30 Sep 2019 12:01:05 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 9B22B5C3A45 for ; Mon, 30 Sep 2019 11:57:38 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id B47AE1005F95; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id B3851BB; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:23 -0400 Message-Id: <1569869810-23848-125-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 124/151] lustre: uapi: record denied OPEN in Changelogs X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Sebastien Buisson Record denied OPEN events in Changelogs, in the same format as successful OPEN events. Recording denied OPEN events is useful for security audit, in order to find out who tried to get access to some data. An NOPEN changlog entry is in the form: 4 24NOPEN 15:45:44.947406626 2017.08.31 0x2 t=[0x200000402:0x1:0x0] ef=0xf u=500:500 nid=10.128.11.158@tcp m=-w- By default, disable recording of NOPEN events in Changelogs. WC-bug-id: https://jira.whamcloud.com/browse/LU-9727 Lustre-commit: ccb6fe4b5994 ("LU-9727 lustre: record denied OPEN in Changelogs") Signed-off-by: Sebastien Buisson Reviewed-on: https://review.whamcloud.com/28812 Reviewed-by: Andreas Dilger Reviewed-by: Matthew S Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- include/uapi/linux/lustre/lustre_user.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index 8a529a1..29be243 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -941,6 +941,7 @@ enum changelog_rec_type { CL_FLRW = 21, /* FLR: file was firstly written */ CL_RESYNC = 22, /* FLR: file was resync-ed */ CL_GETXATTR = 23, + CL_DN_OPEN = 24, /* denied open */ CL_LAST }; @@ -950,7 +951,7 @@ static inline const char *changelog_type2str(int type) "MARK", "CREAT", "MKDIR", "HLINK", "SLINK", "MKNOD", "UNLNK", "RMDIR", "RENME", "RNMTO", "OPEN", "CLOSE", "LYOUT", "TRUNC", "SATTR", "XATTR", "HSM", "MTIME", "CTIME", "ATIME", "", - "FLRW", "RESYNC", "GXATTR", + "FLRW", "RESYNC", "GXATTR", "NOPEN", }; if (type >= 0 && type < CL_LAST) From patchwork Mon Sep 30 18:56:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167295 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 905021599 for ; Mon, 30 Sep 2019 19:06:54 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 78B26224EF for ; Mon, 30 Sep 2019 19:06:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 78B26224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8ABBE5E4D9A; Mon, 30 Sep 2019 12:01:08 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id DE91D5C3A45 for ; Mon, 30 Sep 2019 11:57:38 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id B80361005F96; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id B67D8B4; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:24 -0400 Message-Id: <1569869810-23848-126-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 125/151] lustre: llite: have ll_write_end to sync for DIO X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Vladimir Saveliev , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Vladimir Saveliev direct IO write uses buffered write for pages which could not be released. If not adjacent pages are not releasable, vio->u.write.vui_queue list becomes non-contiguos which makes page_list_sanity_check() to fail. Have ll_write_commit to do vvp_io_write_commit() when it is called in course of direct IO. WC-bug-id: https://jira.whamcloud.com/browse/LU-10356 Cray-bug-id: MRP-4415 Lustre-commit: 6ea9171769db ("LU-10356 llite: have ll_write_end to sync for DIO") Signed-off-by: Vladimir Saveliev Reviewed-on: https://review.whamcloud.com/30659 Reviewed-by: Jinshan Xiong Reviewed-by: Alexey Lyashkov Reviewed-by: Andrew Perepechko Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/rw26.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/fs/lustre/llite/rw26.c b/fs/lustre/llite/rw26.c index 2baab10..f5c1479 100644 --- a/fs/lustre/llite/rw26.c +++ b/fs/lustre/llite/rw26.c @@ -684,8 +684,7 @@ static int ll_write_end(struct file *file, struct address_space *mapping, unplug = true; } - if (unplug || - file->f_flags & O_SYNC || IS_SYNC(file_inode(file))) + if (unplug || io->u.ci_wr.wr_sync) result = vvp_io_write_commit(env, io); if (result < 0) From patchwork Mon Sep 30 18:56:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167331 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BB45713B1 for ; Mon, 30 Sep 2019 19:08:12 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A260B224EF for ; Mon, 30 Sep 2019 19:08:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A260B224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id B6AB2602DC5; Mon, 30 Sep 2019 12:01:39 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 2AD1F5C3B82 for ; Mon, 30 Sep 2019 11:57:39 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id BAF7E1005F97; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id B9E96B5; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:25 -0400 Message-Id: <1569869810-23848-127-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 126/151] lustre: obd: add check to obd_statfs X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Boyko , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Alexander Boyko The race could happened between mount and lctl get_param. Because procfs files are ready before a full obd initialization. For example: 3372:0:(dt_object.h:2509:dt_statfs()) ASSERTION( dev ) 3372:0:(dt_object.h:2509:dt_statfs()) LBUG Pid: 3372, comm: lctl Call Trace: libcfs_call_trace+0x4e/0x60[libcfs] lbug_with_loc+0x4c/0xb0[libcfs] tgt_statfs_internal+0x2ea/0x350[ptlrpc] ofd_statfs+0x66/0x470 [ofd] lprocfs_filesfree_seq_show+0xf6/0x520 [obdclass] ofd_filesfree_seq_show+0x12/0x20 [ofd] The patch adds a check of completed obd_setup to obd_statfs(). WC-bug-id: https://jira.whamcloud.com/browse/LU-10650 ray-bug-id: LUS-2665 Lustre-commit: 4f40429775c4 ("LU-10650 obd: add check to obd_statfs") Signed-off-by: Alexander Boyko Reviewed-on: https://review.whamcloud.com/31243 Reviewed-by: Alexey Lyashkov Reviewed-by: Andriy Skulysh Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/obd_class.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/fs/lustre/include/obd_class.h b/fs/lustre/include/obd_class.h index dd07ec5..0bbf777 100644 --- a/fs/lustre/include/obd_class.h +++ b/fs/lustre/include/obd_class.h @@ -924,6 +924,10 @@ static inline int obd_statfs(const struct lu_env *env, struct obd_export *exp, if (!obd) return -EINVAL; + rc = obd_check_dev_active(obd); + if (rc) + return rc; + if (!obd->obd_type || !obd->obd_type->typ_dt_ops->statfs) { CERROR("%s: no %s operation\n", obd->obd_name, __func__); return -EOPNOTSUPP; From patchwork Mon Sep 30 18:56:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167313 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3CC3E13B1 for ; Mon, 30 Sep 2019 19:07:39 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 25130224EF for ; Mon, 30 Sep 2019 19:07:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 25130224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id E806C5E4F02; Mon, 30 Sep 2019 12:01:25 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 6E3EB5C3C9C for ; Mon, 30 Sep 2019 11:57:39 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id BE4691005F98; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id BCFC6A9; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:26 -0400 Message-Id: <1569869810-23848-128-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 127/151] lustre: obd: fix statfs handling X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: James Simmons , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" The comment 'use the value of cfs_time_current + HZ' for obd_statfs() and obd_statfs_async() needs to updated to the time64_t case. Simplify llite_statfs_internal() handling by calculating max_age inside of llite_statfs_internal(). This makes the code cleaner. WC-bug-id: https://jira.whamcloud.com/browse/LU-10570 Lustre-commit: 87577f4988c1 ("LU-10570 obd: fix statfs handling") Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/31158 Reviewed-by: Bob Glossman Reviewed-by: Dmitry Eremin Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/obd_class.h | 8 ++++---- fs/lustre/llite/llite_internal.h | 2 +- fs/lustre/llite/llite_lib.c | 9 +++++---- fs/lustre/llite/lproc_llite.c | 24 ++++++------------------ 4 files changed, 16 insertions(+), 27 deletions(-) diff --git a/fs/lustre/include/obd_class.h b/fs/lustre/include/obd_class.h index 0bbf777..9e07853 100644 --- a/fs/lustre/include/obd_class.h +++ b/fs/lustre/include/obd_class.h @@ -868,9 +868,9 @@ static inline int obd_destroy_export(struct obd_export *exp) } /* - * @max_age is the oldest time in jiffies that we accept using a cached data. + * @max_age is the oldest time in seconds that we accept using a cached data. * If the cache is older than @max_age we will get a new value from the - * target. Use a value of "jiffies + HZ" to guarantee freshness. + * target. Use a value of 'ktime_get_seconds() + X' to guarantee freshness. */ static inline int obd_statfs_async(struct obd_export *exp, struct obd_info *oinfo, @@ -910,9 +910,9 @@ static inline int obd_statfs_async(struct obd_export *exp, } /* - * @max_age is the oldest time in jiffies that we accept using a cached data. + * @max_age is the oldest time in seconds that we accept using a cached data. * If the cache is older than @max_age we will get a new value from the - * target. Use a value of "jiffies + HZ" to guarantee freshness. + * target. Use a value of 'ktime_get_seconds() + X' to guarantee freshness. */ static inline int obd_statfs(const struct lu_env *env, struct obd_export *exp, struct obd_statfs *osfs, time64_t max_age, diff --git a/fs/lustre/llite/llite_internal.h b/fs/lustre/llite/llite_internal.h index fc4c4db..f23cf65 100644 --- a/fs/lustre/llite/llite_internal.h +++ b/fs/lustre/llite/llite_internal.h @@ -878,7 +878,7 @@ int ll_setattr_raw(struct dentry *dentry, struct iattr *attr, int ll_setattr(struct dentry *de, struct iattr *attr); int ll_statfs(struct dentry *de, struct kstatfs *sfs); int ll_statfs_internal(struct ll_sb_info *sbi, struct obd_statfs *osfs, - time64_t max_age, __u32 flags); + u32 flags); int ll_update_inode(struct inode *inode, struct lustre_md *md); void ll_update_inode_flags(struct inode *inode, int ext_flags); int ll_read_inode2(struct inode *inode, void *opaque); diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index efd1093..758f856 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -1716,11 +1716,14 @@ int ll_setattr(struct dentry *de, struct iattr *attr) } int ll_statfs_internal(struct ll_sb_info *sbi, struct obd_statfs *osfs, - time64_t max_age, u32 flags) + u32 flags) { struct obd_statfs obd_osfs; + time64_t max_age; int rc; + max_age = ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS; + rc = obd_statfs(NULL, sbi->ll_md_exp, osfs, max_age, flags); if (rc) { CERROR("md_statfs fails: rc = %d\n", rc); @@ -1774,9 +1777,7 @@ int ll_statfs(struct dentry *de, struct kstatfs *sfs) ll_stats_ops_tally(ll_s2sbi(sb), LPROC_LL_STAFS, 1); /* Some amount of caching on the client is allowed */ - rc = ll_statfs_internal(ll_s2sbi(sb), &osfs, - ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, - 0); + rc = ll_statfs_internal(ll_s2sbi(sb), &osfs, 0); if (rc) return rc; diff --git a/fs/lustre/llite/lproc_llite.c b/fs/lustre/llite/lproc_llite.c index 9173d94..b69e5d7 100644 --- a/fs/lustre/llite/lproc_llite.c +++ b/fs/lustre/llite/lproc_llite.c @@ -140,9 +140,7 @@ static ssize_t blocksize_show(struct kobject *kobj, struct attribute *attr, struct obd_statfs osfs; int rc; - rc = ll_statfs_internal(sbi, &osfs, - ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, - OBD_STATFS_NODELAY); + rc = ll_statfs_internal(sbi, &osfs, OBD_STATFS_NODELAY); if (rc) return rc; @@ -192,9 +190,7 @@ static ssize_t kbytestotal_show(struct kobject *kobj, struct attribute *attr, u64 result; int rc; - rc = ll_statfs_internal(sbi, &osfs, - ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, - OBD_STATFS_NODELAY); + rc = ll_statfs_internal(sbi, &osfs, OBD_STATFS_NODELAY); if (rc) return rc; @@ -218,9 +214,7 @@ static ssize_t kbytesfree_show(struct kobject *kobj, struct attribute *attr, u64 result; int rc; - rc = ll_statfs_internal(sbi, &osfs, - ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, - OBD_STATFS_NODELAY); + rc = ll_statfs_internal(sbi, &osfs, OBD_STATFS_NODELAY); if (rc) return rc; @@ -244,9 +238,7 @@ static ssize_t kbytesavail_show(struct kobject *kobj, struct attribute *attr, u64 result; int rc; - rc = ll_statfs_internal(sbi, &osfs, - ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, - OBD_STATFS_NODELAY); + rc = ll_statfs_internal(sbi, &osfs, OBD_STATFS_NODELAY); if (rc) return rc; @@ -268,9 +260,7 @@ static ssize_t filestotal_show(struct kobject *kobj, struct attribute *attr, struct obd_statfs osfs; int rc; - rc = ll_statfs_internal(sbi, &osfs, - ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, - OBD_STATFS_NODELAY); + rc = ll_statfs_internal(sbi, &osfs, OBD_STATFS_NODELAY); if (rc) return rc; @@ -286,9 +276,7 @@ static ssize_t filesfree_show(struct kobject *kobj, struct attribute *attr, struct obd_statfs osfs; int rc; - rc = ll_statfs_internal(sbi, &osfs, - ktime_get_seconds() - OBD_STATFS_CACHE_SECONDS, - OBD_STATFS_NODELAY); + rc = ll_statfs_internal(sbi, &osfs, OBD_STATFS_NODELAY); if (rc) return rc; From patchwork Mon Sep 30 18:56:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167335 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CE2E313B1 for ; Mon, 30 Sep 2019 19:08:20 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B62DE224EF for ; Mon, 30 Sep 2019 19:08:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B62DE224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 90B62602E3D; Mon, 30 Sep 2019 12:01:43 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id C39465C3BBF for ; Mon, 30 Sep 2019 11:57:39 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id C11691005F99; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id C0021BB; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:27 -0400 Message-Id: <1569869810-23848-129-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 128/151] lustre: dom: support DATA_VERSION IO type X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mikhail Pershin , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mikhail Pershin add support for DATA_VERSION IO type, return from MDT data version and layout version if requested by CLIO. Also ensure that version is changed on punch and write operations. This fixes HSM archive with DOM files. WC-bug-id: https://jira.whamcloud.com/browse/LU-10318 Lustre-commit: 1e7fc14bbf48 ("LU-10318 dom: support DATA_VERSION IO type") Signed-off-by: Mikhail Pershin Reviewed-on: https://review.whamcloud.com/30449 Reviewed-by: John L. Hammond Reviewed-by: Jinshan Xiong Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/mdc/mdc_dev.c | 119 ++++++++++++++++++++++++++++++++- fs/lustre/mdc/mdc_request.c | 1 - fs/lustre/ptlrpc/pack_generic.c | 4 +- fs/lustre/ptlrpc/wiretest.c | 85 ++++++++++++++++------- include/uapi/linux/lustre/lustre_idl.h | 4 +- 5 files changed, 181 insertions(+), 32 deletions(-) diff --git a/fs/lustre/mdc/mdc_dev.c b/fs/lustre/mdc/mdc_dev.c index 2de849a..22749ea 100644 --- a/fs/lustre/mdc/mdc_dev.c +++ b/fs/lustre/mdc/mdc_dev.c @@ -1114,6 +1114,120 @@ int mdc_io_fsync_start(const struct lu_env *env, return result; } +struct mdc_data_version_args { + struct osc_io *dva_oio; +}; + +static int +mdc_data_version_interpret(const struct lu_env *env, struct ptlrpc_request *req, + void *arg, int rc) +{ + struct mdc_data_version_args *dva = arg; + struct osc_io *oio = dva->dva_oio; + const struct mdt_body *body; + + if (rc < 0) + goto out; + + body = req_capsule_server_get(&req->rq_pill, &RMF_MDT_BODY); + if (!body) { + rc = -EPROTO; + goto out; + } + + /* Prepare OBDO from mdt_body for CLIO */ + oio->oi_oa.o_valid = body->mbo_valid; + oio->oi_oa.o_flags = body->mbo_flags; + oio->oi_oa.o_data_version = body->mbo_version; + oio->oi_oa.o_layout_version = body->mbo_layout_gen; + +out: + oio->oi_cbarg.opc_rc = rc; + complete(&oio->oi_cbarg.opc_sync); + return 0; +} + +static int mdc_io_data_version_start(const struct lu_env *env, + const struct cl_io_slice *slice) +{ + struct cl_data_version_io *dv = &slice->cis_io->u.ci_data_version; + struct osc_io *oio = cl2osc_io(env, slice); + struct osc_async_cbargs *cbargs = &oio->oi_cbarg; + struct osc_object *obj = cl2osc(slice->cis_obj); + struct obd_export *exp = osc_export(obj); + struct ptlrpc_request *req; + struct mdt_body *body; + struct mdc_data_version_args *dva; + int rc; + + memset(&oio->oi_oa, 0, sizeof(oio->oi_oa)); + oio->oi_oa.o_oi.oi_fid = *lu_object_fid(osc2lu(obj)); + oio->oi_oa.o_valid = OBD_MD_FLID; + + init_completion(&cbargs->opc_sync); + + req = ptlrpc_request_alloc(class_exp2cliimp(exp), &RQF_MDS_GETATTR); + if (!req) + return -ENOMEM; + + rc = ptlrpc_request_pack(req, LUSTRE_MDS_VERSION, MDS_GETATTR); + if (rc < 0) { + ptlrpc_request_free(req); + return rc; + } + + body = req_capsule_client_get(&req->rq_pill, &RMF_MDT_BODY); + body->mbo_fid1 = *lu_object_fid(osc2lu(obj)); + body->mbo_valid = OBD_MD_FLID; + /* Indicate that data version is needed */ + body->mbo_valid |= OBD_MD_FLDATAVERSION; + body->mbo_flags = 0; + + if (dv->dv_flags & (LL_DV_RD_FLUSH | LL_DV_WR_FLUSH)) { + body->mbo_valid |= OBD_MD_FLFLAGS; + body->mbo_flags |= OBD_FL_SRVLOCK; + if (dv->dv_flags & LL_DV_WR_FLUSH) + body->mbo_flags |= OBD_FL_FLUSH; + } + + req_capsule_set_size(&req->rq_pill, &RMF_ACL, RCL_SERVER, 0); + req_capsule_set_size(&req->rq_pill, &RMF_MDT_MD, RCL_SERVER, 0); + ptlrpc_request_set_replen(req); + + req->rq_interpret_reply = mdc_data_version_interpret; + BUILD_BUG_ON(sizeof(*dva) > sizeof(req->rq_async_args)); + dva = ptlrpc_req_async_args(req); + dva->dva_oio = oio; + + ptlrpcd_add_req(req); + + return 0; +} + +static void mdc_io_data_version_end(const struct lu_env *env, + const struct cl_io_slice *slice) +{ + struct cl_data_version_io *dv = &slice->cis_io->u.ci_data_version; + struct osc_io *oio = cl2osc_io(env, slice); + struct osc_async_cbargs *cbargs = &oio->oi_cbarg; + + wait_for_completion(&cbargs->opc_sync); + + if (cbargs->opc_rc != 0) { + slice->cis_io->ci_result = cbargs->opc_rc; + } else { + slice->cis_io->ci_result = 0; + if (!(oio->oi_oa.o_valid & + (OBD_MD_LAYOUT_VERSION | OBD_MD_FLDATAVERSION))) + slice->cis_io->ci_result = -ENOTSUPP; + + if (oio->oi_oa.o_valid & OBD_MD_LAYOUT_VERSION) + dv->dv_layout_version = oio->oi_oa.o_layout_version; + if (oio->oi_oa.o_valid & OBD_MD_FLDATAVERSION) + dv->dv_data_version = oio->oi_oa.o_data_version; + } +} + static struct cl_io_operations mdc_io_ops = { .op = { [CIT_READ] = { @@ -1133,10 +1247,9 @@ int mdc_io_fsync_start(const struct lu_env *env, .cio_start = mdc_io_setattr_start, .cio_end = osc_io_setattr_end, }, - /* no support for data version so far */ [CIT_DATA_VERSION] = { - .cio_start = NULL, - .cio_end = NULL, + .cio_start = mdc_io_data_version_start, + .cio_end = mdc_io_data_version_end, }, [CIT_FAULT] = { .cio_iter_init = osc_io_iter_init, diff --git a/fs/lustre/mdc/mdc_request.c b/fs/lustre/mdc/mdc_request.c index e344e78..408bf19 100644 --- a/fs/lustre/mdc/mdc_request.c +++ b/fs/lustre/mdc/mdc_request.c @@ -722,7 +722,6 @@ int mdc_set_open_replay_data(struct obd_export *exp, } rec->cr_fid2 = body->mbo_fid1; - rec->cr_ioepoch = body->mbo_ioepoch; rec->cr_old_handle.cookie = body->mbo_handle.cookie; open_req->rq_replay_cb = mdc_replay_open; if (!fid_is_sane(&body->mbo_fid1)) { diff --git a/fs/lustre/ptlrpc/pack_generic.c b/fs/lustre/ptlrpc/pack_generic.c index 1face33..f7afc4c 100644 --- a/fs/lustre/ptlrpc/pack_generic.c +++ b/fs/lustre/ptlrpc/pack_generic.c @@ -1741,7 +1741,7 @@ void lustre_swab_mdt_body(struct mdt_body *b) __swab64s(&b->mbo_atime); __swab64s(&b->mbo_ctime); __swab64s(&b->mbo_blocks); - __swab64s(&b->mbo_ioepoch); + __swab64s(&b->mbo_version); __swab64s(&b->mbo_t_state); __swab32s(&b->mbo_fsuid); __swab32s(&b->mbo_fsgid); @@ -1752,7 +1752,7 @@ void lustre_swab_mdt_body(struct mdt_body *b) __swab32s(&b->mbo_flags); __swab32s(&b->mbo_rdev); __swab32s(&b->mbo_nlink); - BUILD_BUG_ON(offsetof(typeof(*b), mbo_unused2) == 0); + __swab32s(&b->mbo_layout_gen); __swab32s(&b->mbo_suppgid); __swab32s(&b->mbo_eadatasize); __swab32s(&b->mbo_aclsize); diff --git a/fs/lustre/ptlrpc/wiretest.c b/fs/lustre/ptlrpc/wiretest.c index 70ed2cd..68eb33c 100644 --- a/fs/lustre/ptlrpc/wiretest.c +++ b/fs/lustre/ptlrpc/wiretest.c @@ -589,10 +589,10 @@ void lustre_assert_wire_constants(void) (long long)(int)offsetof(struct lustre_msg_v2, lm_buflens[0])); LASSERTF((int)sizeof(((struct lustre_msg_v2 *)0)->lm_buflens[0]) == 4, "found %lld\n", (long long)(int)sizeof(((struct lustre_msg_v2 *)0)->lm_buflens[0])); - LASSERTF(LUSTRE_MSG_MAGIC_V2 == 0x0BD00BD3, "found 0x%.8x\n", - LUSTRE_MSG_MAGIC_V2); - LASSERTF(LUSTRE_MSG_MAGIC_V2_SWABBED == 0xD30BD00B, "found 0x%.8x\n", - LUSTRE_MSG_MAGIC_V2_SWABBED); + LASSERTF(LUSTRE_MSG_MAGIC_V2 == 0x0bd00bd3UL, "found 0x%.8xUL\n", + (unsigned int)LUSTRE_MSG_MAGIC_V2); + LASSERTF(LUSTRE_MSG_MAGIC_V2_SWABBED == 0xd30bd00bUL, "found 0x%.8xUL\n", + (unsigned int)LUSTRE_MSG_MAGIC_V2_SWABBED); /* Checks for struct ptlrpc_body */ LASSERTF((int)sizeof(struct ptlrpc_body_v3) == 184, "found %lld\n", @@ -807,22 +807,22 @@ void lustre_assert_wire_constants(void) (long long)DLM_REPLY_REC_OFF); LASSERTF(MSG_PTLRPC_HEADER_OFF == 31, "found %lld\n", (long long)MSG_PTLRPC_HEADER_OFF); - LASSERTF(PTLRPC_MSG_VERSION == 0x00000003, "found 0x%.8x\n", - PTLRPC_MSG_VERSION); - LASSERTF(LUSTRE_VERSION_MASK == 0xffff0000, "found 0x%.8x\n", - LUSTRE_VERSION_MASK); - LASSERTF(LUSTRE_OBD_VERSION == 0x00010000, "found 0x%.8x\n", - LUSTRE_OBD_VERSION); - LASSERTF(LUSTRE_MDS_VERSION == 0x00020000, "found 0x%.8x\n", - LUSTRE_MDS_VERSION); - LASSERTF(LUSTRE_OST_VERSION == 0x00030000, "found 0x%.8x\n", - LUSTRE_OST_VERSION); - LASSERTF(LUSTRE_DLM_VERSION == 0x00040000, "found 0x%.8x\n", - LUSTRE_DLM_VERSION); - LASSERTF(LUSTRE_LOG_VERSION == 0x00050000, "found 0x%.8x\n", - LUSTRE_LOG_VERSION); - LASSERTF(LUSTRE_MGS_VERSION == 0x00060000, "found 0x%.8x\n", - LUSTRE_MGS_VERSION); + LASSERTF(PTLRPC_MSG_VERSION == 0x00000003UL, "found 0x%.8xUL\n", + (unsigned int)PTLRPC_MSG_VERSION); + LASSERTF(LUSTRE_VERSION_MASK == 0xffff0000UL, "found 0x%.8xUL\n", + (unsigned int)LUSTRE_VERSION_MASK); + LASSERTF(LUSTRE_OBD_VERSION == 0x00010000UL, "found 0x%.8xUL\n", + (unsigned int)LUSTRE_OBD_VERSION); + LASSERTF(LUSTRE_MDS_VERSION == 0x00020000UL, "found 0x%.8xUL\n", + (unsigned int)LUSTRE_MDS_VERSION); + LASSERTF(LUSTRE_OST_VERSION == 0x00030000UL, "found 0x%.8xUL\n", + (unsigned int)LUSTRE_OST_VERSION); + LASSERTF(LUSTRE_DLM_VERSION == 0x00040000UL, "found 0x%.8xUL\n", + (unsigned int)LUSTRE_DLM_VERSION); + LASSERTF(LUSTRE_LOG_VERSION == 0x00050000UL, "found 0x%.8xUL\n", + (unsigned int)LUSTRE_LOG_VERSION); + LASSERTF(LUSTRE_MGS_VERSION == 0x00060000UL, "found 0x%.8xUL\n", + (unsigned int)LUSTRE_MGS_VERSION); LASSERTF(MSGHDR_AT_SUPPORT == 1, "found %lld\n", (long long)MSGHDR_AT_SUPPORT); LASSERTF(MSGHDR_CKSUM_INCOMPAT18 == 2, "found %lld\n", @@ -1866,6 +1866,37 @@ void lustre_assert_wire_constants(void) LASSERTF((int)sizeof(((struct ll_fid *)0)->f_type) == 4, "found %lld\n", (long long)(int)sizeof(((struct ll_fid *)0)->f_type)); + LASSERTF(MDS_CHECK_SPLIT == 0x00000001UL, "found 0x%.8xUL\n", + (unsigned int)MDS_CHECK_SPLIT); + LASSERTF(MDS_CROSS_REF == 0x00000002UL, "found 0x%.8xUL\n", + (unsigned int)MDS_CROSS_REF); + LASSERTF(MDS_VTX_BYPASS == 0x00000004UL, "found 0x%.8xUL\n", + (unsigned int)MDS_VTX_BYPASS); + LASSERTF(MDS_PERM_BYPASS == 0x00000008UL, "found 0x%.8xUL\n", + (unsigned int)MDS_PERM_BYPASS); + LASSERTF(MDS_QUOTA_IGNORE == 0x00000020UL, "found 0x%.8xUL\n", + (unsigned int)MDS_QUOTA_IGNORE); + LASSERTF(MDS_KEEP_ORPHAN == 0x00000080UL, "found 0x%.8xUL\n", + (unsigned int)MDS_KEEP_ORPHAN); + LASSERTF(MDS_RECOV_OPEN == 0x00000100UL, "found 0x%.8xUL\n", + (unsigned int)MDS_RECOV_OPEN); + LASSERTF(MDS_DATA_MODIFIED == 0x00000200UL, "found 0x%.8xUL\n", + (unsigned int)MDS_DATA_MODIFIED); + LASSERTF(MDS_CREATE_VOLATILE == 0x00000400UL, "found 0x%.8xUL\n", + (unsigned int)MDS_CREATE_VOLATILE); + LASSERTF(MDS_OWNEROVERRIDE == 0x00000800UL, "found 0x%.8xUL\n", + (unsigned int)MDS_OWNEROVERRIDE); + LASSERTF(MDS_HSM_RELEASE == 0x00001000UL, "found 0x%.8xUL\n", + (unsigned int)MDS_HSM_RELEASE); + LASSERTF(MDS_CLOSE_LAYOUT_SWAP == 0x00004000UL, "found 0x%.8xUL\n", + (unsigned int)MDS_CLOSE_LAYOUT_SWAP); + LASSERTF(MDS_CLOSE_LAYOUT_MERGE == 0x00008000UL, "found 0x%.8xUL\n", + (unsigned int)MDS_CLOSE_LAYOUT_MERGE); + LASSERTF(MDS_CLOSE_RESYNC_DONE == 0x00010000UL, "found 0x%.8xUL\n", + (unsigned int)MDS_CLOSE_RESYNC_DONE); + LASSERTF(MDS_CLOSE_LAYOUT_SPLIT == 0x00020000UL, "found 0x%.8xUL\n", + (unsigned int)MDS_CLOSE_LAYOUT_SPLIT); + /* Checks for struct mdt_body */ LASSERTF((int)sizeof(struct mdt_body) == 216, "found %lld\n", (long long)(int)sizeof(struct mdt_body)); @@ -1905,6 +1936,10 @@ void lustre_assert_wire_constants(void) (long long)(int)offsetof(struct mdt_body, mbo_blocks)); LASSERTF((int)sizeof(((struct mdt_body *)0)->mbo_blocks) == 8, "found %lld\n", (long long)(int)sizeof(((struct mdt_body *)0)->mbo_blocks)); + LASSERTF((int)offsetof(struct mdt_body, mbo_version) == 88, "found %lld\n", + (long long)(int)offsetof(struct mdt_body, mbo_version)); + LASSERTF((int)sizeof(((struct mdt_body *)0)->mbo_version) == 8, "found %lld\n", + (long long)(int)sizeof(((struct mdt_body *)0)->mbo_version)); LASSERTF((int)offsetof(struct mdt_body, mbo_t_state) == 96, "found %lld\n", (long long)(int)offsetof(struct mdt_body, mbo_t_state)); LASSERTF((int)sizeof(((struct mdt_body *)0)->mbo_t_state) == 8, @@ -1946,10 +1981,10 @@ void lustre_assert_wire_constants(void) (long long)(int)offsetof(struct mdt_body, mbo_nlink)); LASSERTF((int)sizeof(((struct mdt_body *)0)->mbo_nlink) == 4, "found %lld\n", (long long)(int)sizeof(((struct mdt_body *)0)->mbo_nlink)); - LASSERTF((int)offsetof(struct mdt_body, mbo_unused2) == 140, "found %lld\n", - (long long)(int)offsetof(struct mdt_body, mbo_unused2)); - LASSERTF((int)sizeof(((struct mdt_body *)0)->mbo_unused2) == 4, "found %lld\n", - (long long)(int)sizeof(((struct mdt_body *)0)->mbo_unused2)); + LASSERTF((int)offsetof(struct mdt_body, mbo_layout_gen) == 140, "found %lld\n", + (long long)(int)offsetof(struct mdt_body, mbo_layout_gen)); + LASSERTF((int)sizeof(((struct mdt_body *)0)->mbo_layout_gen) == 4, "found %lld\n", + (long long)(int)sizeof(((struct mdt_body *)0)->mbo_layout_gen)); LASSERTF((int)offsetof(struct mdt_body, mbo_suppgid) == 144, "found %lld\n", (long long)(int)offsetof(struct mdt_body, mbo_suppgid)); LASSERTF((int)sizeof(((struct mdt_body *)0)->mbo_suppgid) == 4, "found %lld\n", @@ -4398,12 +4433,14 @@ void lustre_assert_wire_constants(void) (long long)(int)offsetof(struct hsm_request, hr_data_len)); LASSERTF((int)sizeof(((struct hsm_request *)0)->hr_data_len) == 4, "found %lld\n", (long long)(int)sizeof(((struct hsm_request *)0)->hr_data_len)); +#ifdef HAVE_SERVER_SUPPORT LASSERTF(HSM_FORCE_ACTION == 0x00000001UL, "found 0x%.8xUL\n", (unsigned int)HSM_FORCE_ACTION); LASSERTF(HSM_GHOST_COPY == 0x00000002UL, "found 0x%.8xUL\n", (unsigned int)HSM_GHOST_COPY); /* Checks for struct hsm_user_request */ +#endif LASSERTF((int)sizeof(struct hsm_user_request) == 24, "found %lld\n", (long long)(int)sizeof(struct hsm_user_request)); LASSERTF((int)offsetof(struct hsm_user_request, hur_request) == 0, "found %lld\n", diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index 1d8c607..6ef7d45 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -1539,7 +1539,7 @@ struct mdt_body { __s64 mbo_atime; __s64 mbo_ctime; __u64 mbo_blocks; /* XID, in the case of MDS_READPAGE */ - __u64 mbo_ioepoch; + __u64 mbo_version; /* was mbo_ioepoch before 2.11 */ __u64 mbo_t_state; /* transient file state defined in * enum md_transient_state * was "ino" until 2.4.0 @@ -1553,7 +1553,7 @@ struct mdt_body { __u32 mbo_flags; /* LUSTRE_*_FL file attributes */ __u32 mbo_rdev; __u32 mbo_nlink; /* #bytes to read in the case of MDS_READPAGE */ - __u32 mbo_unused2; /* was "generation" until 2.4.0 */ + __u32 mbo_layout_gen; /* was "generation" until 2.4.0 */ __u32 mbo_suppgid; __u32 mbo_eadatasize; __u32 mbo_aclsize; From patchwork Mon Sep 30 18:56:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167299 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BCDD913B1 for ; Mon, 30 Sep 2019 19:07:03 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A26F3224EF for ; Mon, 30 Sep 2019 19:07:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A26F3224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 9C7BC5E4DE9; Mon, 30 Sep 2019 12:01:11 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 25A125C38DF for ; Mon, 30 Sep 2019 11:57:40 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id C4B321005F9A; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id C3777B4; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:28 -0400 Message-Id: <1569869810-23848-130-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 129/151] lnet: fix contiguous range support X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Kit Westneat This patch fixes the contiguous range check to allow the addition of multiple "full" ([0-255]) ranges. As part of this change, is_contiguous and find_min_max are combined as they were always called together and the logic is fairly similar. This also removes the multiple range expression support, since it was broken. WC-bug-id: https://jira.whamcloud.com/browse/LU-LU-8912 Lustre-commit: eac95a6587af ("LU-8912 nodemap: fix contiguous range support") Signed-off-by: Kit Westneat Signed-off-by: Emoly Liu Reviewed-on: https://review.whamcloud.com/24397 Reviewed-by: Sebastien Buisson Reviewed-by: Andreas Dilger Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- include/uapi/linux/lnet/nidstr.h | 8 +- net/lnet/lnet/nidstrings.c | 324 ++++++++++++++++----------------------- 2 files changed, 133 insertions(+), 199 deletions(-) diff --git a/include/uapi/linux/lnet/nidstr.h b/include/uapi/linux/lnet/nidstr.h index 8bff1fd..43ec232 100644 --- a/include/uapi/linux/lnet/nidstr.h +++ b/include/uapi/linux/lnet/nidstr.h @@ -97,9 +97,8 @@ static inline char *libcfs_nid2str(lnet_nid_t nid) int cfs_ip_addr_parse(char *str, int len, struct list_head *list); int cfs_ip_addr_match(__u32 addr, struct list_head *list); -bool cfs_nidrange_is_contiguous(struct list_head *nidlist); -void cfs_nidrange_find_min_max(struct list_head *nidlist, char *min_nid, - char *max_nid, size_t nidstr_length); +int cfs_nidrange_find_min_max(struct list_head *nidlist, char *min_nid, + char *max_nid, size_t nidstr_length); struct netstrfns { __u32 nf_type; @@ -112,8 +111,7 @@ struct netstrfns { int (*nf_print_addrlist)(char *buffer, int count, struct list_head *list); int (*nf_match_addr)(__u32 addr, struct list_head *list); - bool (*nf_is_contiguous)(struct list_head *nidlist); - void (*nf_min_max)(struct list_head *nidlist, __u32 *min_nid, + int (*nf_min_max)(struct list_head *nidlist, __u32 *min_nid, __u32 *max_nid); }; diff --git a/net/lnet/lnet/nidstrings.c b/net/lnet/lnet/nidstrings.c index b84708d..b4e38e5 100644 --- a/net/lnet/lnet/nidstrings.c +++ b/net/lnet/lnet/nidstrings.c @@ -456,38 +456,64 @@ int cfs_print_nidlist(char *buffer, int count, struct list_head *nidlist) * numeric address range * * @ar - * @min_nid - * @max_nid + * @min_nid *min_nid __u32 representation of min NID + * @max_nid *max_nid __u32 representation of max NID + * + * Return: -EINVAL unsupported LNET range + * -ERANGE non-contiguous LNET range */ -static void cfs_ip_ar_min_max(struct addrrange *ar, u32 *min_nid, - u32 *max_nid) +static int cfs_ip_ar_min_max(struct addrrange *ar, u32 *min_nid, + u32 *max_nid) { - struct cfs_expr_list *el; - struct cfs_range_expr *re; - u32 tmp_ip_addr = 0; + struct cfs_expr_list *expr_list; + struct cfs_range_expr *range; unsigned int min_ip[4] = { 0 }; unsigned int max_ip[4] = { 0 }; - int re_count = 0; + int cur_octet = 0; + bool expect_full_octet = false; + + list_for_each_entry(expr_list, &ar->ar_numaddr_ranges, el_link) { + int re_count = 0; + + list_for_each_entry(range, &expr_list->el_exprs, re_link) { + /* XXX: add support for multiple & non-contig. re's */ + if (re_count > 0) + return -EINVAL; + + /* if a previous octet was ranged, then all remaining + * octets must be full for contiguous range + */ + if (expect_full_octet && (range->re_lo != 0 || + range->re_hi != 255)) + return -ERANGE; + + if (range->re_stride != 1) + return -ERANGE; + + if (range->re_lo > range->re_hi) + return -EINVAL; + + if (range->re_lo != range->re_hi) + expect_full_octet = true; + + min_ip[cur_octet] = range->re_lo; + max_ip[cur_octet] = range->re_hi; - list_for_each_entry(el, &ar->ar_numaddr_ranges, el_link) { - list_for_each_entry(re, &el->el_exprs, re_link) { - min_ip[re_count] = re->re_lo; - max_ip[re_count] = re->re_hi; re_count++; } - } - tmp_ip_addr = ((min_ip[0] << 24) | (min_ip[1] << 16) | - (min_ip[2] << 8) | min_ip[3]); + cur_octet++; + } if (min_nid) - *min_nid = tmp_ip_addr; - - tmp_ip_addr = ((max_ip[0] << 24) | (max_ip[1] << 16) | - (max_ip[2] << 8) | max_ip[3]); + *min_nid = ((min_ip[0] << 24) | (min_ip[1] << 16) | + (min_ip[2] << 8) | min_ip[3]); if (max_nid) - *max_nid = tmp_ip_addr; + *max_nid = ((max_ip[0] << 24) | (max_ip[1] << 16) | + (max_ip[2] << 8) | max_ip[3]); + + return 0; } /** @@ -495,11 +521,13 @@ static void cfs_ip_ar_min_max(struct addrrange *ar, u32 *min_nid, * numeric address range * * @ar - * @min_nid - * @max_nid + * @min_nid *min_nid __u32 representation of min NID + * @max_nid *max_nid __u32 representation of max NID + * + * Return: -EINVAL unsupported LNET range */ -static void cfs_num_ar_min_max(struct addrrange *ar, u32 *min_nid, - u32 *max_nid) +static int cfs_num_ar_min_max(struct addrrange *ar, u32 *min_nid, + u32 *max_nid) { struct cfs_expr_list *el; struct cfs_range_expr *re; @@ -507,11 +535,20 @@ static void cfs_num_ar_min_max(struct addrrange *ar, u32 *min_nid, unsigned int max_addr = 0; list_for_each_entry(el, &ar->ar_numaddr_ranges, el_link) { + int re_count = 0; + list_for_each_entry(re, &el->el_exprs, re_link) { + if (re_count > 0) + return -EINVAL; + if (re->re_lo > re->re_hi) + return -EINVAL; + if (re->re_lo < min_addr || !min_addr) min_addr = re->re_lo; if (re->re_hi > max_addr) max_addr = re->re_hi; + + re_count++; } } @@ -519,143 +556,8 @@ static void cfs_num_ar_min_max(struct addrrange *ar, u32 *min_nid, *min_nid = min_addr; if (max_nid) *max_nid = max_addr; -} - -/** - * Determines whether an expression list in an nidrange contains exactly - * one contiguous address range. Calls the correct netstrfns for the LND - * - * @nidlist - * - * Return: true if contiguous - * false if not contiguous - */ -bool cfs_nidrange_is_contiguous(struct list_head *nidlist) -{ - struct nidrange *nr; - struct netstrfns *nf = NULL; - char *lndname = NULL; - int netnum = -1; - - list_for_each_entry(nr, nidlist, nr_link) { - nf = nr->nr_netstrfns; - if (!lndname) - lndname = nf->nf_name; - if (netnum == -1) - netnum = nr->nr_netnum; - - if (strcmp(lndname, nf->nf_name) || - netnum != nr->nr_netnum) - return false; - } - - if (!nf) - return false; - - if (!nf->nf_is_contiguous(nidlist)) - return false; - - return true; -} -EXPORT_SYMBOL(cfs_nidrange_is_contiguous); - -/** - * Determines whether an expression list in an num nidrange contains exactly - * one contiguous address range. - * - * @nidlist - * - * Return: true if contiguous - * false if not contiguous - */ -static bool cfs_num_is_contiguous(struct list_head *nidlist) -{ - struct nidrange *nr; - struct addrrange *ar; - struct cfs_expr_list *el; - struct cfs_range_expr *re; - int last_hi = 0; - u32 last_end_nid = 0; - u32 current_start_nid = 0; - u32 current_end_nid = 0; - - list_for_each_entry(nr, nidlist, nr_link) { - list_for_each_entry(ar, &nr->nr_addrranges, ar_link) { - cfs_num_ar_min_max(ar, ¤t_start_nid, - ¤t_end_nid); - if (last_end_nid && - (current_start_nid - last_end_nid != 1)) - return false; - last_end_nid = current_end_nid; - list_for_each_entry(el, &ar->ar_numaddr_ranges, - el_link) { - list_for_each_entry(re, &el->el_exprs, - re_link) { - if (re->re_stride > 1) - return false; - else if (last_hi && - re->re_hi - last_hi != 1) - return false; - last_hi = re->re_hi; - } - } - } - } - return true; -} - -/** - * Determines whether an expression list in an ip nidrange contains exactly - * one contiguous address range. - * - * @nidlist - * - * Return: true if contiguous - * false if not contiguous - */ -static bool cfs_ip_is_contiguous(struct list_head *nidlist) -{ - struct nidrange *nr; - struct addrrange *ar; - struct cfs_expr_list *el; - struct cfs_range_expr *re; - int expr_count; - int last_hi = 255; - int last_diff = 0; - u32 last_end_nid = 0; - u32 current_start_nid = 0; - u32 current_end_nid = 0; - - list_for_each_entry(nr, nidlist, nr_link) { - list_for_each_entry(ar, &nr->nr_addrranges, ar_link) { - last_hi = 255; - last_diff = 0; - cfs_ip_ar_min_max(ar, ¤t_start_nid, - ¤t_end_nid); - if (last_end_nid && - (current_start_nid - last_end_nid != 1)) - return false; - last_end_nid = current_end_nid; - list_for_each_entry(el, &ar->ar_numaddr_ranges, - el_link) { - expr_count = 0; - list_for_each_entry(re, &el->el_exprs, - re_link) { - expr_count++; - if (re->re_stride > 1 || - (last_diff > 0 && last_hi != 255) || - (last_diff > 0 && last_hi == 255 && - re->re_lo > 0)) - return false; - last_hi = re->re_hi; - last_diff = re->re_hi - re->re_lo; - } - } - } - } - - return true; + return 0; } /** @@ -663,29 +565,35 @@ static bool cfs_ip_is_contiguous(struct list_head *nidlist) * and maximum nid and creates appropriate nid structures * * @nidlist - * @min_nid - * @max_nid + * @min_nid *min_nid string representation of min NID + * @max_nid *max_nid string representation of max NID + * + * Return: -EINVAL unsupported LNET range + * -ERANGE non-contiguous LNET range */ -void cfs_nidrange_find_min_max(struct list_head *nidlist, char *min_nid, - char *max_nid, size_t nidstr_length) +int cfs_nidrange_find_min_max(struct list_head *nidlist, char *min_nid, + char *max_nid, size_t nidstr_length) { - struct nidrange *nr; - struct netstrfns *nf = NULL; - int netnum = -1; + struct nidrange *first_nidrange; + int netnum; + struct netstrfns *nf; + char *lndname; u32 min_addr; u32 max_addr; - char *lndname = NULL; char min_addr_str[IPSTRING_LENGTH]; char max_addr_str[IPSTRING_LENGTH]; + int rc; - list_for_each_entry(nr, nidlist, nr_link) { - nf = nr->nr_netstrfns; - lndname = nf->nf_name; - if (netnum == -1) - netnum = nr->nr_netnum; + first_nidrange = list_entry(nidlist->next, struct nidrange, nr_link); + + netnum = first_nidrange->nr_netnum; + nf = first_nidrange->nr_netstrfns; + lndname = nf->nf_name; + + rc = nf->nf_min_max(nidlist, &min_addr, &max_addr); + if (rc < 0) + return rc; - nf->nf_min_max(nidlist, &min_addr, &max_addr); - } nf->nf_addr2str(min_addr, min_addr_str, sizeof(min_addr_str)); nf->nf_addr2str(max_addr, max_addr_str, sizeof(max_addr_str)); @@ -693,6 +601,8 @@ void cfs_nidrange_find_min_max(struct list_head *nidlist, char *min_nid, netnum); snprintf(max_nid, nidstr_length, "%s@%s%d", max_addr_str, lndname, netnum); + + return 0; } EXPORT_SYMBOL(cfs_nidrange_find_min_max); @@ -700,11 +610,14 @@ void cfs_nidrange_find_min_max(struct list_head *nidlist, char *min_nid, * Determines the min and max NID values for num LNDs * * @nidlist - * @min_nid - * @max_nid + * @min_nid *min_nid if provided, returns string representation of min NID + * @max_nid *max_nid if provided, returns string representation of max NID + * + * Return: -EINVAL unsupported LNET range + * -ERANGE non-contiguous LNET range */ -static void cfs_num_min_max(struct list_head *nidlist, u32 *min_nid, - u32 *max_nid) +static int cfs_num_min_max(struct list_head *nidlist, u32 *min_nid, + u32 *max_nid) { struct nidrange *nr; struct addrrange *ar; @@ -712,19 +625,32 @@ static void cfs_num_min_max(struct list_head *nidlist, u32 *min_nid, unsigned int tmp_max_addr = 0; unsigned int min_addr = 0; unsigned int max_addr = 0; + int nidlist_count = 0; + int rc; list_for_each_entry(nr, nidlist, nr_link) { + if (nidlist_count > 0) + return -EINVAL; + list_for_each_entry(ar, &nr->nr_addrranges, ar_link) { - cfs_num_ar_min_max(ar, &tmp_min_addr, - &tmp_max_addr); + rc = cfs_num_ar_min_max(ar, &tmp_min_addr, + &tmp_max_addr); + if (rc) + return rc; + if (tmp_min_addr < min_addr || !min_addr) min_addr = tmp_min_addr; if (tmp_max_addr > max_addr) max_addr = tmp_min_addr; } } - *max_nid = max_addr; - *min_nid = min_addr; + + if (max_nid) + *max_nid = max_addr; + if (min_nid) + *min_nid = min_addr; + + return 0; } /** @@ -732,11 +658,14 @@ static void cfs_num_min_max(struct list_head *nidlist, u32 *min_nid, * ip addresses. * * @nidlist - * @min_nid - * @max_nid + * @min_nid *min_nid if provided, returns string representation of min NID + * @max_nid *max_nid if provided, returns string representation of max NID + * + * Return: -EINVAL unsupported LNET range + * -ERANGE non-contiguous LNET range */ -static void cfs_ip_min_max(struct list_head *nidlist, u32 *min_nid, - u32 *max_nid) +static int cfs_ip_min_max(struct list_head *nidlist, u32 *min_nid, + u32 *max_nid) { struct nidrange *nr; struct addrrange *ar; @@ -744,22 +673,34 @@ static void cfs_ip_min_max(struct list_head *nidlist, u32 *min_nid, u32 tmp_max_ip_addr = 0; u32 min_ip_addr = 0; u32 max_ip_addr = 0; + int nidlist_count = 0; + int rc; list_for_each_entry(nr, nidlist, nr_link) { + if (nidlist_count > 0) + return -EINVAL; + list_for_each_entry(ar, &nr->nr_addrranges, ar_link) { - cfs_ip_ar_min_max(ar, &tmp_min_ip_addr, - &tmp_max_ip_addr); + rc = cfs_ip_ar_min_max(ar, &tmp_min_ip_addr, + &tmp_max_ip_addr); + if (rc) + return rc; + if (tmp_min_ip_addr < min_ip_addr || !min_ip_addr) min_ip_addr = tmp_min_ip_addr; if (tmp_max_ip_addr > max_ip_addr) max_ip_addr = tmp_max_ip_addr; } + + nidlist_count++; } if (min_nid) *min_nid = min_ip_addr; if (max_nid) *max_nid = max_ip_addr; + + return 0; } static int @@ -966,7 +907,6 @@ static void cfs_ip_min_max(struct list_head *nidlist, u32 *min_nid, .nf_parse_addrlist = libcfs_num_parse, .nf_print_addrlist = libcfs_num_addr_range_print, .nf_match_addr = libcfs_num_match, - .nf_is_contiguous = cfs_num_is_contiguous, .nf_min_max = cfs_num_min_max }, { .nf_type = SOCKLND, .nf_name = "tcp", @@ -976,7 +916,6 @@ static void cfs_ip_min_max(struct list_head *nidlist, u32 *min_nid, .nf_parse_addrlist = cfs_ip_addr_parse, .nf_print_addrlist = libcfs_ip_addr_range_print, .nf_match_addr = cfs_ip_addr_match, - .nf_is_contiguous = cfs_ip_is_contiguous, .nf_min_max = cfs_ip_min_max }, { .nf_type = O2IBLND, .nf_name = "o2ib", @@ -986,7 +925,6 @@ static void cfs_ip_min_max(struct list_head *nidlist, u32 *min_nid, .nf_parse_addrlist = cfs_ip_addr_parse, .nf_print_addrlist = libcfs_ip_addr_range_print, .nf_match_addr = cfs_ip_addr_match, - .nf_is_contiguous = cfs_ip_is_contiguous, .nf_min_max = cfs_ip_min_max }, { .nf_type = GNILND, .nf_name = "gni", @@ -996,7 +934,6 @@ static void cfs_ip_min_max(struct list_head *nidlist, u32 *min_nid, .nf_parse_addrlist = libcfs_num_parse, .nf_print_addrlist = libcfs_num_addr_range_print, .nf_match_addr = libcfs_num_match, - .nf_is_contiguous = cfs_num_is_contiguous, .nf_min_max = cfs_num_min_max }, { .nf_type = GNIIPLND, .nf_name = "gip", @@ -1006,7 +943,6 @@ static void cfs_ip_min_max(struct list_head *nidlist, u32 *min_nid, .nf_parse_addrlist = cfs_ip_addr_parse, .nf_print_addrlist = libcfs_ip_addr_range_print, .nf_match_addr = cfs_ip_addr_match, - .nf_is_contiguous = cfs_ip_is_contiguous, .nf_min_max = cfs_ip_min_max }, }; From patchwork Mon Sep 30 18:56:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167317 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A942A1599 for ; Mon, 30 Sep 2019 19:07:46 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 91EE0224EF for ; Mon, 30 Sep 2019 19:07:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 91EE0224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 00DC65E4F4F; Mon, 30 Sep 2019 12:01:28 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 7C1E15C38DF for ; Mon, 30 Sep 2019 11:57:40 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id C91671005F9B; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id C684CB5; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:29 -0400 Message-Id: <1569869810-23848-131-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 130/151] lustre: osc: add a bit to indicate osc_page in cache tree X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Bobi Jam , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Bobi Jam Add osc_page::ops_intree to indicate whether the osc_page is in the osc_object's cache tree, so that when page cannot insert in the cache as race happens, the cleanup code won't try to remove it from the cache. WC-bug-id: https://jira.whamcloud.com/browse/LU-10244 Lustre-commit: ed91ee6bd642 ("LU-10244 osc: add a bit to indicate osc_page in cache tree") Signed-off-by: Bobi Jam Reviewed-on: https://review.whamcloud.com/30096 Reviewed-by: Jinshan Xiong Reviewed-by: Fan Yong Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_osc.h | 6 +++++- fs/lustre/osc/osc_page.c | 17 ++++++++++++----- 2 files changed, 17 insertions(+), 6 deletions(-) diff --git a/fs/lustre/include/lustre_osc.h b/fs/lustre/include/lustre_osc.h index 38ae83a8..5ba4f97 100644 --- a/fs/lustre/include/lustre_osc.h +++ b/fs/lustre/include/lustre_osc.h @@ -524,7 +524,11 @@ struct osc_page { /* * Set if the page must be transferred with OBD_BRW_SRVLOCK. */ - ops_srvlock:1; + ops_srvlock:1, + /** + * If the page is in osc_object::oo_tree. + */ + ops_intree:1; /* * lru page list. See osc_lru_{del|use}() in osc_page.c for usage. */ diff --git a/fs/lustre/osc/osc_page.c b/fs/lustre/osc/osc_page.c index 731fd27..9236e02 100644 --- a/fs/lustre/osc/osc_page.c +++ b/fs/lustre/osc/osc_page.c @@ -192,12 +192,17 @@ static void osc_page_delete(const struct lu_env *env, osc_lru_del(osc_cli(obj), opg); if (slice->cpl_page->cp_type == CPT_CACHEABLE) { - void *value; + void *value = NULL; spin_lock(&obj->oo_tree_lock); - value = radix_tree_delete(&obj->oo_tree, osc_index(opg)); - if (value) - --obj->oo_npages; + if (opg->ops_intree) { + value = radix_tree_delete(&obj->oo_tree, + osc_index(opg)); + if (value) { + --obj->oo_npages; + opg->ops_intree = 0; + } + } spin_unlock(&obj->oo_tree_lock); LASSERT(ergo(value, value == opg)); @@ -275,8 +280,10 @@ int osc_page_init(const struct lu_env *env, struct cl_object *obj, spin_lock(&osc->oo_tree_lock); result = radix_tree_insert(&osc->oo_tree, index, opg); - if (result == 0) + if (result == 0) { ++osc->oo_npages; + opg->ops_intree = 1; + } spin_unlock(&osc->oo_tree_lock); radix_tree_preload_end(); From patchwork Mon Sep 30 18:56:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167303 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C78B11599 for ; Mon, 30 Sep 2019 19:07:14 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AFC0B224EF for ; Mon, 30 Sep 2019 19:07:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AFC0B224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 0D5275E4E3D; Mon, 30 Sep 2019 12:01:15 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id BE0725C3A7B for ; Mon, 30 Sep 2019 11:57:40 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id CCA9B1005F9C; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id C98D0BD; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:30 -0400 Message-Id: <1569869810-23848-132-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 131/151] lustre: ldlm: fix export reference X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Hongchao Zhang , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Hongchao Zhang In ptlrpc_connect_interpert, the export reference could be leaked if there is error before the following class_exp_put. WC-bug-id: https://jira.whamcloud.com/browse/LU-10656 Lustre-commit: e52f2ea40e55 ("LU-10656 ldlm: fix export reference") Signed-off-by: Hongchao Zhang Reviewed-on: https://review.whamcloud.com/31139 Reviewed-by: Andreas Dilger Reviewed-by: Li Dongyang Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/ptlrpc/import.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/fs/lustre/ptlrpc/import.c b/fs/lustre/ptlrpc/import.c index 152a84c..6fc0c6b 100644 --- a/fs/lustre/ptlrpc/import.c +++ b/fs/lustre/ptlrpc/import.c @@ -935,7 +935,7 @@ static int ptlrpc_connect_interpret(const struct lu_env *env, u64 old_connect_flags; int msg_flags; struct obd_connect_data *ocd; - struct obd_export *exp; + struct obd_export *exp = NULL; int ret; spin_lock(&imp->imp_lock); @@ -1040,6 +1040,8 @@ static int ptlrpc_connect_interpret(const struct lu_env *env, rc = ptlrpc_connect_set_flags(imp, ocd, old_connect_flags, exp, aa->pcaa_initial_connect); class_export_put(exp); + exp = NULL; + if (rc) goto out; @@ -1205,6 +1207,9 @@ static int ptlrpc_connect_interpret(const struct lu_env *env, imp->imp_connect_tried = 1; spin_unlock(&imp->imp_lock); + if (exp) + class_export_put(exp); + if (rc != 0) { IMPORT_SET_STATE(imp, LUSTRE_IMP_DISCON); if (rc == -EACCES) { From patchwork Mon Sep 30 18:56:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167339 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 109551599 for ; Mon, 30 Sep 2019 19:08:30 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EAAB1224EF for ; Mon, 30 Sep 2019 19:08:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EAAB1224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 30BF7603686; Mon, 30 Sep 2019 12:01:49 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 0D5F75C3CA7 for ; Mon, 30 Sep 2019 11:57:41 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id CE3441005F9D; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id CCF54A9; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:31 -0400 Message-Id: <1569869810-23848-133-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 132/151] lustre: llite: Add exit for filedata allocation failed X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Ben Evans When the filedata allocation fails, we need to exit to a later point than out_openerr, which calls deauthorize_statahead and ll_file_data_put, neither of which is valid. (This leads to a panic.) WC-bug-id: https://jira.whamcloud.com/browse/LU-10662 Cray-bug-id: LUS-1321 Lustre-commit: d9edfa50683a ('LU-10662 llite: Add exit for filedata allocation failed") Signed-off-by: Ben Evans Reviewed-on: https://review.whamcloud.com/31296 Reviewed-by: Patrick Farrell Reviewed-by: Alexandr Boyko Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/file.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index da5bf86..6732b68 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -540,7 +540,7 @@ int ll_file_open(struct inode *inode, struct file *file) fd = ll_file_data_get(); if (!fd) { rc = -ENOMEM; - goto out_openerr; + goto out_nofiledata; } fd->fd_file = file; @@ -723,6 +723,7 @@ int ll_file_open(struct inode *inode, struct file *file) ll_stats_ops_tally(ll_i2sbi(inode), LPROC_LL_OPEN, 1); } +out_nofiledata: if (it && it_disposition(it, DISP_ENQ_OPEN_REF)) { ptlrpc_req_finished(it->it_request); it_clear_disposition(it, DISP_ENQ_OPEN_REF); From patchwork Mon Sep 30 18:56:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167341 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F035213B1 for ; Mon, 30 Sep 2019 19:08:36 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D87C7224EF for ; Mon, 30 Sep 2019 19:08:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D87C7224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id DFE536037A1; Mon, 30 Sep 2019 12:01:53 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 4EBBE5C3CA7 for ; Mon, 30 Sep 2019 11:57:41 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id D2DDD1005F9E; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id D069CB4; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:32 -0400 Message-Id: <1569869810-23848-134-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 133/151] lustre: misc: Wrong checksum return value X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Qian Yingjin , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Qian Yingjin In the checksum calculation functions: tgt_checksum_niobuf and osc_checksum_bulk, it is wrongly taken the error return value of cfs_crypto_hash_init as the checksum value. This patch fixes the problem. WC-bug-id: https://jira.whamcloud.com/browse/LU-10737 Lustre-commit: 28b8f3d9296d ("LU-10737 misc: Wrong checksum return value") Signed-off-by: Qian Yingjin Reviewed-on: https://review.whamcloud.com/31448 Reviewed-by: Dmitry Eremin Reviewed-by: James Simmons Reviewed-by: Li Xi Reviewed-by: Andreas Dilger Signed-off-by: James Simmons --- fs/lustre/osc/osc_request.c | 51 ++++++++++++++++++++++++++++----------------- 1 file changed, 32 insertions(+), 19 deletions(-) diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index 5797535..0a3d05e 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -1028,11 +1028,11 @@ static inline int can_merge_pages(struct brw_page *p1, struct brw_page *p2) return (p1->off + p1->count == p2->off); } -static u32 osc_checksum_bulk(int nob, u32 pg_count, +static int osc_checksum_bulk(int nob, u32 pg_count, struct brw_page **pga, int opc, - enum cksum_type cksum_type) + enum cksum_type cksum_type, + u32 *cksum) { - u32 cksum; int i = 0; struct ahash_request *hdesc; unsigned int bufsize; @@ -1076,16 +1076,16 @@ static u32 osc_checksum_bulk(int nob, u32 pg_count, i++; } - bufsize = sizeof(cksum); - cfs_crypto_hash_final(hdesc, (unsigned char *)&cksum, &bufsize); + bufsize = sizeof(*cksum); + cfs_crypto_hash_final(hdesc, (unsigned char *)cksum, &bufsize); /* For sending we only compute the wrong checksum instead * of corrupting the data so it is still correct on a redo */ if (opc == OST_WRITE && OBD_FAIL_CHECK(OBD_FAIL_OSC_CHECKSUM_SEND)) - cksum++; + (*cksum)++; - return cksum; + return 0; } static int osc_brw_prep_request(int cmd, struct client_obd *cli, @@ -1296,12 +1296,18 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli, body->oa.o_flags |= cksum_type_pack(cksum_type); body->oa.o_valid |= OBD_MD_FLCKSUM | OBD_MD_FLFLAGS; - body->oa.o_cksum = osc_checksum_bulk(requested_nob, - page_count, pga, - OST_WRITE, - cksum_type); + + rc = osc_checksum_bulk(requested_nob, page_count, + pga, OST_WRITE, cksum_type, + &body->oa.o_cksum); + if (rc < 0) { + CDEBUG(D_PAGE, "failed to checksum, rc = %d\n", + rc); + goto out; + } CDEBUG(D_PAGE, "checksum at write origin: %x\n", body->oa.o_cksum); + /* save this in 'oa', too, for later checking */ oa->o_valid |= OBD_MD_FLCKSUM | OBD_MD_FLFLAGS; oa->o_flags |= cksum_type_pack(cksum_type); @@ -1427,6 +1433,7 @@ static int check_write_checksum(struct obdo *oa, u32 new_cksum; char *msg; enum cksum_type cksum_type; + int rc; if (server_cksum == client_cksum) { CDEBUG(D_PAGE, "checksum %x confirmed\n", client_cksum); @@ -1439,10 +1446,13 @@ static int check_write_checksum(struct obdo *oa, cksum_type = cksum_type_unpack(oa->o_valid & OBD_MD_FLFLAGS ? oa->o_flags : 0); - new_cksum = osc_checksum_bulk(aa->aa_requested_nob, aa->aa_page_count, - aa->aa_ppga, OST_WRITE, cksum_type); + rc = osc_checksum_bulk(aa->aa_requested_nob, aa->aa_page_count, + aa->aa_ppga, OST_WRITE, cksum_type, + &new_cksum); - if (cksum_type != cksum_type_unpack(aa->aa_oa->o_flags)) + if (rc < 0) + msg = "failed to calculate the client write checksum"; + else if (cksum_type != cksum_type_unpack(aa->aa_oa->o_flags)) msg = "the server did not use the checksum type specified in the original request - likely a protocol problem"; else if (new_cksum == server_cksum) msg = "changed on the client after we checksummed it - likely false positive due to mmap IO (bug 11742)"; @@ -1599,13 +1609,16 @@ static int osc_brw_fini_request(struct ptlrpc_request *req, int rc) char *router = ""; enum cksum_type cksum_type; - cksum_type = cksum_type_unpack(body->oa.o_valid & - OBD_MD_FLFLAGS ? + cksum_type = cksum_type_unpack(body->oa.o_valid & OBD_MD_FLFLAGS ? body->oa.o_flags : 0); - client_cksum = osc_checksum_bulk(rc, aa->aa_page_count, - aa->aa_ppga, OST_READ, - cksum_type); + rc = osc_checksum_bulk(rc, aa->aa_page_count, aa->aa_ppga, + OST_READ, cksum_type, &client_cksum); + if (rc < 0) { + CDEBUG(D_PAGE, + "failed to calculate checksum, rc = %d\n", rc); + goto out; + } if (req->rq_bulk && peer->nid != req->rq_bulk->bd_sender) { via = " via "; From patchwork Mon Sep 30 18:56:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167321 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 529151599 for ; Mon, 30 Sep 2019 19:07:55 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3B45B224EF for ; Mon, 30 Sep 2019 19:07:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3B45B224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 6EC7D5E4F9F; Mon, 30 Sep 2019 12:01:32 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id A6ADC5C3CA7 for ; Mon, 30 Sep 2019 11:57:41 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id D5C961005F9F; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id D36A9BB; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:33 -0400 Message-Id: <1569869810-23848-135-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 134/151] lustre: llite: fix mount error handing X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Vladimir Saveliev , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Vladimir Saveliev lustre_fill_super() allocates lsi and assumes that on failures lsi will be freed by server_fill_super() or ll_fill_super(). - server_fill_super() does not free lsi when lsi_prepare() fails. - ll_fill_super() does not free lsi when OBD_ALLOC_PTR(cfg) or ll_init_sbi() fail. WC-bug-id: https://jira.whamcloud.com/browse/LU-5991 Cray-bug-id: MRP-2229 Lustre-commit: acabfb9594c9 ("LU-5991 llite: fix mount error handing") Signed-off-by: Vladimir Saveliev Reviewed-on: https://review.whamcloud.com/12959 Reviewed-by: Sergey Cheremencev Reviewed-by: Andriy Skulysh Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/llite_lib.c | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index 758f856..c94bc65 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -939,7 +939,7 @@ int ll_fill_super(struct super_block *sb) { struct lustre_profile *lprof = NULL; struct lustre_sb_info *lsi = s2lsi(sb); - struct ll_sb_info *sbi; + struct ll_sb_info *sbi = NULL; char *dt = NULL, *md = NULL; char *profilenm = get_profile_name(sb); struct config_llog_instance *cfg; @@ -950,21 +950,20 @@ int ll_fill_super(struct super_block *sb) CDEBUG(D_VFSTRACE, "VFS Op: sb %p\n", sb); + try_module_get(THIS_MODULE); + cfg = kzalloc(sizeof(*cfg), GFP_NOFS); if (!cfg) { - ll_common_put_super(sb); - return -ENOMEM; + err = -ENOMEM; + goto out_free; } - try_module_get(THIS_MODULE); /* client additional sb info */ sbi = ll_init_sbi(); lsi->lsi_llsbi = sbi; if (!sbi) { - module_put(THIS_MODULE); - kfree(cfg); - ll_common_put_super(sb); - return -ENOMEM; + err = -ENOMEM; + goto out_free; } err = ll_options(lsi->lsi_lmd->lmd_opts, &sbi->ll_flags); @@ -1048,12 +1047,12 @@ int ll_fill_super(struct super_block *sb) kfree(dt); if (lprof) class_put_profile(lprof); + kfree(cfg); if (err) ll_put_super(sb); else if (sbi->ll_flags & LL_SBI_VERBOSE) LCONSOLE_WARN("Mounted %s\n", profilenm); - kfree(cfg); return err; } /* ll_fill_super */ @@ -1073,6 +1072,9 @@ void ll_put_super(struct super_block *sb) int next, force = 1, rc = 0; long ccc_count; + if (!sbi) + goto out_no_sbi; + CDEBUG(D_VFSTRACE, "VFS Op: sb %p - %s\n", sb, profilenm); cfg.cfg_instance = sb; @@ -1125,6 +1127,7 @@ void ll_put_super(struct super_block *sb) ll_free_sbi(sb); lsi->lsi_llsbi = NULL; +out_no_sbi: ll_common_put_super(sb); cl_env_cache_purge(~0); From patchwork Mon Sep 30 18:56:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167261 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 209DC1599 for ; Mon, 30 Sep 2019 19:05:37 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 08642224EF for ; Mon, 30 Sep 2019 19:05:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 08642224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 333E65E4AA1; Mon, 30 Sep 2019 12:00:38 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id E82735C3CBE for ; Mon, 30 Sep 2019 11:57:41 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id D811C1005FA0; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id D6798B5; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:34 -0400 Message-Id: <1569869810-23848-136-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 135/151] lustre: llite: Disable tiny writes for append X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Patrick Farrell Unfortunately, tiny writes do not work correctly with appending to files. When appending to a file, we must take DLM locks to EOF on all stripes, in order to protect file size so we can append correctly. If we dirty a page with a normal write then append to it with a tiny write, these DLM locks are not present, and we can use an incorrect size if another client writes to a different stripe, increasing the size without cancelling the lock which is protecting our dirty page. We could theoretically check to make sure the required DLM locks are held, but this would be time consuming. The simplest solution is to just not allow tiny writes when appending. Also add option to disable tiny writes at runtime. WC-bug-id: https://jira.whamcloud.com/browse/LU-10681 Cray-bug-id: LUS-5723 Lustre-commit: d79ffa3ff746 ("LU-10681 llite: Disable tiny writes for append") Signed-off-by: Patrick Farrell Reviewed-on: https://review.whamcloud.com/31353 Reviewed-by: Andreas Dilger Reviewed-by: Bobi Jam Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/file.c | 70 +++++++++------------------------------- fs/lustre/llite/llite_internal.h | 8 +++++ fs/lustre/llite/llite_lib.c | 1 + fs/lustre/llite/lproc_llite.c | 36 +++++++++++++++++++++ 4 files changed, 61 insertions(+), 54 deletions(-) diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 6732b68..fe4340d 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -1484,70 +1484,29 @@ static ssize_t ll_file_read_iter(struct kiocb *iocb, struct iov_iter *to) * and will write it out. This saves a lot of processing time. * * All writes here are within one page, so exclusion is handled by the page - * lock on the vm page. Exception is appending, which requires locking the - * full file to handle size issues. We do not do tiny writes for writes which - * touch multiple pages because it's very unlikely multiple sequential pages + * lock on the vm page. We do not do tiny writes for writes which touch + * multiple pages because it's very unlikely multiple sequential pages are * are already dirty. * * We limit these to < PAGE_SIZE because PAGE_SIZE writes are relatively common * and are unlikely to be to already dirty pages. * - * Attribute updates are important here, we do it in ll_tiny_write_end. + * Attribute updates are important here, we do them in ll_tiny_write_end. */ static ssize_t ll_do_tiny_write(struct kiocb *iocb, struct iov_iter *iter) { ssize_t count = iov_iter_count(iter); struct file *file = iocb->ki_filp; struct inode *inode = file_inode(file); - struct ll_inode_info *lli = ll_i2info(inode); - struct range_lock range; ssize_t result = 0; - bool append = false; - - /* NB: we can't do direct IO for tiny writes because they use the page - * cache, and we can't do sync writes because tiny writes can't flush - * pages. - */ - if (file->f_flags & (O_DIRECT | O_SYNC)) - return 0; - /* It is relatively unlikely we will overwrite a full dirty page, so - * limit tiny writes to < PAGE_SIZE + /* Restrict writes to single page and < PAGE_SIZE. See comment at top + * of function for why. */ - if (count >= PAGE_SIZE) + if (count >= PAGE_SIZE || + (iocb->ki_pos & (PAGE_SIZE-1)) + count > PAGE_SIZE) return 0; - /* For append writes, we must take the range lock to protect size - * and also move pos to current size before writing. - */ - if (file->f_flags & O_APPEND) { - struct lu_env *env; - u16 refcheck; - - append = true; - range_lock_init(&range, 0, LUSTRE_EOF); - result = range_lock(&lli->lli_write_tree, &range); - if (result) - return result; - env = cl_env_get(&refcheck); - if (IS_ERR(env)) { - result = PTR_ERR(env); - goto out; - } - ll_merge_attr(env, inode); - cl_env_put(env, &refcheck); - iocb->ki_pos = i_size_read(inode); - } - - /* Does this write touch multiple pages? - * - * This partly duplicates the PAGE_SIZE check above, but must come - * after range locking for append writes because it depends on the - * write position (ki_pos). - */ - if ((iocb->ki_pos & (PAGE_SIZE-1)) + count > PAGE_SIZE) - goto out; - result = __generic_file_write_iter(iocb, iter); /* If the page is not already dirty, ll_tiny_write_begin returns @@ -1562,10 +1521,6 @@ static ssize_t ll_do_tiny_write(struct kiocb *iocb, struct iov_iter *iter) set_bit(LLIF_DATA_MODIFIED, &ll_i2info(inode)->lli_flags); } -out: - if (append) - range_unlock(&lli->lli_write_tree, &range); - CDEBUG(D_VFSTRACE, "result: %zu, original count %zu\n", result, count); return result; @@ -1578,10 +1533,17 @@ static ssize_t ll_file_write_iter(struct kiocb *iocb, struct iov_iter *from) { struct lu_env *env; struct vvp_io_args *args; - ssize_t rc_tiny, rc_normal; + ssize_t rc_tiny = 0, rc_normal; u16 refcheck; - rc_tiny = ll_do_tiny_write(iocb, from); + /* NB: we can't do direct IO for tiny writes because they use the page + * cache, we can't do sync writes because tiny writes can't flush + * pages, and we can't do append writes because we can't guarantee the + * required DLM locks are held to protect file size. + */ + if (ll_sbi_has_tiny_write(ll_i2sbi(file_inode(iocb->ki_filp))) && + !(iocb->ki_filp->f_flags & (O_DIRECT | O_SYNC | O_APPEND))) + rc_tiny = ll_do_tiny_write(iocb, from); /* In case of error, go on and try normal write - Only stop if tiny * write completed I/O. diff --git a/fs/lustre/llite/llite_internal.h b/fs/lustre/llite/llite_internal.h index f23cf65..e067ba4 100644 --- a/fs/lustre/llite/llite_internal.h +++ b/fs/lustre/llite/llite_internal.h @@ -419,6 +419,7 @@ enum stats_track_type { #define LL_SBI_FILE_SECCTX 0x800000 /* set file security context at * create */ +#define LL_SBI_TINY_WRITE 0x2000000 /* tiny write support */ #define LL_SBI_FLAGS { \ "nolck", \ @@ -445,6 +446,8 @@ enum stats_track_type { "always_ping", \ "fast_read", \ "file_secctx", \ + "pio", \ + "tiny_write", \ } /* @@ -705,6 +708,11 @@ static inline bool ll_sbi_has_fast_read(struct ll_sb_info *sbi) return !!(sbi->ll_flags & LL_SBI_FAST_READ); } +static inline bool ll_sbi_has_tiny_write(struct ll_sb_info *sbi) +{ + return !!(sbi->ll_flags & LL_SBI_TINY_WRITE); +} + void ll_ras_enter(struct file *f); /* llite/lcommon_misc.c */ diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index c94bc65..4d14ce1 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -125,6 +125,7 @@ static struct ll_sb_info *ll_init_sbi(void) atomic_set(&sbi->ll_agl_total, 0); sbi->ll_flags |= LL_SBI_AGL_ENABLED; sbi->ll_flags |= LL_SBI_FAST_READ; + sbi->ll_flags |= LL_SBI_TINY_WRITE; /* root squash */ sbi->ll_squash.rsi_uid = 0; diff --git a/fs/lustre/llite/lproc_llite.c b/fs/lustre/llite/lproc_llite.c index b69e5d7..e108326 100644 --- a/fs/lustre/llite/lproc_llite.c +++ b/fs/lustre/llite/lproc_llite.c @@ -1024,6 +1024,41 @@ static ssize_t xattr_cache_store(struct kobject *kobj, } LUSTRE_RW_ATTR(xattr_cache); +static ssize_t tiny_write_show(struct kobject *kobj, + struct attribute *attr, + char *buf) +{ + struct ll_sb_info *sbi = container_of(kobj, struct ll_sb_info, + ll_kset.kobj); + + return sprintf(buf, "%u\n", !!(sbi->ll_flags & LL_SBI_TINY_WRITE)); +} + +static ssize_t tiny_write_store(struct kobject *kobj, + struct attribute *attr, + const char *buffer, + size_t count) +{ + struct ll_sb_info *sbi = container_of(kobj, struct ll_sb_info, + ll_kset.kobj); + bool val; + int rc; + + rc = kstrtobool(buffer, &val); + if (rc) + return rc; + + spin_lock(&sbi->ll_lock); + if (val) + sbi->ll_flags |= LL_SBI_TINY_WRITE; + else + sbi->ll_flags &= ~LL_SBI_TINY_WRITE; + spin_unlock(&sbi->ll_lock); + + return count; +} +LUSTRE_RW_ATTR(tiny_write); + static ssize_t fast_read_show(struct kobject *kobj, struct attribute *attr, char *buf) @@ -1225,6 +1260,7 @@ static ssize_t ll_nosquash_nids_seq_write(struct file *file, &lustre_attr_default_easize.attr, &lustre_attr_xattr_cache.attr, &lustre_attr_fast_read.attr, + &lustre_attr_tiny_write.attr, NULL, }; From patchwork Mon Sep 30 18:56:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167325 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4DCC513B1 for ; Mon, 30 Sep 2019 19:08:03 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 36444224EF for ; Mon, 30 Sep 2019 19:08:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 36444224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 841425E4FE8; Mon, 30 Sep 2019 12:01:35 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 4C38D5C3472 for ; Mon, 30 Sep 2019 11:57:42 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id DAD311005FA1; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id D9903A9; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:35 -0400 Message-Id: <1569869810-23848-137-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 136/151] lustre: uapi: replace FMODE_{READ, WRITE} with MDS_* equivs X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Sebastien Buisson In UAPI header file lustre/lustre_user.h, replace direct use of FMODE_READ and FMODE_WRITE with MDS_* equivalents. That will avoid name clashes with the kernel symbols, and avoid problems if their values ever change. WC-bug-id: https://jira.whamcloud.com/browse/LU-10483 Lustre-commit: 5185ba6381ac ("LU-10483 lustre: replace FMODE_{READ,WRITE} with MDS_* equivs") Signed-off-by: Sebastien Buisson Reviewed-on: https://review.whamcloud.com/30824 Reviewed-by: Andreas Dilger Reviewed-by: John L. Hammond Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/mdc/mdc_lib.c | 8 ++++++-- fs/lustre/mdc/mdc_locks.c | 4 ++-- fs/lustre/obdclass/genops.c | 2 +- include/uapi/linux/lustre/lustre_user.h | 3 +++ 4 files changed, 12 insertions(+), 5 deletions(-) diff --git a/fs/lustre/mdc/mdc_lib.c b/fs/lustre/mdc/mdc_lib.c index 1031a01..ba4eb3f 100644 --- a/fs/lustre/mdc/mdc_lib.c +++ b/fs/lustre/mdc/mdc_lib.c @@ -196,8 +196,12 @@ void mdc_create_pack(struct ptlrpc_request *req, struct md_op_data *op_data, static inline u64 mds_pack_open_flags(u64 flags) { - u64 cr_flags = (flags & (FMODE_READ | FMODE_WRITE | - MDS_OPEN_FL_INTERNAL)); + u64 cr_flags = (flags & MDS_OPEN_FL_INTERNAL); + + if (flags & FMODE_READ) + cr_flags |= MDS_FMODE_READ; + if (flags & FMODE_WRITE) + cr_flags |= MDS_FMODE_WRITE; if (flags & O_CREAT) cr_flags |= MDS_OPEN_CREAT; if (flags & O_EXCL) diff --git a/fs/lustre/mdc/mdc_locks.c b/fs/lustre/mdc/mdc_locks.c index 0787ba3..123f0f0 100644 --- a/fs/lustre/mdc/mdc_locks.c +++ b/fs/lustre/mdc/mdc_locks.c @@ -263,12 +263,12 @@ static int mdc_save_lovea(struct ptlrpc_request *req, /* If inode is known, cancel conflicting OPEN locks. */ if (fid_is_sane(&op_data->op_fid2)) { if (it->it_flags & MDS_OPEN_LEASE) { /* try to get lease */ - if (it->it_flags & FMODE_WRITE) + if (it->it_flags & MDS_FMODE_WRITE) mode = LCK_EX; else mode = LCK_PR; } else { - if (it->it_flags & (FMODE_WRITE | MDS_OPEN_TRUNC)) + if (it->it_flags & (MDS_FMODE_WRITE | MDS_OPEN_TRUNC)) mode = LCK_CW; else if (it->it_flags & __FMODE_EXEC) mode = LCK_PR; diff --git a/fs/lustre/obdclass/genops.c b/fs/lustre/obdclass/genops.c index f809d00c..5c9b3c4 100644 --- a/fs/lustre/obdclass/genops.c +++ b/fs/lustre/obdclass/genops.c @@ -1540,7 +1540,7 @@ static inline bool obd_skip_mod_rpc_slot(const struct lookup_intent *it) if (it && (it->it_op == IT_GETATTR || it->it_op == IT_LOOKUP || it->it_op == IT_READDIR || - (it->it_op == IT_LAYOUT && !(it->it_flags & FMODE_WRITE)))) + (it->it_op == IT_LAYOUT && !(it->it_flags & MDS_FMODE_WRITE)))) return true; return false; } diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index 29be243..2c4f837 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -866,6 +866,9 @@ enum la_valid { LA_REMOTE_ATTR_SET = (LA_UID | LA_GID | LA_PROJID | LA_LAYOUT_VERSION) }; +#define MDS_FMODE_READ 00000001 +#define MDS_FMODE_WRITE 00000002 + #define MDS_FMODE_CLOSED 00000000 #define MDS_FMODE_EXEC 00000004 /* MDS_FMODE_EPOCH 01000000 obsolete since 2.8.0 */ From patchwork Mon Sep 30 18:56:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167307 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7F96013B1 for ; Mon, 30 Sep 2019 19:07:24 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 67D90224EF for ; Mon, 30 Sep 2019 19:07:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 67D90224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 48D615E4E82; Mon, 30 Sep 2019 12:01:18 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id A41215C3CCD for ; Mon, 30 Sep 2019 11:57:42 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id DE0B41005FA2; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id DC98ABD; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:36 -0400 Message-Id: <1569869810-23848-138-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 137/151] lnet: reduce discovery timeout X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Amir Shehata , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Amir Shehata Discovery protocol sends a ping (GET) to the peer and expects a REPLY back with the interface information. Discovery uses the DEFAULT_PEER_TIMEOUT which 180s. This could lead to extended delay during mounting if the OSTs are down or if the ping fails for any reason. This patch adds a module parameter lnet_transaction_timeout which defaults to 5 seconds. lnet_transaction_timeout is used for the discovery timeout. WC-bug-id: https://jira.whamcloud.com/browse/LU-10800 Lustre-commit: 1cf929df259a ("LU-10800 lnet: reduce discovery timeout") Signed-off-by: Amir Shehata Reviewed-on: https://review.whamcloud.com/31663 Reviewed-by: Andreas Dilger Reviewed-by: Sonia Sharma Reviewed-by: Dmitry Eremin Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- include/linux/lnet/lib-lnet.h | 1 + net/lnet/lnet/api-ni.c | 44 +++++++++++++++++++++++++++++++++++++++++++ net/lnet/lnet/peer.c | 16 ++++++++-------- 3 files changed, 53 insertions(+), 8 deletions(-) diff --git a/include/linux/lnet/lib-lnet.h b/include/linux/lnet/lib-lnet.h index 3d7867f..22c6152 100644 --- a/include/linux/lnet/lib-lnet.h +++ b/include/linux/lnet/lib-lnet.h @@ -477,6 +477,7 @@ struct lnet_ni * bool lnet_is_ni_healthy_locked(struct lnet_ni *ni); struct lnet_net *lnet_get_net_locked(u32 net_id); +extern unsigned int lnet_transaction_timeout; extern unsigned int lnet_numa_range; extern unsigned int lnet_peer_discovery_disabled; extern int portal_rotor; diff --git a/net/lnet/lnet/api-ni.c b/net/lnet/lnet/api-ni.c index fc4fe5d..8be3354 100644 --- a/net/lnet/lnet/api-ni.c +++ b/net/lnet/lnet/api-ni.c @@ -92,6 +92,13 @@ struct lnet the_lnet = { MODULE_PARM_DESC(lnet_peer_discovery_disabled, "Set to 1 to disable peer discovery on this node."); +unsigned int lnet_transaction_timeout = 5; +static int transaction_to_set(const char *val, const struct kernel_param *kp); +module_param_call(lnet_transaction_timeout, transaction_to_set, param_get_int, + &lnet_transaction_timeout, 0444); +MODULE_PARM_DESC(lnet_transaction_timeout, + "Time in seconds to wait for a REPLY or an ACK"); + /* * This sequence number keeps track of how many times DLC was used to * update the local NIs. It is incremented when a NI is added or @@ -158,6 +165,43 @@ static int lnet_discover(struct lnet_process_id id, u32 force, } static int +transaction_to_set(const char *val, const struct kernel_param *kp) +{ + unsigned int *transaction_to = (unsigned int *)kp->arg; + unsigned long value; + int rc; + + rc = kstrtoul(val, 0, &value); + if (rc) { + CERROR("Invalid module parameter value for 'lnet_transaction_timeout'\n"); + return rc; + } + + /* The purpose of locking the api_mutex here is to ensure that + * the correct value ends up stored properly. + */ + mutex_lock(&the_lnet.ln_api_mutex); + + if (value == 0) { + mutex_unlock(&the_lnet.ln_api_mutex); + CERROR("Invalid value for lnet_transaction_timeout (%lu).\n", + value); + return -EINVAL; + } + + if (value == *transaction_to) { + mutex_unlock(&the_lnet.ln_api_mutex); + return 0; + } + + *transaction_to = value; + + mutex_unlock(&the_lnet.ln_api_mutex); + + return 0; +} + +static int intf_max_set(const char *val, const struct kernel_param *kp) { int value, rc; diff --git a/net/lnet/lnet/peer.c b/net/lnet/lnet/peer.c index e2f8c28..1534ab2 100644 --- a/net/lnet/lnet/peer.c +++ b/net/lnet/lnet/peer.c @@ -2942,7 +2942,7 @@ static int lnet_peer_rediscover(struct lnet_peer *lp) * obsessively re-check the clock. The oldest discovery request will * be at the head of the queue. */ -static struct lnet_peer *lnet_peer_dc_timed_out(time64_t now) +static struct lnet_peer *lnet_peer_get_dc_timed_out(time64_t now) { struct lnet_peer *lp; @@ -2950,7 +2950,7 @@ static struct lnet_peer *lnet_peer_dc_timed_out(time64_t now) return NULL; lp = list_first_entry(&the_lnet.ln_dc_working, struct lnet_peer, lp_dc_list); - if (now < lp->lp_last_queued + DEFAULT_PEER_TIMEOUT) + if (now < lp->lp_last_queued + lnet_transaction_timeout) return NULL; return lp; } @@ -2961,7 +2961,7 @@ static struct lnet_peer *lnet_peer_dc_timed_out(time64_t now) * lnet_discovery_event_handler() will proceed from here and complete * the cleanup. */ -static void lnet_peer_discovery_timeout(struct lnet_peer *lp) +static void lnet_peer_cancel_discovery(struct lnet_peer *lp) { struct lnet_handle_md ping_mdh; struct lnet_handle_md push_mdh; @@ -3010,7 +3010,7 @@ static int lnet_peer_discovery_wait_for_work(void) break; if (!list_empty(&the_lnet.ln_msg_resend)) break; - if (lnet_peer_dc_timed_out(ktime_get_real_seconds())) + if (lnet_peer_get_dc_timed_out(ktime_get_real_seconds())) break; lnet_net_unlock(cpt); @@ -3177,14 +3177,14 @@ static int lnet_peer_discovery(void *arg) * taking too long. Move all that are found to the * ln_dc_expired queue and time out any pending * Ping or Push. We have to drop the lnet_net_lock - * in the loop because lnet_peer_discovery_timeout() + * in the loop because lnet_peer_cancel_discovery() * calls LNetMDUnlink(). */ now = ktime_get_real_seconds(); - while ((lp = lnet_peer_dc_timed_out(now)) != NULL) { + while ((lp = lnet_peer_get_dc_timed_out(now)) != NULL) { list_move(&lp->lp_dc_list, &the_lnet.ln_dc_expired); lnet_net_unlock(LNET_LOCK_EX); - lnet_peer_discovery_timeout(lp); + lnet_peer_cancel_discovery(lp); lnet_net_lock(LNET_LOCK_EX); } @@ -3208,7 +3208,7 @@ static int lnet_peer_discovery(void *arg) struct lnet_peer, lp_dc_list); list_move(&lp->lp_dc_list, &the_lnet.ln_dc_expired); lnet_net_unlock(LNET_LOCK_EX); - lnet_peer_discovery_timeout(lp); + lnet_peer_cancel_discovery(lp); lnet_net_lock(LNET_LOCK_EX); } lnet_net_unlock(LNET_LOCK_EX); From patchwork Mon Sep 30 18:56:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167329 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E31091599 for ; Mon, 30 Sep 2019 19:08:11 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CAEAF224EF for ; Mon, 30 Sep 2019 19:08:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CAEAF224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 1B5DD21CA23; Mon, 30 Sep 2019 12:01:39 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 055B75C3CD0 for ; Mon, 30 Sep 2019 11:57:43 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id E11311005FA3; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id DF8C4B4; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:37 -0400 Message-Id: <1569869810-23848-139-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 138/151] lustre: update version to 2.10.99 X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" With all of the the missing patches from the lustre 2.11 version merged upstream its time to update the upstream clients version. Signed-off-by: James Simmons --- include/uapi/linux/lustre/lustre_ver.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/uapi/linux/lustre/lustre_ver.h b/include/uapi/linux/lustre/lustre_ver.h index e7a2eda..d7c53c5 100644 --- a/include/uapi/linux/lustre/lustre_ver.h +++ b/include/uapi/linux/lustre/lustre_ver.h @@ -2,10 +2,10 @@ #define _LUSTRE_VER_H_ #define LUSTRE_MAJOR 2 -#define LUSTRE_MINOR 9 +#define LUSTRE_MINOR 10 #define LUSTRE_PATCH 99 #define LUSTRE_FIX 0 -#define LUSTRE_VERSION_STRING "2.9.99" +#define LUSTRE_VERSION_STRING "2.10.99" #define OBD_OCD_VERSION(major, minor, patch, fix) \ (((major) << 24) + ((minor) << 16) + ((patch) << 8) + (fix)) From patchwork Mon Sep 30 18:56:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167265 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 95E0C13B1 for ; Mon, 30 Sep 2019 19:05:45 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7E17E224EF for ; Mon, 30 Sep 2019 19:05:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7E17E224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id AEF485E4AF2; Mon, 30 Sep 2019 12:00:41 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 4869D5C3CC9 for ; Mon, 30 Sep 2019 11:57:43 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id E42AA1005FA4; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id E2E4BB5; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:38 -0400 Message-Id: <1569869810-23848-140-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 139/151] lustre: ptlrpc: clarify 64 bit time usage X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: James Simmons , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" In the lustre code various small bits are left for the move to 64 bit time handling. This finishes some of the last bits left. To make clear that a value is in seconds change unsigned int values used for seconds to time64_t. Migrate scp_at_checktime to ktime. WC-bug-id: https://jira.whamcloud.com/browse/LU-9019 Lustre-commit: fdb5d3d508ed ("libcfs: remove the remaining cfs_time wrappers") Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/31068 Reviewed-by: Andreas Dilger Reviewed-by: Ben Evans Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_net.h | 4 ++-- fs/lustre/ldlm/ldlm_pool.c | 4 ++-- fs/lustre/ptlrpc/import.c | 2 +- fs/lustre/ptlrpc/service.c | 19 ++++++++++--------- net/lnet/libcfs/debug.c | 2 +- 5 files changed, 16 insertions(+), 15 deletions(-) diff --git a/fs/lustre/include/lustre_net.h b/fs/lustre/include/lustre_net.h index 22c9668..ffa6977 100644 --- a/fs/lustre/include/lustre_net.h +++ b/fs/lustre/include/lustre_net.h @@ -1558,7 +1558,7 @@ struct ptlrpc_service_part { struct list_head scp_rqbd_posted; /** incoming reqs */ struct list_head scp_req_incoming; - /** timeout before re-posting reqs, in tick */ + /** timeout before re-posting reqs, in jiffies */ long scp_rqbd_timeout; /** * all threads sleep on this. This wait-queue is signalled when new @@ -1611,7 +1611,7 @@ struct ptlrpc_service_part { /** early reply timer */ struct timer_list scp_at_timer; /** debug */ - unsigned long scp_at_checktime; + ktime_t scp_at_checktime; /** check early replies */ unsigned scp_at_check; /** @} */ diff --git a/fs/lustre/ldlm/ldlm_pool.c b/fs/lustre/ldlm/ldlm_pool.c index 25b1ada..04bf5de 100644 --- a/fs/lustre/ldlm/ldlm_pool.c +++ b/fs/lustre/ldlm/ldlm_pool.c @@ -887,7 +887,7 @@ static void ldlm_pools_recalc(struct work_struct *ws) struct ldlm_namespace *ns; struct ldlm_namespace *ns_old = NULL; /* seconds of sleep if no active namespaces */ - int time = LDLM_POOL_CLI_DEF_RECALC_PERIOD; + time64_t time = LDLM_POOL_CLI_DEF_RECALC_PERIOD; int nr; /* @@ -953,7 +953,7 @@ static void ldlm_pools_recalc(struct work_struct *ws) * After setup is done - recalc the pool. */ if (!skip) { - int ttime = ldlm_pool_recalc(&ns->ns_pool); + time64_t ttime = ldlm_pool_recalc(&ns->ns_pool); if (ttime < time) time = ttime; diff --git a/fs/lustre/ptlrpc/import.c b/fs/lustre/ptlrpc/import.c index 6fc0c6b..32bd218 100644 --- a/fs/lustre/ptlrpc/import.c +++ b/fs/lustre/ptlrpc/import.c @@ -242,7 +242,7 @@ void ptlrpc_deactivate_import(struct obd_import *imp) return dl - now; } -static unsigned int ptlrpc_inflight_timeout(struct obd_import *imp) +static time64_t ptlrpc_inflight_timeout(struct obd_import *imp) { time64_t now = ktime_get_real_seconds(); struct ptlrpc_request *req, *n; diff --git a/fs/lustre/ptlrpc/service.c b/fs/lustre/ptlrpc/service.c index 57c5e28..f07f549 100644 --- a/fs/lustre/ptlrpc/service.c +++ b/fs/lustre/ptlrpc/service.c @@ -342,7 +342,7 @@ static void ptlrpc_at_timer(struct timer_list *t) svcpt = from_timer(svcpt, t, scp_at_timer); svcpt->scp_at_check = 1; - svcpt->scp_at_checktime = jiffies; + svcpt->scp_at_checktime = ktime_get(); wake_up(&svcpt->scp_waitq); } @@ -933,7 +933,7 @@ static int ptlrpc_check_req(struct ptlrpc_request *req) static void ptlrpc_at_set_timer(struct ptlrpc_service_part *svcpt) { struct ptlrpc_at_array *array = &svcpt->scp_at_array; - s32 next; + time64_t next; if (array->paa_count == 0) { del_timer(&svcpt->scp_at_timer); @@ -941,13 +941,14 @@ static void ptlrpc_at_set_timer(struct ptlrpc_service_part *svcpt) } /* Set timer for closest deadline */ - next = (s32)(array->paa_deadline - ktime_get_real_seconds() - - at_early_margin); + next = array->paa_deadline - ktime_get_real_seconds() - + at_early_margin; if (next <= 0) { ptlrpc_at_timer(&svcpt->scp_at_timer); } else { - mod_timer(&svcpt->scp_at_timer, jiffies + next * HZ); - CDEBUG(D_INFO, "armed %s at %+ds\n", + mod_timer(&svcpt->scp_at_timer, + jiffies + nsecs_to_jiffies(next * NSEC_PER_SEC)); + CDEBUG(D_INFO, "armed %s at %+llds\n", svcpt->scp_service->srv_name, next); } } @@ -1181,7 +1182,7 @@ static void ptlrpc_at_check_timed(struct ptlrpc_service_part *svcpt) u32 index, count; time64_t deadline; time64_t now = ktime_get_real_seconds(); - long delay; + s64 delay; int first, counter = 0; spin_lock(&svcpt->scp_at_lock); @@ -1189,7 +1190,7 @@ static void ptlrpc_at_check_timed(struct ptlrpc_service_part *svcpt) spin_unlock(&svcpt->scp_at_lock); return; } - delay = jiffies - svcpt->scp_at_checktime; + delay = ktime_ms_delta(ktime_get(), svcpt->scp_at_checktime); svcpt->scp_at_check = 0; if (array->paa_count == 0) { @@ -1254,7 +1255,7 @@ static void ptlrpc_at_check_timed(struct ptlrpc_service_part *svcpt) */ LCONSOLE_WARN("%s: This server is not able to keep up with request traffic (cpu-bound).\n", svcpt->scp_service->srv_name); - CWARN("earlyQ=%d reqQ=%d recA=%d, svcEst=%d, delay=%ld(jiff)\n", + CWARN("earlyQ=%d reqQ=%d recA=%d, svcEst=%d, delay=%lld\n", counter, svcpt->scp_nreqs_incoming, svcpt->scp_nreqs_active, at_get(&svcpt->scp_at_estimate), delay); diff --git a/net/lnet/libcfs/debug.c b/net/lnet/libcfs/debug.c index fd8ed81..6b8f783e6 100644 --- a/net/lnet/libcfs/debug.c +++ b/net/lnet/libcfs/debug.c @@ -129,7 +129,7 @@ static int param_get_delay(char *buffer, const struct kernel_param *kp) { unsigned int d = *(unsigned int *)kp->arg; - return sprintf(buffer, "%u", (unsigned int)(d * 100) / HZ); + return sprintf(buffer, "%lu", jiffies_to_msecs(d * 10) / MSEC_PER_SEC); } unsigned int libcfs_console_max_delay; From patchwork Mon Sep 30 18:56:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167271 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 44CF013B1 for ; Mon, 30 Sep 2019 19:05:54 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2D0552168B for ; Mon, 30 Sep 2019 19:05:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2D0552168B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 6B12B5E4B40; Mon, 30 Sep 2019 12:00:45 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id A297E5C3CCF for ; Mon, 30 Sep 2019 11:57:43 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id E6CEC1005FA5; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id E5DD0A9; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:39 -0400 Message-Id: <1569869810-23848-141-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 140/151] lustre: ptlrpc: add watchdog for ptlrpc service threads. X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: James Simmons , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: NeilBrown If a ptlrpc thread takes longer than expected, it can be used to get a stack trace to see where the problem is. This patch restores this functionality by using a delayed work struct to report a back trace of the thread if it ever fires. WC-bug-id: https://jira.whamcloud.com/browse/LU-9859 Lustre-commit: fc9de679a4c2 ("LU-9859 ptlrpc: add watchdog for ptlrpc service threads") Signed-off-by: NeilBrown Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/33018 Reviewed-by: Andreas Dilger Reviewed-by: Ben Evans Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_net.h | 11 ++---- fs/lustre/ptlrpc/service.c | 79 ++++++++++++++++++++++++++++++------- include/linux/libcfs/libcfs_debug.h | 1 + net/lnet/libcfs/debug.c | 3 ++ net/lnet/libcfs/module.c | 12 ++++++ 5 files changed, 84 insertions(+), 22 deletions(-) diff --git a/fs/lustre/include/lustre_net.h b/fs/lustre/include/lustre_net.h index ffa6977..a8aa327 100644 --- a/fs/lustre/include/lustre_net.h +++ b/fs/lustre/include/lustre_net.h @@ -1298,18 +1298,15 @@ struct ptlrpc_thread { */ unsigned int t_id; /** - * service thread pid + * service thread */ + struct task_struct *t_task; + ktime_t t_touched; pid_t t_pid; /** * put watchdog in the structure per thread b=14840 - * - * Lustre watchdog is removed for client in the hope - * of a generic watchdog can be merged in kernel. - * When that happens, we should add below back. - * - * struct lc_watchdog *t_watchdog; */ + struct delayed_work t_watchdog; /** * the svc this thread belonged to b=18582 */ diff --git a/fs/lustre/ptlrpc/service.c b/fs/lustre/ptlrpc/service.c index f07f549..e0cce15 100644 --- a/fs/lustre/ptlrpc/service.c +++ b/fs/lustre/ptlrpc/service.c @@ -34,7 +34,9 @@ #define DEBUG_SUBSYSTEM S_RPC #include +#include #include + #include #include #include @@ -1977,6 +1979,57 @@ static bool ptlrpc_server_normal_pending(struct ptlrpc_service_part *svcpt, return svcpt->scp_at_check; } +/* + * If a thread runs too long or spends to much time on a single request, + * we want to know about it, so we set up a delayed work item as a watchdog. + * If it fires, we display a stack trace of the delayed thread, + * providing we aren't rate-limited + * + * Watchdog stack traces are limited to 3 per 'libcfs_watchdog_ratelimit' + * seconds + */ +static struct ratelimit_state watchdog_limit; + +static void ptlrpc_watchdog_fire(struct work_struct *w) +{ + struct ptlrpc_thread *thread = container_of(w, struct ptlrpc_thread, + t_watchdog.work); + s64 ms_lapse = ktime_ms_delta(ktime_get(), thread->t_touched); + u32 ms_frac = do_div(ms_lapse, MSEC_PER_SEC); + + if (!__ratelimit(&watchdog_limit)) { + LCONSOLE_WARN("%s: service thread pid %u was inactive for %llu.%.03u seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes:\n", + thread->t_task->comm, thread->t_task->pid, + ms_lapse, ms_frac); + + libcfs_debug_dumpstack(thread->t_task); + } else { + LCONSOLE_WARN("%s: service thread pid %u was inactive for %llu.%.03u seconds. Watchdog stack traces are limited to 3 per %u seconds, skipping this one.\n", + thread->t_task->comm, thread->t_task->pid, + ms_lapse, ms_frac, libcfs_watchdog_ratelimit); + } +} + +static void ptlrpc_watchdog_init(struct delayed_work *work, time_t time) +{ + INIT_DELAYED_WORK(work, ptlrpc_watchdog_fire); + schedule_delayed_work(work, time * HZ); +} + +static void ptlrpc_watchdog_disable(struct delayed_work *work) +{ + cancel_delayed_work_sync(work); +} + +static void ptlrpc_watchdog_touch(struct delayed_work *work, time_t time) +{ + struct ptlrpc_thread *thread = container_of(&work->work, + struct ptlrpc_thread, + t_watchdog.work); + thread->t_touched = ktime_get(); + mod_delayed_work(system_wq, work, time * HZ); +} + /** * requests wait on preprocessing * user can call it w/o any lock but need to hold @@ -2000,9 +2053,7 @@ static __attribute__((__noinline__)) int { /* Don't exit while there are replies to be handled */ - /* XXX: Add this back when libcfs watchdog is merged upstream - lc_watchdog_disable(thread->t_watchdog); - */ + ptlrpc_watchdog_disable(&thread->t_watchdog); cond_resched(); @@ -2029,10 +2080,8 @@ static __attribute__((__noinline__)) int if (ptlrpc_thread_stopping(thread)) return -EINTR; - /* - lc_watchdog_touch(thread->t_watchdog, - ptlrpc_server_get_timeout(svcpt)); - */ + ptlrpc_watchdog_touch(&thread->t_watchdog, + ptlrpc_server_get_timeout(svcpt)); return 0; } @@ -2052,6 +2101,7 @@ static int ptlrpc_main(void *arg) struct lu_env *env; int counter = 0, rc = 0; + thread->t_task = current; thread->t_pid = current->pid; unshare_fs_struct(); @@ -2132,10 +2182,9 @@ static int ptlrpc_main(void *arg) /* wake up our creator in case he's still waiting. */ wake_up(&thread->t_ctl_waitq); - /* - thread->t_watchdog = lc_watchdog_add(ptlrpc_server_get_timeout(svcpt), - NULL, NULL); - */ + thread->t_touched = ktime_get(); + ptlrpc_watchdog_init(&thread->t_watchdog, + ptlrpc_server_get_timeout(svcpt)); spin_lock(&svcpt->scp_rep_lock); list_add(&rs->rs_list, &svcpt->scp_rep_idle); @@ -2191,10 +2240,7 @@ static int ptlrpc_main(void *arg) } } - /* - lc_watchdog_delete(thread->t_watchdog); - thread->t_watchdog = NULL; - */ + ptlrpc_watchdog_disable(&thread->t_watchdog); out_srv_fini: /* @@ -2559,6 +2605,9 @@ int ptlrpc_hr_init(void) if (!ptlrpc_hr.hr_partitions) return -ENOMEM; + ratelimit_state_init(&watchdog_limit, libcfs_watchdog_ratelimit * HZ, + 3); + init_waitqueue_head(&ptlrpc_hr.hr_waitq); preempt_disable(); diff --git a/include/linux/libcfs/libcfs_debug.h b/include/linux/libcfs/libcfs_debug.h index 89deab4..ed00320 100644 --- a/include/linux/libcfs/libcfs_debug.h +++ b/include/linux/libcfs/libcfs_debug.h @@ -49,6 +49,7 @@ extern unsigned int libcfs_stack; extern unsigned int libcfs_debug; extern unsigned int libcfs_printk; +extern unsigned int libcfs_watchdog_ratelimit; extern unsigned int libcfs_console_ratelimit; extern unsigned int libcfs_console_max_delay; extern unsigned int libcfs_console_min_delay; diff --git a/net/lnet/libcfs/debug.c b/net/lnet/libcfs/debug.c index 6b8f783e6..e779d6b 100644 --- a/net/lnet/libcfs/debug.c +++ b/net/lnet/libcfs/debug.c @@ -214,6 +214,9 @@ static int param_set_uintpos(const char *val, const struct kernel_param *kp) unsigned int libcfs_catastrophe; EXPORT_SYMBOL(libcfs_catastrophe); +unsigned int libcfs_watchdog_ratelimit = 300; +EXPORT_SYMBOL(libcfs_watchdog_ratelimit); + unsigned int libcfs_panic_on_lbug = 1; module_param(libcfs_panic_on_lbug, uint, 0644); MODULE_PARM_DESC(libcfs_panic_on_lbug, "Lustre kernel panic on LBUG"); diff --git a/net/lnet/libcfs/module.c b/net/lnet/libcfs/module.c index 49731bd..bee2581 100644 --- a/net/lnet/libcfs/module.c +++ b/net/lnet/libcfs/module.c @@ -336,6 +336,9 @@ static int proc_dobitmasks(struct ctl_table *table, int write, return rc; } +static int min_watchdog_ratelimit; /* disable ratelimiting */ +static int max_watchdog_ratelimit = (24 * 60 * 60); /* limit to once per day */ + static int proc_dump_kernel(struct ctl_table *table, int write, void __user *buffer, size_t *lenp, loff_t *ppos) { @@ -531,6 +534,15 @@ static int proc_cpt_distance(struct ctl_table *table, int write, .proc_handler = &proc_daemon_file, }, { + .procname = "watchdog_ratelimit", + .data = &libcfs_watchdog_ratelimit, + .maxlen = sizeof(int), + .mode = 0644, + .proc_handler = &proc_dointvec_minmax, + .extra1 = &min_watchdog_ratelimit, + .extra2 = &max_watchdog_ratelimit, + }, + { .procname = "force_lbug", .data = NULL, .maxlen = 0, From patchwork Mon Sep 30 18:56:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167275 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E123D16B1 for ; Mon, 30 Sep 2019 19:06:01 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C9D8F2245A for ; Mon, 30 Sep 2019 19:06:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C9D8F2245A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id D9F525E4BA1; Mon, 30 Sep 2019 12:00:48 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 0A99C5C3CCF for ; Mon, 30 Sep 2019 11:57:44 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id E9BE21005FA6; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id E8D35BB; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:40 -0400 Message-Id: <1569869810-23848-142-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 141/151] lustre: handles: discard h_owner in favour of h_ops X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: NeilBrown lustre_handles assigned a 64bit unique identifier (a 'cookie') to objects of various types and stored them in a hash table, allowing them to be accessed by the cookie. There is a facility for type checking by recording an 'owner' for each object, and checking the owner on lookup. Unfortunately this is not used - owner is always zero for the client. Each object also contains an h_ops pointer which can be used to reliably identify an owner. So discard h_owner, pass and 'ops' pointer to class_handle2object(), and only return objects for which the h_ops matches. Note: this h_owner is now quiet different from the similar h_owner in the server code. When the server code is merged the "med" pointer should be stored in the "mfd" and validated separately. This reduces the size of the portals_handle by one pointer, which benefits various other structures including struct ldlm_lock which can be very populous and so is best keep small. WC-bug-id: https://jira.whamcloud.com/browse/LU-12542 Lustre-commit: 9bda0a5cad19 ("LU-12542 handles: discard h_owner in favour of h_ops") Signed-off-by: NeilBrown Reviewed-on: https://review.whamcloud.com/35739 Reviewed-by: Andreas Dilger Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_handles.h | 7 +++---- fs/lustre/ldlm/ldlm_lock.c | 2 +- fs/lustre/obdclass/genops.c | 4 +++- fs/lustre/obdclass/lustre_handles.c | 6 +++--- 4 files changed, 10 insertions(+), 9 deletions(-) diff --git a/fs/lustre/include/lustre_handles.h b/fs/lustre/include/lustre_handles.h index d4ec795..0440970 100644 --- a/fs/lustre/include/lustre_handles.h +++ b/fs/lustre/include/lustre_handles.h @@ -65,8 +65,7 @@ struct portals_handle_ops { struct portals_handle { struct list_head h_link; u64 h_cookie; - const void *h_owner; - struct portals_handle_ops *h_ops; + const struct portals_handle_ops *h_ops; /* newly added fields to handle the RCU issue. -jxiong */ struct rcu_head h_rcu; @@ -79,9 +78,9 @@ struct portals_handle { /* Add a handle to the hash table */ void class_handle_hash(struct portals_handle *, - struct portals_handle_ops *ops); + const struct portals_handle_ops *ops); void class_handle_unhash(struct portals_handle *); -void *class_handle2object(u64 cookie, const void *owner); +void *class_handle2object(u64 cookie, const struct portals_handle_ops *ops); void class_handle_free_cb(struct rcu_head *rcu); int class_handle_init(void); void class_handle_cleanup(void); diff --git a/fs/lustre/ldlm/ldlm_lock.c b/fs/lustre/ldlm/ldlm_lock.c index f0657d9..6eebf5f 100644 --- a/fs/lustre/ldlm/ldlm_lock.c +++ b/fs/lustre/ldlm/ldlm_lock.c @@ -518,7 +518,7 @@ struct ldlm_lock *__ldlm_handle2lock(const struct lustre_handle *handle, LASSERT(handle); - lock = class_handle2object(handle->cookie, NULL); + lock = class_handle2object(handle->cookie, &lock_handle_ops); if (!lock) return NULL; diff --git a/fs/lustre/obdclass/genops.c b/fs/lustre/obdclass/genops.c index 5c9b3c4..fc50aba 100644 --- a/fs/lustre/obdclass/genops.c +++ b/fs/lustre/obdclass/genops.c @@ -680,6 +680,8 @@ int obd_init_caches(void) return -ENOMEM; } +static struct portals_handle_ops export_handle_ops; + /* map connection to client */ struct obd_export *class_conn2export(struct lustre_handle *conn) { @@ -696,7 +698,7 @@ struct obd_export *class_conn2export(struct lustre_handle *conn) } CDEBUG(D_INFO, "looking for export cookie %#llx\n", conn->cookie); - export = class_handle2object(conn->cookie, NULL); + export = class_handle2object(conn->cookie, &export_handle_ops); return export; } EXPORT_SYMBOL(class_conn2export); diff --git a/fs/lustre/obdclass/lustre_handles.c b/fs/lustre/obdclass/lustre_handles.c index 7c63d9c..7fa3ef6 100644 --- a/fs/lustre/obdclass/lustre_handles.c +++ b/fs/lustre/obdclass/lustre_handles.c @@ -59,7 +59,7 @@ * global (per-node) hash-table. */ void class_handle_hash(struct portals_handle *h, - struct portals_handle_ops *ops) + const struct portals_handle_ops *ops) { struct handle_bucket *bucket; @@ -132,7 +132,7 @@ void class_handle_unhash(struct portals_handle *h) } EXPORT_SYMBOL(class_handle_unhash); -void *class_handle2object(u64 cookie, const void *owner) +void *class_handle2object(u64 cookie, const struct portals_handle_ops *ops) { struct handle_bucket *bucket; struct portals_handle *h; @@ -147,7 +147,7 @@ void *class_handle2object(u64 cookie, const void *owner) rcu_read_lock(); list_for_each_entry_rcu(h, &bucket->head, h_link) { - if (h->h_cookie != cookie || h->h_owner != owner) + if (h->h_cookie != cookie || h->h_ops != ops) continue; spin_lock(&h->h_lock); From patchwork Mon Sep 30 18:56:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167279 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CD08116B1 for ; Mon, 30 Sep 2019 19:06:10 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B5521224D2 for ; Mon, 30 Sep 2019 19:06:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B5521224D2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 01ABD5E4BFE; Mon, 30 Sep 2019 12:00:51 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 650E35C3BB1 for ; Mon, 30 Sep 2019 11:57:44 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id EEA871005FA7; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id ED3FAB4; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:41 -0400 Message-Id: <1569869810-23848-143-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 142/151] lustre: ldlm: Remove use of SLAB_DESTROY_BY_RCU for ldlm lock slab X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: NeilBrown ldlm_locks are always freed with kfree_rcu(), so there is no need for the underlying pages to also be freed after a grace period. So remove this flag, it is not useful. WC-bug-id: https://jira.whamcloud.com/browse/LU-11568 Lustre-commit: 82d014e71e14 ("LU-11568 ldlm: Remove use of SLAB_DESTROY_BY_RCU for ldlm lock slab") Signed-off-by: NeilBrown Signed-off-by: Oleg Drokin Reviewed-on: https://review.whamcloud.com/34147 Reviewed-by: Andreas Dilger Reviewed-by: James Simmons Signed-off-by: James Simmons --- fs/lustre/ldlm/ldlm_lockd.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/fs/lustre/ldlm/ldlm_lockd.c b/fs/lustre/ldlm/ldlm_lockd.c index 7e1ae0c..6ad2c1a 100644 --- a/fs/lustre/ldlm/ldlm_lockd.c +++ b/fs/lustre/ldlm/ldlm_lockd.c @@ -1115,8 +1115,7 @@ int ldlm_init(void) ldlm_lock_slab = kmem_cache_create("ldlm_locks", sizeof(struct ldlm_lock), 0, - SLAB_HWCACHE_ALIGN | - SLAB_TYPESAFE_BY_RCU, NULL); + SLAB_HWCACHE_ALIGN, NULL); if (!ldlm_lock_slab) goto out_resource; From patchwork Mon Sep 30 18:56:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167343 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 69DFF13B1 for ; Mon, 30 Sep 2019 19:08:41 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 51FD6224EF for ; Mon, 30 Sep 2019 19:08:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 51FD6224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 0DC896037C0; Mon, 30 Sep 2019 12:01:57 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id A61765C3CDE for ; Mon, 30 Sep 2019 11:57:44 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id F25C71005FA8; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id F0469B5; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:42 -0400 Message-Id: <1569869810-23848-144-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 143/151] lustre: ldlm: simplify lock_mode_to_index() X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: NeilBrown This function has the same effect as ilog2(), so just use ilog2 directly. WC-bug-id: https://jira.whamcloud.com/browse/LU-12542 Lustre-commit: 32ae82e57cf8 ("LU-12542 ldlm: simplify lock_mode_to_index()") Signed-off-by: NeilBrown Reviewed-on: https://review.whamcloud.com/35486 Reviewed-by: Andreas Dilger Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/ldlm/ldlm_extent.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/fs/lustre/ldlm/ldlm_extent.c b/fs/lustre/ldlm/ldlm_extent.c index 77d3e79..7c72d04 100644 --- a/fs/lustre/ldlm/ldlm_extent.c +++ b/fs/lustre/ldlm/ldlm_extent.c @@ -139,8 +139,7 @@ static inline int lock_mode_to_index(enum ldlm_mode mode) LASSERT(mode != 0); LASSERT(is_power_of_2(mode)); - for (index = -1; mode; index++) - mode >>= 1; + index = ilog2(mode); LASSERT(index < LCK_MODE_NUM); return index; } From patchwork Mon Sep 30 18:56:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167283 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1E1D31599 for ; Mon, 30 Sep 2019 19:06:22 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 06BF6224D2 for ; Mon, 30 Sep 2019 19:06:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 06BF6224D2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 3069C5E4C5A; Mon, 30 Sep 2019 12:00:55 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id E6B835C3CE4 for ; Mon, 30 Sep 2019 11:57:44 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 00F691005FA9; Mon, 30 Sep 2019 14:56:58 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id F3843A9; Mon, 30 Sep 2019 14:56:57 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:43 -0400 Message-Id: <1569869810-23848-145-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 144/151] lustre: ptlrpc: use list_move where appropriate. X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: NeilBrown There are several places in lustre where "list_del" (or occasionally "list_del_init") is followed by "list_add" or "list_add_tail" which moves the object to a different list. These can be combined into "list_move" or "list_move_tail". WC-bug-id: https://jira.whamcloud.com/browse/LU-4423 Lustre-commit: a71948a4e6de ("LU-4423 ptlrpc: use list_move where appropriate") Signed-off-by: NeilBrown Reviewed-on: https://review.whamcloud.com/35507 Reviewed-by: Andreas Dilger Signed-off-by: James Simmons --- fs/lustre/ldlm/ldlm_lib.c | 5 ++--- fs/lustre/ldlm/ldlm_request.c | 3 +-- fs/lustre/ptlrpc/client.c | 10 ++++------ fs/lustre/ptlrpc/import.c | 3 +-- fs/lustre/ptlrpc/service.c | 13 ++++--------- 5 files changed, 12 insertions(+), 22 deletions(-) diff --git a/fs/lustre/ldlm/ldlm_lib.c b/fs/lustre/ldlm/ldlm_lib.c index 45dc70d..7bc1d10 100644 --- a/fs/lustre/ldlm/ldlm_lib.c +++ b/fs/lustre/ldlm/ldlm_lib.c @@ -86,9 +86,8 @@ static int import_set_conn(struct obd_import *imp, struct obd_uuid *uuid, list_for_each_entry(item, &imp->imp_conn_list, oic_item) { if (obd_uuid_equals(uuid, &item->oic_uuid)) { if (priority) { - list_del(&item->oic_item); - list_add(&item->oic_item, - &imp->imp_conn_list); + list_move(&item->oic_item, + &imp->imp_conn_list); item->oic_last_attempt = 0; } CDEBUG(D_HA, "imp %p@%s: found existing conn %s%s\n", diff --git a/fs/lustre/ldlm/ldlm_request.c b/fs/lustre/ldlm/ldlm_request.c index 4c6db33..00336ca 100644 --- a/fs/lustre/ldlm/ldlm_request.c +++ b/fs/lustre/ldlm/ldlm_request.c @@ -1132,8 +1132,7 @@ int ldlm_cli_cancel_list_local(struct list_head *cancels, int count, */ if (!(flags & LCF_BL_AST) && (rc == LDLM_FL_BL_AST)) { LDLM_DEBUG(lock, "Cancel lock separately"); - list_del_init(&lock->l_bl_ast); - list_add(&lock->l_bl_ast, &head); + list_move(&lock->l_bl_ast, &head); bl_ast++; continue; } diff --git a/fs/lustre/ptlrpc/client.c b/fs/lustre/ptlrpc/client.c index e3aee32..6055980 100644 --- a/fs/lustre/ptlrpc/client.c +++ b/fs/lustre/ptlrpc/client.c @@ -1833,9 +1833,8 @@ int ptlrpc_check_set(const struct lu_env *env, struct ptlrpc_request_set *set) * put on delay list - only if we wait * recovery finished - before send */ - list_del_init(&req->rq_list); - list_add_tail(&req->rq_list, - &imp->imp_delayed_list); + list_move_tail(&req->rq_list, + &imp->imp_delayed_list); spin_unlock(&imp->imp_lock); continue; } @@ -1856,9 +1855,8 @@ int ptlrpc_check_set(const struct lu_env *env, struct ptlrpc_request_set *set) goto interpret; } - list_del_init(&req->rq_list); - list_add_tail(&req->rq_list, - &imp->imp_sending_list); + list_move_tail(&req->rq_list, + &imp->imp_sending_list); spin_unlock(&imp->imp_lock); diff --git a/fs/lustre/ptlrpc/import.c b/fs/lustre/ptlrpc/import.c index 32bd218..eb19ca6 100644 --- a/fs/lustre/ptlrpc/import.c +++ b/fs/lustre/ptlrpc/import.c @@ -753,8 +753,7 @@ static int ptlrpc_connect_set_flags(struct obd_import *imp, static bool warned; spin_lock(&imp->imp_lock); - list_del(&imp->imp_conn_current->oic_item); - list_add(&imp->imp_conn_current->oic_item, &imp->imp_conn_list); + list_move(&imp->imp_conn_current->oic_item, &imp->imp_conn_list); imp->imp_last_success_conn = imp->imp_conn_current->oic_last_attempt; spin_unlock(&imp->imp_lock); diff --git a/fs/lustre/ptlrpc/service.c b/fs/lustre/ptlrpc/service.c index e0cce15..c2a00de 100644 --- a/fs/lustre/ptlrpc/service.c +++ b/fs/lustre/ptlrpc/service.c @@ -307,11 +307,10 @@ void ptlrpc_dispatch_difficult_reply(struct ptlrpc_reply_state *rs) rqbd = list_first_entry(&svcpt->scp_rqbd_idle, struct ptlrpc_request_buffer_desc, rqbd_list); - list_del(&rqbd->rqbd_list); /* assume we will post successfully */ svcpt->scp_nrqbds_posted++; - list_add(&rqbd->rqbd_list, &svcpt->scp_rqbd_posted); + list_move(&rqbd->rqbd_list, &svcpt->scp_rqbd_posted); spin_unlock(&svcpt->scp_lock); @@ -325,8 +324,7 @@ void ptlrpc_dispatch_difficult_reply(struct ptlrpc_reply_state *rs) spin_lock(&svcpt->scp_lock); svcpt->scp_nrqbds_posted--; - list_del(&rqbd->rqbd_list); - list_add_tail(&rqbd->rqbd_list, &svcpt->scp_rqbd_idle); + list_move_tail(&rqbd->rqbd_list, &svcpt->scp_rqbd_idle); /* Don't complain if no request buffers are posted right now; LNET * won't drop requests because we set the portal lazy! @@ -769,9 +767,7 @@ static void ptlrpc_server_drop_request(struct ptlrpc_request *req) refcount = --(rqbd->rqbd_refcount); if (refcount == 0) { /* request buffer is now idle: add to history */ - list_del(&rqbd->rqbd_list); - - list_add_tail(&rqbd->rqbd_list, &svcpt->scp_hist_rqbds); + list_move_tail(&rqbd->rqbd_list, &svcpt->scp_hist_rqbds); svcpt->scp_hist_nrqbds++; /* cull some history? @@ -2416,8 +2412,7 @@ static void ptlrpc_svcpt_stop_threads(struct ptlrpc_service_part *svcpt) struct ptlrpc_thread, t_link)) != NULL) { if (thread_is_stopped(thread)) { - list_del(&thread->t_link); - list_add(&thread->t_link, &zombie); + list_move(&thread->t_link, &zombie); continue; } spin_unlock(&svcpt->scp_lock); From patchwork Mon Sep 30 18:56:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167345 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 90FB31599 for ; Mon, 30 Sep 2019 19:08:48 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 79369224EF for ; Mon, 30 Sep 2019 19:08:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 79369224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id AF74921E08E; Mon, 30 Sep 2019 12:02:00 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 49CDA5C3CEB for ; Mon, 30 Sep 2019 11:57:45 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 0386E1005FAA; Mon, 30 Sep 2019 14:56:58 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 02436BB; Mon, 30 Sep 2019 14:56:58 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:44 -0400 Message-Id: <1569869810-23848-146-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 145/151] lustre: ptlrpc: simplify locking in ptlrpc_add_rqs_to_pool() X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: NeilBrown This spinlock is only needed in this function to protect the list_add_tail(), so make that more obvious. This change means that prp_rq_size is set outside of the spinlock, but as it is only ever read outside of the spinlock, that cannot matter. WC-bug-id: https://jira.whamcloud.com/browse/LU-4423 Lustre-commit: a0240c3ef723 ("LU-4423 ptlrpc: simplify locking in ptlrpc_add_rqs_to_pool()") Signed-off-by: NeilBrown Reviewed-on: https://review.whamcloud.com/35511 Reviewed-by: Petros Koutoupis Reviewed-by: Shaun Tancheff Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/ptlrpc/client.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/fs/lustre/ptlrpc/client.c b/fs/lustre/ptlrpc/client.c index 6055980..8328c71 100644 --- a/fs/lustre/ptlrpc/client.c +++ b/fs/lustre/ptlrpc/client.c @@ -566,13 +566,11 @@ int ptlrpc_add_rqs_to_pool(struct ptlrpc_request_pool *pool, int num_rq) "Trying to change pool size with nonempty pool from %d to %d bytes\n", pool->prp_rq_size, size); - spin_lock(&pool->prp_lock); pool->prp_rq_size = size; for (i = 0; i < num_rq; i++) { struct ptlrpc_request *req; struct lustre_msg *msg; - spin_unlock(&pool->prp_lock); req = ptlrpc_request_cache_alloc(GFP_KERNEL); if (!req) return i; @@ -586,8 +584,8 @@ int ptlrpc_add_rqs_to_pool(struct ptlrpc_request_pool *pool, int num_rq) req->rq_pool = pool; spin_lock(&pool->prp_lock); list_add_tail(&req->rq_list, &pool->prp_req_list); + spin_unlock(&pool->prp_lock); } - spin_unlock(&pool->prp_lock); return num_rq; } EXPORT_SYMBOL(ptlrpc_add_rqs_to_pool); From patchwork Mon Sep 30 18:56:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167333 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 15E2513B1 for ; Mon, 30 Sep 2019 19:08:19 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F2467224EF for ; Mon, 30 Sep 2019 19:08:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F2467224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 7A418602CF3; Mon, 30 Sep 2019 12:01:42 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8B04C5C3CEB for ; Mon, 30 Sep 2019 11:57:45 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 067701006243; Mon, 30 Sep 2019 14:56:58 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 0549BB4; Mon, 30 Sep 2019 14:56:58 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:45 -0400 Message-Id: <1569869810-23848-147-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 146/151] lustre: ptlrpc: incorporate BUILD_BUG_ON into ptlrpc_req_async_args() X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: NeilBrown Every call to ptlrpc_req_async_args() should be preceded by a BUILD_BUG_ON() (aka LASSERT()), though a few aren't. To improve maintainability, include the BUILD_BUG_ON into the ptlrpc_req_async_args() macro. WC-bug-id: https://jira.whamcloud.com/browse/LU-4423 Lustre-commit: b56177451975 ("LU-4423 ptlrpc: incorporate BUILD_BUG_ON into ptlrpc_req_async_args()") Signed-off-by: NeilBrown Reviewed-on: https://review.whamcloud.com/35509 Reviewed-by: Mike Pershin Reviewed-by: Petros Koutoupis Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_net.h | 7 +++++-- fs/lustre/ldlm/ldlm_request.c | 3 +-- fs/lustre/mdc/mdc_dev.c | 5 ++--- fs/lustre/mdc/mdc_locks.c | 7 +++---- fs/lustre/osc/osc_io.c | 3 +-- fs/lustre/osc/osc_request.c | 29 ++++++++++------------------- fs/lustre/ptlrpc/client.c | 6 ++---- fs/lustre/ptlrpc/import.c | 3 +-- 8 files changed, 25 insertions(+), 38 deletions(-) diff --git a/fs/lustre/include/lustre_net.h b/fs/lustre/include/lustre_net.h index a8aa327..8024367 100644 --- a/fs/lustre/include/lustre_net.h +++ b/fs/lustre/include/lustre_net.h @@ -309,8 +309,11 @@ #define OBD_MAX_SHORT_IO_BYTES (min(max(PAGE_SIZE, 16UL * 1024UL), \ OST_SHORT_IO_SPACE & PAGE_MASK)) -/* Macro to hide a typecast. */ -#define ptlrpc_req_async_args(req) ((void *)&req->rq_async_args) +/* Macro to hide a typecast and BUILD_BUG. */ +#define ptlrpc_req_async_args(_var, req) ({ \ + BUILD_BUG_ON(sizeof(*_var) > sizeof(req->rq_async_args)); \ + (typeof(_var))&req->rq_async_args; \ + }) struct ptlrpc_replay_async_args { int praa_old_state; diff --git a/fs/lustre/ldlm/ldlm_request.c b/fs/lustre/ldlm/ldlm_request.c index 00336ca..92e4f69 100644 --- a/fs/lustre/ldlm/ldlm_request.c +++ b/fs/lustre/ldlm/ldlm_request.c @@ -2016,8 +2016,7 @@ static int replay_one_lock(struct obd_import *imp, struct ldlm_lock *lock) LDLM_DEBUG(lock, "replaying lock:"); atomic_inc(&req->rq_import->imp_replay_inflight); - BUILD_BUG_ON(sizeof(*aa) > sizeof(req->rq_async_args)); - aa = ptlrpc_req_async_args(req); + aa = ptlrpc_req_async_args(aa, req); aa->lock_handle = body->lock_handle[0]; req->rq_interpret_reply = (ptlrpc_interpterer_t)replay_lock_interpret; ptlrpcd_add_req(req); diff --git a/fs/lustre/mdc/mdc_dev.c b/fs/lustre/mdc/mdc_dev.c index 22749ea..ca0822d 100644 --- a/fs/lustre/mdc/mdc_dev.c +++ b/fs/lustre/mdc/mdc_dev.c @@ -733,7 +733,7 @@ int mdc_enqueue_send(const struct lu_env *env, struct obd_export *exp, if (!rc) { struct osc_enqueue_args *aa; - aa = ptlrpc_req_async_args(req); + aa = ptlrpc_req_async_args(aa, req); aa->oa_exp = exp; aa->oa_mode = einfo->ei_mode; aa->oa_type = einfo->ei_type; @@ -1195,8 +1195,7 @@ static int mdc_io_data_version_start(const struct lu_env *env, ptlrpc_request_set_replen(req); req->rq_interpret_reply = mdc_data_version_interpret; - BUILD_BUG_ON(sizeof(*dva) > sizeof(req->rq_async_args)); - dva = ptlrpc_req_async_args(req); + dva = ptlrpc_req_async_args(dva, req); dva->dva_oio = oio; ptlrpcd_add_req(req); diff --git a/fs/lustre/mdc/mdc_locks.c b/fs/lustre/mdc/mdc_locks.c index 123f0f0..2e4a5c6 100644 --- a/fs/lustre/mdc/mdc_locks.c +++ b/fs/lustre/mdc/mdc_locks.c @@ -49,8 +49,8 @@ #include "mdc_internal.h" struct mdc_getattr_args { - struct obd_export *ga_exp; - struct md_enqueue_info *ga_minfo; + struct obd_export *ga_exp; + struct md_enqueue_info *ga_minfo; }; int it_open_error(int phase, struct lookup_intent *it) @@ -1329,8 +1329,7 @@ int mdc_intent_getattr_async(struct obd_export *exp, return rc; } - BUILD_BUG_ON(sizeof(*ga) > sizeof(req->rq_async_args)); - ga = ptlrpc_req_async_args(req); + ga = ptlrpc_req_async_args(ga, req); ga->ga_exp = exp; ga->ga_minfo = minfo; diff --git a/fs/lustre/osc/osc_io.c b/fs/lustre/osc/osc_io.c index 98726cd..970e8a7 100644 --- a/fs/lustre/osc/osc_io.c +++ b/fs/lustre/osc/osc_io.c @@ -689,8 +689,7 @@ static int osc_io_data_version_start(const struct lu_env *env, ptlrpc_request_set_replen(req); req->rq_interpret_reply = osc_data_version_interpret; - BUILD_BUG_ON(sizeof(*dva) > sizeof(req->rq_async_args)); - dva = ptlrpc_req_async_args(req); + dva = ptlrpc_req_async_args(dva, req); dva->dva_oio = oio; ptlrpcd_add_req(req); diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index 0a3d05e..300dee5 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -236,8 +236,7 @@ int osc_setattr_async(struct obd_export *exp, struct obdo *oa, req->rq_interpret_reply = (ptlrpc_interpterer_t)osc_setattr_interpret; - BUILD_BUG_ON(sizeof(*sa) > sizeof(req->rq_async_args)); - sa = ptlrpc_req_async_args(req); + sa = ptlrpc_req_async_args(sa, req); sa->sa_oa = oa; sa->sa_upcall = upcall; sa->sa_cookie = cookie; @@ -321,8 +320,7 @@ int osc_ladvise_base(struct obd_export *exp, struct obdo *oa, } req->rq_interpret_reply = osc_ladvise_interpret; - BUILD_BUG_ON(sizeof(*la) > sizeof(req->rq_async_args)); - la = ptlrpc_req_async_args(req); + la = ptlrpc_req_async_args(la, req); la->la_oa = oa; la->la_upcall = upcall; la->la_cookie = cookie; @@ -417,8 +415,7 @@ int osc_punch_send(struct obd_export *exp, struct obdo *oa, ptlrpc_request_set_replen(req); req->rq_interpret_reply = (ptlrpc_interpterer_t)osc_setattr_interpret; - BUILD_BUG_ON(sizeof(*sa) > sizeof(req->rq_async_args)); - sa = ptlrpc_req_async_args(req); + sa = ptlrpc_req_async_args(sa, req); sa->sa_oa = oa; sa->sa_upcall = upcall; sa->sa_cookie = cookie; @@ -497,8 +494,7 @@ int osc_sync_base(struct osc_object *obj, struct obdo *oa, ptlrpc_request_set_replen(req); req->rq_interpret_reply = osc_sync_interpret; - BUILD_BUG_ON(sizeof(*fa) > sizeof(req->rq_async_args)); - fa = ptlrpc_req_async_args(req); + fa = ptlrpc_req_async_args(fa, req); fa->fa_obj = obj; fa->fa_oa = oa; fa->fa_upcall = upcall; @@ -1338,8 +1334,7 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli, } ptlrpc_request_set_replen(req); - BUILD_BUG_ON(sizeof(*aa) > sizeof(req->rq_async_args)); - aa = ptlrpc_req_async_args(req); + aa = ptlrpc_req_async_args(aa, req); aa->aa_oa = oa; aa->aa_requested_nob = requested_nob; aa->aa_nio_count = niocount; @@ -1725,7 +1720,7 @@ static int osc_brw_redo_request(struct ptlrpc_request *request, new_req->rq_generation_set = 1; new_req->rq_import_generation = request->rq_import_generation; - new_aa = ptlrpc_req_async_args(new_req); + new_aa = ptlrpc_req_async_args(new_aa, new_req); INIT_LIST_HEAD(&new_aa->aa_oaps); list_splice_init(&aa->aa_oaps, &new_aa->aa_oaps); @@ -2074,8 +2069,7 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, cl_req_attr_set(env, osc2cl(obj), crattr); lustre_msg_set_jobid(req->rq_reqmsg, crattr->cra_jobid); - BUILD_BUG_ON(sizeof(*aa) > sizeof(req->rq_async_args)); - aa = ptlrpc_req_async_args(req); + aa = ptlrpc_req_async_args(aa, req); INIT_LIST_HEAD(&aa->aa_oaps); list_splice_init(&rpc_list, &aa->aa_oaps); INIT_LIST_HEAD(&aa->aa_exts); @@ -2365,8 +2359,7 @@ int osc_enqueue_base(struct obd_export *exp, struct ldlm_res_id *res_id, if (!rc) { struct osc_enqueue_args *aa; - BUILD_BUG_ON(sizeof(*aa) > sizeof(req->rq_async_args)); - aa = ptlrpc_req_async_args(req); + aa = ptlrpc_req_async_args(aa, req); aa->oa_exp = exp; aa->oa_mode = einfo->ei_mode; aa->oa_type = einfo->ei_type; @@ -2519,8 +2512,7 @@ static int osc_statfs_async(struct obd_export *exp, } req->rq_interpret_reply = (ptlrpc_interpterer_t)osc_statfs_interpret; - BUILD_BUG_ON(sizeof(*aa) > sizeof(req->rq_async_args)); - aa = ptlrpc_req_async_args(req); + aa = ptlrpc_req_async_args(aa, req); aa->aa_oi = oinfo; ptlrpc_set_add_req(rqset, req); @@ -2725,8 +2717,7 @@ int osc_set_info_async(const struct lu_env *env, struct obd_export *exp, struct osc_brw_async_args *aa; struct obdo *oa; - BUILD_BUG_ON(sizeof(*aa) > sizeof(req->rq_async_args)); - aa = ptlrpc_req_async_args(req); + aa = ptlrpc_req_async_args(aa, req); oa = kmem_cache_zalloc(obdo_cachep, GFP_NOFS); if (!oa) { ptlrpc_req_finished(req); diff --git a/fs/lustre/ptlrpc/client.c b/fs/lustre/ptlrpc/client.c index 8328c71..6086770 100644 --- a/fs/lustre/ptlrpc/client.c +++ b/fs/lustre/ptlrpc/client.c @@ -2926,8 +2926,7 @@ int ptlrpc_replay_req(struct ptlrpc_request *req) LASSERT(req->rq_import->imp_state == LUSTRE_IMP_REPLAY); - BUILD_BUG_ON(sizeof(*aa) > sizeof(req->rq_async_args)); - aa = ptlrpc_req_async_args(req); + aa = ptlrpc_req_async_args(aa, req); memset(aa, 0, sizeof(*aa)); /* Prepare request to be resent with ptlrpcd */ @@ -3258,8 +3257,7 @@ void *ptlrpcd_alloc_work(struct obd_import *imp, req->rq_no_resend = 1; req->rq_pill.rc_fmt = (void *)&worker_format; - BUILD_BUG_ON(sizeof(*args) > sizeof(req->rq_async_args)); - args = ptlrpc_req_async_args(req); + args = ptlrpc_req_async_args(args, req); args->cb = cb; args->cbdata = cbdata; diff --git a/fs/lustre/ptlrpc/import.c b/fs/lustre/ptlrpc/import.c index eb19ca6..d032962 100644 --- a/fs/lustre/ptlrpc/import.c +++ b/fs/lustre/ptlrpc/import.c @@ -696,8 +696,7 @@ int ptlrpc_connect_import(struct obd_import *imp) ptlrpc_request_set_replen(request); request->rq_interpret_reply = ptlrpc_connect_interpret; - BUILD_BUG_ON(sizeof(*aa) > sizeof(request->rq_async_args)); - aa = ptlrpc_req_async_args(request); + aa = ptlrpc_req_async_args(aa, request); memset(aa, 0, sizeof(*aa)); aa->pcaa_peer_committed = committed_before_reconnect; From patchwork Mon Sep 30 18:56:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167337 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 997AB1599 for ; Mon, 30 Sep 2019 19:08:27 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 81B93224EF for ; Mon, 30 Sep 2019 19:08:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 81B93224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 382A7603005; Mon, 30 Sep 2019 12:01:47 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id E21405C3CEE for ; Mon, 30 Sep 2019 11:57:45 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 0A5871006244; Mon, 30 Sep 2019 14:56:58 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 08398BD; Mon, 30 Sep 2019 14:56:58 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:46 -0400 Message-Id: <1569869810-23848-148-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 147/151] lustre: introduce CONFIG_LUSTRE_FS_POSIX_ACL X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: NeilBrown Lustre (or any file system) should not conditionally compile code based on CONFIG_FS_POSIX_ACL. This config option enables library support. A file system can define it's own config option, which then selects CONFIG_FS_POSIX_ACL (if needed). It should act on its own option, not the library one. This patch makes that change. WC-bug-id: https://jira.whamcloud.com/browse/LU-6142 Lustre-commit: 19b2bc9bbc25 ("LU-6142 lustre: introduce CONFIG_LUSTRE_FS_POSIX_ACL") Signed-off-by: NeilBrown Reviewed-on: https://review.whamcloud.com/36085 Reviewed-by: Petros Koutoupis Reviewed-by: Arshad Hussain Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/Kconfig | 11 +++++++++++ fs/lustre/include/lustre_acl.h | 6 +++--- fs/lustre/include/obd.h | 2 +- fs/lustre/llite/Makefile | 2 +- fs/lustre/llite/llite_internal.h | 4 ++-- fs/lustre/llite/llite_lib.c | 10 +++++----- fs/lustre/llite/xattr.c | 4 ++-- fs/lustre/mdc/mdc_request.c | 6 +++--- 8 files changed, 28 insertions(+), 17 deletions(-) diff --git a/fs/lustre/Kconfig b/fs/lustre/Kconfig index ccb78a9..2ea3f24 100644 --- a/fs/lustre/Kconfig +++ b/fs/lustre/Kconfig @@ -30,6 +30,17 @@ config LUSTRE_FS See also http://wiki.lustre.org/ +config LUSTRE_FS_POSIX_ACL + bool "Lustre POSIX Access Control Lists" + depends on LUSTRE_FS + select FS_POSIX_ACL + help + POSIX Access Control Lists (ACLs) support permissions for users and + groups beyond the owner/group/world scheme. + Lustre can support these ACLs. + + In in doubt, say Y. + config LUSTRE_DEBUG_EXPENSIVE_CHECK bool "Enable Lustre DEBUG checks" depends on LUSTRE_FS diff --git a/fs/lustre/include/lustre_acl.h b/fs/lustre/include/lustre_acl.h index 067ed3f..bb161ba 100644 --- a/fs/lustre/include/lustre_acl.h +++ b/fs/lustre/include/lustre_acl.h @@ -38,7 +38,7 @@ #include #include -#ifdef CONFIG_FS_POSIX_ACL +#ifdef CONFIG_LUSTRE_FS_POSIX_ACL #include #define LUSTRE_POSIX_ACL_MAX_ENTRIES 32 @@ -46,8 +46,8 @@ (sizeof(struct posix_acl_xattr_header) + \ LUSTRE_POSIX_ACL_MAX_ENTRIES * sizeof(struct posix_acl_xattr_entry)) -#else /* ! CONFIG_FS_POSIX_ACL */ +#else /* ! CONFIG_LUSTRE_FS_POSIX_ACL */ #define LUSTRE_POSIX_ACL_MAX_SIZE_OLD 0 -#endif /* CONFIG_FS_POSIX_ACL */ +#endif /* CONFIG_LUSTRE_FS_POSIX_ACL */ #endif diff --git a/fs/lustre/include/obd.h b/fs/lustre/include/obd.h index 2db2596..e94774b 100644 --- a/fs/lustre/include/obd.h +++ b/fs/lustre/include/obd.h @@ -903,7 +903,7 @@ struct lustre_md { struct mdt_body *body; struct lu_buf layout; struct lmv_stripe_md *lmv; -#ifdef CONFIG_FS_POSIX_ACL +#ifdef CONFIG_LUSTRE_FS_POSIX_ACL struct posix_acl *posix_acl; #endif struct mdt_remote_perm *remote_perm; diff --git a/fs/lustre/llite/Makefile b/fs/lustre/llite/Makefile index 6d8a33b..811b9ab 100644 --- a/fs/lustre/llite/Makefile +++ b/fs/lustre/llite/Makefile @@ -9,4 +9,4 @@ lustre-y := dcache.o dir.o file.o llite_lib.o llite_nfs.o \ vvp_dev.o vvp_page.o vvp_io.o vvp_object.o \ lproc_llite.o -lustre-$(CONFIG_FS_POSIX_ACL) += acl.o +lustre-$(CONFIG_LUSTRE_FS_POSIX_ACL) += acl.o diff --git a/fs/lustre/llite/llite_internal.h b/fs/lustre/llite/llite_internal.h index e067ba4..2cb67b0 100644 --- a/fs/lustre/llite/llite_internal.h +++ b/fs/lustre/llite/llite_internal.h @@ -825,13 +825,13 @@ enum ldlm_mode ll_take_md_lock(struct inode *inode, u64 bits, int ll_md_real_close(struct inode *inode, fmode_t fmode); int ll_getattr(const struct path *path, struct kstat *stat, u32 request_mask, unsigned int flags); -#ifdef CONFIG_FS_POSIX_ACL +#ifdef CONFIG_LUSTRE_FS_POSIX_ACL struct posix_acl *ll_get_acl(struct inode *inode, int type); int ll_set_acl(struct inode *inode, struct posix_acl *acl, int type); #else #define ll_get_acl NULL #define ll_set_acl NULL -#endif /* CONFIG_FS_POSIX_ACL */ +#endif /* CONFIG_LUSTRE_FS_POSIX_ACL */ int ll_migrate(struct inode *parent, struct file *file, int mdtidx, const char *name, int namelen); diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index 4d14ce1..a48d753 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -213,7 +213,7 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt) if (sbi->ll_flags & LL_SBI_LRU_RESIZE) data->ocd_connect_flags |= OBD_CONNECT_LRU_RESIZE; -#ifdef CONFIG_FS_POSIX_ACL +#ifdef CONFIG_LUSTRE_FS_POSIX_ACL data->ocd_connect_flags |= OBD_CONNECT_ACL | OBD_CONNECT_UMASK | OBD_CONNECT_LARGE_ACL; #endif @@ -542,7 +542,7 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt) ptlrpc_req_finished(request); if (IS_ERR(root)) { -#ifdef CONFIG_FS_POSIX_ACL +#ifdef CONFIG_LUSTRE_FS_POSIX_ACL if (lmd.posix_acl) { posix_acl_release(lmd.posix_acl); lmd.posix_acl = NULL; @@ -1425,7 +1425,7 @@ void ll_clear_inode(struct inode *inode) ll_xattr_cache_destroy(inode); -#ifdef CONFIG_FS_POSIX_ACL +#ifdef CONFIG_LUSTRE_FS_POSIX_ACL forget_all_cached_acls(inode); if (lli->lli_posix_acl) { posix_acl_release(lli->lli_posix_acl); @@ -1857,7 +1857,7 @@ int ll_update_inode(struct inode *inode, struct lustre_md *md) return rc; } -#ifdef CONFIG_FS_POSIX_ACL +#ifdef CONFIG_LUSTRE_FS_POSIX_ACL if (body->mbo_valid & OBD_MD_FLACL) { spin_lock(&lli->lli_lock); if (lli->lli_posix_acl) @@ -2272,7 +2272,7 @@ int ll_prep_inode(struct inode **inode, struct ptlrpc_request *req, sbi->ll_flags & LL_SBI_32BIT_API), &md); if (IS_ERR(*inode)) { -#ifdef CONFIG_FS_POSIX_ACL +#ifdef CONFIG_LUSTRE_FS_POSIX_ACL if (md.posix_acl) { posix_acl_release(md.posix_acl); md.posix_acl = NULL; diff --git a/fs/lustre/llite/xattr.c b/fs/lustre/llite/xattr.c index 378c190..f25ae59 100644 --- a/fs/lustre/llite/xattr.c +++ b/fs/lustre/llite/xattr.c @@ -422,7 +422,7 @@ static int ll_xattr_get_common(const struct xattr_handler *handler, if (rc) return rc; -#ifdef CONFIG_FS_POSIX_ACL +#ifdef CONFIG_LUSTRE_FS_POSIX_ACL /* posix acl is under protection of LOOKUP lock. when calling to this, * we just have path resolution to the target inode, so we have great * chance that cached ACL is uptodate. @@ -665,7 +665,7 @@ ssize_t ll_listxattr(struct dentry *dentry, char *buffer, size_t size) &ll_user_xattr_handler, &ll_trusted_xattr_handler, &ll_security_xattr_handler, -#ifdef CONFIG_FS_POSIX_ACL +#ifdef CONFIG_LUSTRE_FS_POSIX_ACL &ll_acl_access_xattr_handler, &ll_acl_default_xattr_handler, #endif diff --git a/fs/lustre/mdc/mdc_request.c b/fs/lustre/mdc/mdc_request.c index 408bf19..5718db2 100644 --- a/fs/lustre/mdc/mdc_request.c +++ b/fs/lustre/mdc/mdc_request.c @@ -440,7 +440,7 @@ static int mdc_getxattr(struct obd_export *exp, const struct lu_fid *fid, req); } -#ifdef CONFIG_FS_POSIX_ACL +#ifdef CONFIG_LUSTRE_FS_POSIX_ACL static int mdc_unpack_acl(struct ptlrpc_request *req, struct lustre_md *md) { struct req_capsule *pill = &req->rq_pill; @@ -567,7 +567,7 @@ static int mdc_get_lustre_md(struct obd_export *exp, rc = mdc_unpack_acl(req, md); if (rc) goto out; -#ifdef CONFIG_FS_POSIX_ACL +#ifdef CONFIG_LUSTRE_FS_POSIX_ACL } else { md->posix_acl = NULL; #endif @@ -576,7 +576,7 @@ static int mdc_get_lustre_md(struct obd_export *exp, out: if (rc) { -#ifdef CONFIG_FS_POSIX_ACL +#ifdef CONFIG_LUSTRE_FS_POSIX_ACL posix_acl_release(md->posix_acl); #endif } From patchwork Mon Sep 30 18:56:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167285 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 042431599 for ; Mon, 30 Sep 2019 19:06:30 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DF8A4224EF for ; Mon, 30 Sep 2019 19:06:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DF8A4224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8DDED5E4CAC; Mon, 30 Sep 2019 12:00:58 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 464C95C3CF6 for ; Mon, 30 Sep 2019 11:57:46 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 0CD081006245; Mon, 30 Sep 2019 14:56:58 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 0B51EB5; Mon, 30 Sep 2019 14:56:58 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:47 -0400 Message-Id: <1569869810-23848-149-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 148/151] lustre: ptlrpc: discard a server-only waitq. X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: NeilBrown This is needed on the client. The server should probably be used wait_var_event() / wake_up_var(). Signed-off-by: NeilBrown Signed-off-by: James Simmons --- fs/lustre/include/lustre_net.h | 1 - fs/lustre/ptlrpc/client.c | 1 - 2 files changed, 2 deletions(-) diff --git a/fs/lustre/include/lustre_net.h b/fs/lustre/include/lustre_net.h index 8024367..23cd565 100644 --- a/fs/lustre/include/lustre_net.h +++ b/fs/lustre/include/lustre_net.h @@ -1235,7 +1235,6 @@ struct ptlrpc_bulk_desc { /** Back pointer to the request */ struct ptlrpc_request *bd_req; const struct ptlrpc_bulk_frag_ops *bd_frag_ops; - wait_queue_head_t bd_waitq; /* server side only WQ */ int bd_iov_count; /* # entries in bd_iov */ int bd_max_iov; /* allocated size of bd_iov */ int bd_nob; /* # bytes covered */ diff --git a/fs/lustre/ptlrpc/client.c b/fs/lustre/ptlrpc/client.c index 6086770..69c14d2 100644 --- a/fs/lustre/ptlrpc/client.c +++ b/fs/lustre/ptlrpc/client.c @@ -166,7 +166,6 @@ struct ptlrpc_bulk_desc *ptlrpc_new_bulk(unsigned int nfrags, } spin_lock_init(&desc->bd_lock); - init_waitqueue_head(&desc->bd_waitq); desc->bd_max_iov = nfrags; desc->bd_iov_count = 0; desc->bd_portal = portal; From patchwork Mon Sep 30 18:56:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167293 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1EA9B13B1 for ; Mon, 30 Sep 2019 19:06:48 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 06DBB224EF for ; Mon, 30 Sep 2019 19:06:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 06DBB224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 839125E429C; Mon, 30 Sep 2019 12:01:06 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 88AD05C3A82 for ; Mon, 30 Sep 2019 11:57:46 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 1050B1006247; Mon, 30 Sep 2019 14:56:58 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 0EA07A9; Mon, 30 Sep 2019 14:56:58 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:48 -0400 Message-Id: <1569869810-23848-150-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 149/151] lustre: llite: remove // comments. X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: NeilBrown Apart from the SPDX-License-Identifier comments, the only remaining // comments are now themselves inside /* */ comments, so they don't really count as comments. The comments about rw_semaphore alignment are completely outdated. Signed-off-by: NeilBrown Signed-off-by: James Simmons --- fs/lustre/llite/llite_internal.h | 7 ------- 1 file changed, 7 deletions(-) diff --git a/fs/lustre/llite/llite_internal.h b/fs/lustre/llite/llite_internal.h index 2cb67b0..ee8d160 100644 --- a/fs/lustre/llite/llite_internal.h +++ b/fs/lustre/llite/llite_internal.h @@ -183,13 +183,6 @@ struct ll_inode_info { struct { struct mutex lli_size_mutex; char *lli_symlink_name; - /* - * struct rw_semaphore { - * signed long count; // align d.d_def_acl - * spinlock_t wait_lock; // align d.d_sa_lock - * struct list_head wait_list; - * } - */ struct rw_semaphore lli_trunc_sem; struct range_lock_tree lli_write_tree; From patchwork Mon Sep 30 18:56:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167315 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 464821599 for ; Mon, 30 Sep 2019 19:07:40 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2EB4D224EF for ; Mon, 30 Sep 2019 19:07:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2EB4D224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 6DFC15E4F12; Mon, 30 Sep 2019 12:01:26 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id CB6B85C3A82 for ; Mon, 30 Sep 2019 11:57:46 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 147851006248; Mon, 30 Sep 2019 14:56:58 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 12ED4B4; Mon, 30 Sep 2019 14:56:58 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:49 -0400 Message-Id: <1569869810-23848-151-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 150/151] lustre: remove outdated comments about ->ap_* functions. X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: NeilBrown ->ap_make_ready, ->ap_completion, ap_refresh count all no longer exist. So remove these comments. Signed-off-by: NeilBrown Signed-off-by: James Simmons --- fs/lustre/include/obd.h | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/fs/lustre/include/obd.h b/fs/lustre/include/obd.h index e94774b..6e779cf 100644 --- a/fs/lustre/include/obd.h +++ b/fs/lustre/include/obd.h @@ -224,14 +224,7 @@ struct client_obd { */ /* * ->cl_loi_list_lock protects consistency of - * ->cl_loi_{ready,read,write}_list. ->ap_make_ready() and - * ->ap_completion() call-backs are executed under this lock. As we - * cannot guarantee that these call-backs never block on all platforms - * (as a matter of fact they do block on Mac OS X), type of - * ->cl_loi_list_lock is platform dependent: it's a spin-lock on Linux - * and blocking mutex on Mac OS X. (Alternative is to make this lock - * blocking everywhere, but we don't want to slow down fast-path of - * our main platform.) + * ->cl_loi_{ready,read,write}_list. * * NB by Jinshan: though field names are still _loi_, but actually * osc_object{}s are in the list. From patchwork Mon Sep 30 18:56:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11167297 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 21A371599 for ; Mon, 30 Sep 2019 19:06:58 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 09AE7224EF for ; Mon, 30 Sep 2019 19:06:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 09AE7224EF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id BD8FA5E4DB8; Mon, 30 Sep 2019 12:01:09 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 17C5D5C3BEB for ; Mon, 30 Sep 2019 11:57:47 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 16691100624A; Mon, 30 Sep 2019 14:56:58 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 155EBBB; Mon, 30 Sep 2019 14:56:58 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Mon, 30 Sep 2019 14:56:50 -0400 Message-Id: <1569869810-23848-152-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> References: <1569869810-23848-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 151/151] lustre: clean up some comment alignment. X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: NeilBrown This patch resulted from search for "/*" and cleaning up the alignment, often removing the space and possibly inserting a tab, but sometimes leaving spaces after tabs, particularly in structure declarations. Signed-off-by: NeilBrown Signed-off-by: James Simmons --- fs/lustre/include/lprocfs_status.h | 2 +- fs/lustre/include/lustre_import.h | 4 +- fs/lustre/include/lustre_net.h | 8 +-- fs/lustre/include/obd.h | 14 ++--- fs/lustre/include/obd_support.h | 2 +- fs/lustre/ldlm/ldlm_lockd.c | 2 +- fs/lustre/llite/llite_internal.h | 10 ++-- fs/lustre/mdc/mdc_lib.c | 2 +- fs/lustre/obdclass/llog.c | 2 +- fs/lustre/obdclass/lprocfs_status.c | 4 +- fs/lustre/obdecho/echo_client.c | 2 +- fs/lustre/ptlrpc/client.c | 4 +- fs/lustre/ptlrpc/niobuf.c | 4 +- fs/lustre/ptlrpc/pack_generic.c | 2 +- fs/lustre/ptlrpc/sec_null.c | 4 +- fs/lustre/ptlrpc/sec_plain.c | 6 +- fs/lustre/ptlrpc/service.c | 10 ++-- fs/lustre/ptlrpc/wiretest.c | 12 ++-- include/linux/libcfs/libcfs_debug.h | 4 +- include/linux/lnet/lib-types.h | 8 +-- include/uapi/linux/lustre/lustre_idl.h | 34 +++++------ include/uapi/linux/lustre/lustre_param.h | 28 +++++----- include/uapi/linux/lustre/lustre_user.h | 32 +++++------ net/lnet/klnds/o2iblnd/o2iblnd.c | 6 +- net/lnet/klnds/o2iblnd/o2iblnd.h | 78 +++++++++++++++----------- net/lnet/klnds/o2iblnd/o2iblnd_cb.c | 84 ++++++++++++++-------------- net/lnet/klnds/socklnd/socklnd.c | 10 ++-- net/lnet/klnds/socklnd/socklnd_cb.c | 26 ++++----- net/lnet/klnds/socklnd/socklnd_lib.c | 8 +-- net/lnet/libcfs/debug.c | 4 +- net/lnet/libcfs/tracefile.c | 8 +-- net/lnet/lnet/acceptor.c | 6 +- net/lnet/lnet/config.c | 6 +- net/lnet/lnet/lib-move.c | 16 +++--- net/lnet/lnet/lo.c | 2 +- net/lnet/lnet/router.c | 8 +-- net/lnet/selftest/conrpc.c | 2 +- net/lnet/selftest/conrpc.h | 16 +++--- net/lnet/selftest/console.h | 96 ++++++++++++++++---------------- net/lnet/selftest/framework.c | 14 ++--- net/lnet/selftest/rpc.h | 28 +++++----- net/lnet/selftest/selftest.h | 38 ++++++------- 42 files changed, 336 insertions(+), 320 deletions(-) diff --git a/fs/lustre/include/lprocfs_status.h b/fs/lustre/include/lprocfs_status.h index c3644f9..965f8a1 100644 --- a/fs/lustre/include/lprocfs_status.h +++ b/fs/lustre/include/lprocfs_status.h @@ -216,7 +216,7 @@ struct lprocfs_stats { static inline int opcode_offset(u32 opc) { if (opc < OST_LAST_OPC) { - /* OST opcode */ + /* OST opcode */ return (opc - OST_FIRST_OPC); } else if (opc < MDS_LAST_OPC) { /* MDS opcode */ diff --git a/fs/lustre/include/lustre_import.h b/fs/lustre/include/lustre_import.h index ad06f30..522e5b7 100644 --- a/fs/lustre/include/lustre_import.h +++ b/fs/lustre/include/lustre_import.h @@ -54,8 +54,8 @@ * @{ */ #define D_ADAPTTO D_OTHER -#define AT_BINS 4 /* "bin" means "N seconds of history" */ -#define AT_FLG_NOHIST 0x1 /* use last reported value only */ +#define AT_BINS 4 /* "bin" means "N seconds of history" */ +#define AT_FLG_NOHIST 0x1 /* use last reported value only */ struct adaptive_timeout { time64_t at_binstart; /* bin start time */ diff --git a/fs/lustre/include/lustre_net.h b/fs/lustre/include/lustre_net.h index 23cd565..961b8cb 100644 --- a/fs/lustre/include/lustre_net.h +++ b/fs/lustre/include/lustre_net.h @@ -418,7 +418,7 @@ struct ptlrpc_request_set { */ struct ptlrpc_cb_id { void (*cbid_fn)(struct lnet_event *ev); /* specific callback fn */ - void *cbid_arg; /* additional arg */ + void *cbid_arg; /* additional arg */ }; /** Maximum number of locks to fit into reply state */ @@ -1235,9 +1235,9 @@ struct ptlrpc_bulk_desc { /** Back pointer to the request */ struct ptlrpc_request *bd_req; const struct ptlrpc_bulk_frag_ops *bd_frag_ops; - int bd_iov_count; /* # entries in bd_iov */ - int bd_max_iov; /* allocated size of bd_iov */ - int bd_nob; /* # bytes covered */ + int bd_iov_count; /* # entries in bd_iov */ + int bd_max_iov; /* allocated size of bd_iov */ + int bd_nob; /* # bytes covered */ int bd_nob_transferred; /* # bytes GOT/PUT */ u64 bd_last_mbits; diff --git a/fs/lustre/include/obd.h b/fs/lustre/include/obd.h index 6e779cf..99577e4 100644 --- a/fs/lustre/include/obd.h +++ b/fs/lustre/include/obd.h @@ -544,21 +544,21 @@ struct obd_device { char obd_name[MAX_OBD_NAME]; /* bitfield modification is protected by obd_dev_lock */ - unsigned long obd_attached:1, /* finished attach */ - obd_set_up:1, /* finished setup */ + unsigned long obd_attached:1, /* finished attach */ + obd_set_up:1, /* finished setup */ obd_version_recov:1, /* obd uses version checking */ - obd_replayable:1,/* recovery is enabled; inform clients */ - obd_no_transno:1, /* no committed-transno notification */ + obd_replayable:1, /* recovery is enabled; inform clients */ + obd_no_transno:1, /* no committed-transno notification */ obd_no_recov:1, /* fail instead of retry messages */ obd_stopping:1, /* started cleanup */ obd_starting:1, /* started setup */ - obd_force:1, /* cleanup with > 0 obd refcount */ - obd_fail:1, /* cleanup with failover */ + obd_force:1, /* cleanup with > 0 obd refcount */ + obd_fail:1, /* cleanup with failover */ obd_no_conn:1, /* deny new connections */ obd_inactive:1, /* device active/inactive * (for sysfs status only!!) */ - obd_no_ir:1, /* no imperative recovery. */ + obd_no_ir:1, /* no imperative recovery. */ obd_process_conf:1, /* device is processing mgs config */ obd_checksum_dump:1; /* dump pages upon cksum error */ /* use separate field as it is set in interrupt to don't mess with diff --git a/fs/lustre/include/obd_support.h b/fs/lustre/include/obd_support.h index 3e273f8..e10b372 100644 --- a/fs/lustre/include/obd_support.h +++ b/fs/lustre/include/obd_support.h @@ -46,7 +46,7 @@ /* obd_timeout should only be used for recovery, not for * networking / disk / timings affected by load (use Adaptive Timeouts) */ -extern unsigned int obd_timeout; /* seconds */ +extern unsigned int obd_timeout; /* seconds */ extern unsigned int obd_timeout_set; extern unsigned int at_min; extern unsigned int at_max; diff --git a/fs/lustre/ldlm/ldlm_lockd.c b/fs/lustre/ldlm/ldlm_lockd.c index 6ad2c1a..84d73e6 100644 --- a/fs/lustre/ldlm/ldlm_lockd.c +++ b/fs/lustre/ldlm/ldlm_lockd.c @@ -659,7 +659,7 @@ static int ldlm_callback_handler(struct ptlrpc_request *req) ldlm_handle_gl_callback(req, ns, dlm_req, lock); break; default: - LBUG(); /* checked above */ + LBUG(); /* checked above */ } return 0; diff --git a/fs/lustre/llite/llite_internal.h b/fs/lustre/llite/llite_internal.h index ee8d160..f0a50fc 100644 --- a/fs/lustre/llite/llite_internal.h +++ b/fs/lustre/llite/llite_internal.h @@ -330,11 +330,11 @@ struct ll_ra_info { * counted by page index. */ struct ra_io_arg { - unsigned long ria_start; /* start offset of read-ahead*/ - unsigned long ria_end; /* end offset of read-ahead*/ - unsigned long ria_reserved; /* reserved pages for read-ahead */ - unsigned long ria_end_min; /* minimum end to cover current read */ - bool ria_eof; /* reach end of file */ + unsigned long ria_start; /* start offset of read-ahead*/ + unsigned long ria_end; /* end offset of read-ahead*/ + unsigned long ria_reserved; /* reserved pages for read-ahead */ + unsigned long ria_end_min; /* minimum end to cover current read */ + bool ria_eof; /* reach end of file */ /* If stride read pattern is detected, ria_stoff means where * stride read is started. Note: for normal read-ahead, the * value here is meaningless, and also it will not be accessed diff --git a/fs/lustre/mdc/mdc_lib.c b/fs/lustre/mdc/mdc_lib.c index ba4eb3f..d4b2bb9 100644 --- a/fs/lustre/mdc/mdc_lib.c +++ b/fs/lustre/mdc/mdc_lib.c @@ -145,7 +145,7 @@ void mdc_readdir_pack(struct ptlrpc_request *req, u64 pgoff, size_t size, &RMF_MDT_BODY); b->mbo_fid1 = *fid; b->mbo_valid |= OBD_MD_FLID; - b->mbo_size = pgoff; /* !! */ + b->mbo_size = pgoff; /* !! */ b->mbo_nlink = size; /* !! */ __mdc_pack_body(b, -1); b->mbo_mode = LUDA_FID | LUDA_TYPE; diff --git a/fs/lustre/obdclass/llog.c b/fs/lustre/obdclass/llog.c index 3187a9c..65384ded 100644 --- a/fs/lustre/obdclass/llog.c +++ b/fs/lustre/obdclass/llog.c @@ -123,7 +123,7 @@ static int llog_read_header(const struct lu_env *env, /* lrh_len should be initialized in llog_init_handle */ handle->lgh_last_idx = 0; /* header is record with index 0 */ - llh->llh_count = 1; /* for the header record */ + llh->llh_count = 1; /* for the header record */ llh->llh_hdr.lrh_type = LLOG_HDR_MAGIC; LASSERT(handle->lgh_ctxt->loc_chunk_size >= LLOG_MIN_CHUNK_SIZE); llh->llh_hdr.lrh_len = handle->lgh_ctxt->loc_chunk_size; diff --git a/fs/lustre/obdclass/lprocfs_status.c b/fs/lustre/obdclass/lprocfs_status.c index 0e7fc70..33c76c1 100644 --- a/fs/lustre/obdclass/lprocfs_status.c +++ b/fs/lustre/obdclass/lprocfs_status.c @@ -1604,8 +1604,8 @@ char *lprocfs_find_named_value(const char *buffer, const char *name, if (!val) return (char *)buffer; - val += strlen(name); /* skip prefix */ - while (val < buffer + buflen && isspace(*val)) /* skip separator */ + val += strlen(name); /* skip prefix */ + while (val < buffer + buflen && isspace(*val)) /* skip separator */ val++; *count = 0; diff --git a/fs/lustre/obdecho/echo_client.c b/fs/lustre/obdecho/echo_client.c index 4ae21fb..ca963bb 100644 --- a/fs/lustre/obdecho/echo_client.c +++ b/fs/lustre/obdecho/echo_client.c @@ -1506,7 +1506,7 @@ static int echo_client_brw_ioctl(const struct lu_env *env, int rw, } switch (cmd) { - case OBD_IOC_CREATE: /* may create echo object */ + case OBD_IOC_CREATE: /* may create echo object */ if (!capable(CAP_SYS_ADMIN)) { rc = -EPERM; goto out; diff --git a/fs/lustre/ptlrpc/client.c b/fs/lustre/ptlrpc/client.c index 69c14d2..a533cbb 100644 --- a/fs/lustre/ptlrpc/client.c +++ b/fs/lustre/ptlrpc/client.c @@ -277,8 +277,8 @@ int ptlrpc_prep_bulk_frag(struct ptlrpc_bulk_desc *desc, void ptlrpc_free_bulk(struct ptlrpc_bulk_desc *desc) { - LASSERT(desc->bd_iov_count != LI_POISON); /* not freed already */ - LASSERT(desc->bd_md_count == 0); /* network hands off */ + LASSERT(desc->bd_iov_count != LI_POISON); /* not freed already */ + LASSERT(desc->bd_md_count == 0); /* network hands off */ LASSERT((desc->bd_export != NULL) ^ (desc->bd_import != NULL)); LASSERT(desc->bd_frag_ops); diff --git a/fs/lustre/ptlrpc/niobuf.c b/fs/lustre/ptlrpc/niobuf.c index 47d4159..02ed373 100644 --- a/fs/lustre/ptlrpc/niobuf.c +++ b/fs/lustre/ptlrpc/niobuf.c @@ -171,7 +171,7 @@ static int ptlrpc_register_bulk(struct ptlrpc_request *req) desc->bd_md_count = total_md; md.user_ptr = &desc->bd_cbid; md.eq_handle = ptlrpc_eq_h; - md.threshold = 1; /* PUT or GET */ + md.threshold = 1; /* PUT or GET */ for (posted_md = 0; posted_md < total_md; posted_md++, mbits++) { md.options = PTLRPC_MD_OPTIONS | @@ -416,7 +416,7 @@ int ptlrpc_send_reply(struct ptlrpc_request *req, int flags) CERROR("not replying on NULL connection\n"); /* bug 9635 */ return -ENOTCONN; } - ptlrpc_rs_addref(rs); /* +1 ref for the network */ + ptlrpc_rs_addref(rs); /* +1 ref for the network */ rc = sptlrpc_svc_wrap_reply(req); if (unlikely(rc)) diff --git a/fs/lustre/ptlrpc/pack_generic.c b/fs/lustre/ptlrpc/pack_generic.c index f7afc4c..b6a4fd8 100644 --- a/fs/lustre/ptlrpc/pack_generic.c +++ b/fs/lustre/ptlrpc/pack_generic.c @@ -703,7 +703,7 @@ char *lustre_msg_string(struct lustre_msg *m, u32 index, u32 max_len) slen = strnlen(str, blen); - if (slen == blen) { /* not NULL terminated */ + if (slen == blen) { /* not NULL terminated */ CERROR("can't unpack non-NULL terminated string in msg %p buffer[%d] len %d\n", m, index, blen); return NULL; diff --git a/fs/lustre/ptlrpc/sec_null.c b/fs/lustre/ptlrpc/sec_null.c index 726db6e..3c7fb68 100644 --- a/fs/lustre/ptlrpc/sec_null.c +++ b/fs/lustre/ptlrpc/sec_null.c @@ -409,7 +409,7 @@ static void null_init_internal(void) static HLIST_HEAD(__list); null_sec.ps_policy = &null_policy; - atomic_set(&null_sec.ps_refcount, 1); /* always busy */ + atomic_set(&null_sec.ps_refcount, 1); /* always busy */ null_sec.ps_id = -1; null_sec.ps_import = NULL; null_sec.ps_flvr.sf_rpc = SPTLRPC_FLVR_NULL; @@ -417,7 +417,7 @@ static void null_init_internal(void) null_sec.ps_part = LUSTRE_SP_ANY; null_sec.ps_dying = 0; spin_lock_init(&null_sec.ps_lock); - atomic_set(&null_sec.ps_nctx, 1); /* for "null_cli_ctx" */ + atomic_set(&null_sec.ps_nctx, 1); /* for "null_cli_ctx" */ INIT_LIST_HEAD(&null_sec.ps_gc_list); null_sec.ps_gc_interval = 0; null_sec.ps_gc_next = 0; diff --git a/fs/lustre/ptlrpc/sec_plain.c b/fs/lustre/ptlrpc/sec_plain.c index 84ff1fd..2358c3f 100644 --- a/fs/lustre/ptlrpc/sec_plain.c +++ b/fs/lustre/ptlrpc/sec_plain.c @@ -76,10 +76,10 @@ static inline struct plain_sec *sec2plsec(struct ptlrpc_sec *sec) #define PLAIN_FL_BULK (0x02) struct plain_header { - u8 ph_ver; /* 0 */ + u8 ph_ver; /* 0 */ u8 ph_flags; - u8 ph_sp; /* source */ - u8 ph_bulk_hash_alg; /* complete flavor desc */ + u8 ph_sp; /* source */ + u8 ph_bulk_hash_alg; /* complete flavor desc */ u8 ph_pad[4]; }; diff --git a/fs/lustre/ptlrpc/service.c b/fs/lustre/ptlrpc/service.c index c2a00de..d57df36 100644 --- a/fs/lustre/ptlrpc/service.c +++ b/fs/lustre/ptlrpc/service.c @@ -712,9 +712,9 @@ static void ptlrpc_server_free_request(struct ptlrpc_request *req) LASSERT(atomic_read(&req->rq_refcount) == 0); LASSERT(list_empty(&req->rq_timed_list)); - /* DEBUG_REQ() assumes the reply state of a request with a valid - * ref will not be destroyed until that reference is dropped. - */ + /* DEBUG_REQ() assumes the reply state of a request with a valid + * ref will not be destroyed until that reference is dropped. + */ ptlrpc_req_drop_rs(req); sptlrpc_svc_ctx_decref(req); @@ -1852,8 +1852,8 @@ static bool ptlrpc_server_normal_pending(struct ptlrpc_service_part *svcpt, been_handled = rs->rs_handled; rs->rs_handled = 1; - nlocks = rs->rs_nlocks; /* atomic "steal", but */ - rs->rs_nlocks = 0; /* locks still on rs_locks! */ + nlocks = rs->rs_nlocks; /* atomic "steal", but */ + rs->rs_nlocks = 0; /* locks still on rs_locks! */ if (nlocks == 0 && !been_handled) { /* If we see this, we should already have seen the warning diff --git a/fs/lustre/ptlrpc/wiretest.c b/fs/lustre/ptlrpc/wiretest.c index 68eb33c..21698cc 100644 --- a/fs/lustre/ptlrpc/wiretest.c +++ b/fs/lustre/ptlrpc/wiretest.c @@ -46,12 +46,12 @@ void lustre_assert_wire_constants(void) { - /* Wire protocol assertions generated by 'wirecheck' - * (make -C lustre/utils newwiretest) - * running on Linux centos6-bis 2.6.32-358.0.1.el6-head - * #3 SMP Wed Apr 17 17:37:43 CEST 2013 - * with gcc version 4.4.6 20110731 (Red Hat 4.4.6-3) (GCC) - */ + /* Wire protocol assertions generated by 'wirecheck' + * (make -C lustre/utils newwiretest) + * running on Linux centos6-bis 2.6.32-358.0.1.el6-head + * #3 SMP Wed Apr 17 17:37:43 CEST 2013 + * with gcc version 4.4.6 20110731 (Red Hat 4.4.6-3) (GCC) + */ /* Constants... */ LASSERTF(PTL_RPC_MSG_REQUEST == 4711, "found %lld\n", diff --git a/include/linux/libcfs/libcfs_debug.h b/include/linux/libcfs/libcfs_debug.h index ed00320..ee46deaf 100644 --- a/include/linux/libcfs/libcfs_debug.h +++ b/include/linux/libcfs/libcfs_debug.h @@ -101,8 +101,8 @@ # define DEBUG_SUBSYSTEM S_UNDEFINED #endif -#define CDEBUG_DEFAULT_MAX_DELAY (600 * HZ) /* jiffies */ -#define CDEBUG_DEFAULT_MIN_DELAY ((HZ + 1) / 2) /* jiffies */ +#define CDEBUG_DEFAULT_MAX_DELAY (600 * HZ) /* jiffies */ +#define CDEBUG_DEFAULT_MIN_DELAY ((HZ + 1) / 2) /* jiffies */ #define CDEBUG_DEFAULT_BACKOFF 2 struct cfs_debug_limit_state { unsigned long cdls_next; diff --git a/include/linux/lnet/lib-types.h b/include/linux/lnet/lib-types.h index 15968ba..e9560a9 100644 --- a/include/linux/lnet/lib-types.h +++ b/include/linux/lnet/lib-types.h @@ -57,7 +57,7 @@ struct lnet_msg { struct list_head msg_activelist; - struct list_head msg_list; /* Q for credits/MD */ + struct list_head msg_list; /* Q for credits/MD */ struct lnet_process_id msg_target; /* Primary NID of the source. */ @@ -101,8 +101,8 @@ struct lnet_msg { unsigned int msg_onactivelist:1; /* on the activelist */ unsigned int msg_rdma_get:1; - struct lnet_peer_ni *msg_txpeer; /* peer I'm sending to */ - struct lnet_peer_ni *msg_rxpeer; /* peer I received from */ + struct lnet_peer_ni *msg_txpeer; /* peer I'm sending to */ + struct lnet_peer_ni *msg_rxpeer; /* peer I received from */ void *msg_private; struct lnet_libmd *msg_md; @@ -470,7 +470,7 @@ struct lnet_peer_ni { unsigned int lpni_ping_notsent; /* # times router went dead<->alive */ int lpni_alive_count; - /* ytes queued for sending */ + /* ytes queued for sending */ long lpni_txqnob; /* time of last aliveness news */ time64_t lpni_timestamp; diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index 6ef7d45..401f7ef 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -846,22 +846,22 @@ struct obd_connect_data { * may result in out-of-bound memory access and kernel oops. */ __u16 ocd_maxmodrpcs; /* Maximum modify RPCs in parallel */ - __u16 padding0; /* added 2.1.0. also fix lustre_swab_connect */ - __u32 padding1; /* added 2.1.0. also fix lustre_swab_connect */ + __u16 padding0; /* added 2.1.0. also fix lustre_swab_connect */ + __u32 padding1; /* added 2.1.0. also fix lustre_swab_connect */ __u64 ocd_connect_flags2; - __u64 padding3; /* added 2.1.0. also fix lustre_swab_connect */ - __u64 padding4; /* added 2.1.0. also fix lustre_swab_connect */ - __u64 padding5; /* added 2.1.0. also fix lustre_swab_connect */ - __u64 padding6; /* added 2.1.0. also fix lustre_swab_connect */ - __u64 padding7; /* added 2.1.0. also fix lustre_swab_connect */ - __u64 padding8; /* added 2.1.0. also fix lustre_swab_connect */ - __u64 padding9; /* added 2.1.0. also fix lustre_swab_connect */ - __u64 paddingA; /* added 2.1.0. also fix lustre_swab_connect */ - __u64 paddingB; /* added 2.1.0. also fix lustre_swab_connect */ - __u64 paddingC; /* added 2.1.0. also fix lustre_swab_connect */ - __u64 paddingD; /* added 2.1.0. also fix lustre_swab_connect */ - __u64 paddingE; /* added 2.1.0. also fix lustre_swab_connect */ - __u64 paddingF; /* added 2.1.0. also fix lustre_swab_connect */ + __u64 padding3; /* added 2.1.0. also fix lustre_swab_connect */ + __u64 padding4; /* added 2.1.0. also fix lustre_swab_connect */ + __u64 padding5; /* added 2.1.0. also fix lustre_swab_connect */ + __u64 padding6; /* added 2.1.0. also fix lustre_swab_connect */ + __u64 padding7; /* added 2.1.0. also fix lustre_swab_connect */ + __u64 padding8; /* added 2.1.0. also fix lustre_swab_connect */ + __u64 padding9; /* added 2.1.0. also fix lustre_swab_connect */ + __u64 paddingA; /* added 2.1.0. also fix lustre_swab_connect */ + __u64 paddingB; /* added 2.1.0. also fix lustre_swab_connect */ + __u64 paddingC; /* added 2.1.0. also fix lustre_swab_connect */ + __u64 paddingD; /* added 2.1.0. also fix lustre_swab_connect */ + __u64 paddingE; /* added 2.1.0. also fix lustre_swab_connect */ + __u64 paddingF; /* added 2.1.0. also fix lustre_swab_connect */ }; /* XXX README XXX: @@ -2208,8 +2208,8 @@ struct ldlm_reply { enum mgs_cmd { MGS_CONNECT = 250, MGS_DISCONNECT, - MGS_EXCEPTION, /* node died, etc. */ - MGS_TARGET_REG, /* whenever target starts up */ + MGS_EXCEPTION, /* node died, etc. */ + MGS_TARGET_REG, /* whenever target starts up */ MGS_TARGET_DEL, MGS_SET_INFO, MGS_CONFIG_READ, diff --git a/include/uapi/linux/lustre/lustre_param.h b/include/uapi/linux/lustre/lustre_param.h index 1eab2ce..ab026e3 100644 --- a/include/uapi/linux/lustre/lustre_param.h +++ b/include/uapi/linux/lustre/lustre_param.h @@ -55,20 +55,20 @@ /* System global or special params not handled in obd's proc * See mgs_write_log_sys() */ -#define PARAM_TIMEOUT "timeout=" /* global */ -#define PARAM_LDLM_TIMEOUT "ldlm_timeout=" /* global */ -#define PARAM_AT_MIN "at_min=" /* global */ -#define PARAM_AT_MAX "at_max=" /* global */ -#define PARAM_AT_EXTRA "at_extra=" /* global */ -#define PARAM_AT_EARLY_MARGIN "at_early_margin=" /* global */ -#define PARAM_AT_HISTORY "at_history=" /* global */ -#define PARAM_JOBID_VAR "jobid_var=" /* global */ -#define PARAM_MGSNODE "mgsnode=" /* only at mounttime */ -#define PARAM_FAILNODE "failover.node=" /* add failover nid */ -#define PARAM_FAILMODE "failover.mode=" /* initial mount only */ -#define PARAM_ACTIVE "active=" /* activate/deactivate */ -#define PARAM_NETWORK "network=" /* bind on nid */ -#define PARAM_ID_UPCALL "identity_upcall=" /* identity upcall */ +#define PARAM_TIMEOUT "timeout=" /* global */ +#define PARAM_LDLM_TIMEOUT "ldlm_timeout=" /* global */ +#define PARAM_AT_MIN "at_min=" /* global */ +#define PARAM_AT_MAX "at_max=" /* global */ +#define PARAM_AT_EXTRA "at_extra=" /* global */ +#define PARAM_AT_EARLY_MARGIN "at_early_margin=" /* global */ +#define PARAM_AT_HISTORY "at_history=" /* global */ +#define PARAM_JOBID_VAR "jobid_var=" /* global */ +#define PARAM_MGSNODE "mgsnode=" /* only at mounttime */ +#define PARAM_FAILNODE "failover.node=" /* add failover nid */ +#define PARAM_FAILMODE "failover.mode=" /* initial mount only */ +#define PARAM_ACTIVE "active=" /* activate/deactivate */ +#define PARAM_NETWORK "network=" /* bind on nid */ +#define PARAM_ID_UPCALL "identity_upcall=" /* identity upcall */ /* Prefixes for parameters handled by obd's proc methods (XXX_process_config) */ #define PARAM_OST "ost." diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index 2c4f837..0566afa 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -417,12 +417,12 @@ struct lov_user_md_v1 { /* LOV EA user data (host-endian) */ __u32 lmm_stripe_size; /* size of stripe in bytes */ __u16 lmm_stripe_count; /* num stripes in use for this object */ union { - __u16 lmm_stripe_offset; /* starting stripe offset in - * lmm_objects, use when writing - */ - __u16 lmm_layout_gen; /* layout generation number - * used when reading - */ + __u16 lmm_stripe_offset; /* starting stripe offset in + * lmm_objects, use when writing + */ + __u16 lmm_layout_gen; /* layout generation number + * used when reading + */ }; struct lov_user_ost_data_v1 lmm_objects[0]; /* per-stripe data */ } __attribute__((packed, __may_alias__)); @@ -434,12 +434,12 @@ struct lov_user_md_v3 { /* LOV EA user data (host-endian) */ __u32 lmm_stripe_size; /* size of stripe in bytes */ __u16 lmm_stripe_count; /* num stripes in use for this object */ union { - __u16 lmm_stripe_offset; /* starting stripe offset in - * lmm_objects, use when writing - */ - __u16 lmm_layout_gen; /* layout generation number - * used when reading - */ + __u16 lmm_stripe_offset; /* starting stripe offset in + * lmm_objects, use when writing + */ + __u16 lmm_layout_gen; /* layout generation number + * used when reading + */ }; char lmm_pool_name[LOV_MAXPOOLNAME + 1]; /* pool name */ struct lov_user_ost_data_v1 lmm_objects[0]; /* per-stripe data */ @@ -571,13 +571,13 @@ static inline __u32 lov_user_md_size(__u16 stripes, __u32 lmm_magic) #ifdef HAVE_LOV_USER_MDS_DATA #define lov_user_mds_data lov_user_mds_data_v1 struct lov_user_mds_data_v1 { - lstat_t lmd_st; /* MDS stat struct */ - struct lov_user_md_v1 lmd_lmm; /* LOV EA V1 user data */ + lstat_t lmd_st; /* MDS stat struct */ + struct lov_user_md_v1 lmd_lmm; /* LOV EA V1 user data */ } __packed; struct lov_user_mds_data_v3 { - lstat_t lmd_st; /* MDS stat struct */ - struct lov_user_md_v3 lmd_lmm; /* LOV EA V3 user data */ + lstat_t lmd_st; /* MDS stat struct */ + struct lov_user_md_v3 lmd_lmm; /* LOV EA V3 user data */ } __packed; #endif diff --git a/net/lnet/klnds/o2iblnd/o2iblnd.c b/net/lnet/klnds/o2iblnd/o2iblnd.c index 4b3018e..825fe30 100644 --- a/net/lnet/klnds/o2iblnd/o2iblnd.c +++ b/net/lnet/klnds/o2iblnd/o2iblnd.c @@ -513,7 +513,7 @@ static int kiblnd_del_peer(struct lnet_ni *ni, lnet_nid_t nid) } kiblnd_del_peer_locked(peer_ni); - rc = 0; /* matched something */ + rc = 0; /* matched something */ } } @@ -730,8 +730,8 @@ struct kib_conn *kiblnd_create_conn(struct kib_peer_ni *peer_ni, conn->ibc_state = IBLND_CONN_INIT; conn->ibc_version = version; - conn->ibc_peer = peer_ni; /* I take the caller's ref */ - cmid->context = conn; /* for future CM callbacks */ + conn->ibc_peer = peer_ni; /* I take the caller's ref */ + cmid->context = conn; /* for future CM callbacks */ conn->ibc_cmid = cmid; conn->ibc_max_frags = peer_ni->ibp_max_frags; conn->ibc_queue_depth = peer_ni->ibp_queue_depth; diff --git a/net/lnet/klnds/o2iblnd/o2iblnd.h b/net/lnet/klnds/o2iblnd/o2iblnd.h index 541f95a..9021051 100644 --- a/net/lnet/klnds/o2iblnd/o2iblnd.h +++ b/net/lnet/klnds/o2iblnd/o2iblnd.h @@ -222,10 +222,10 @@ struct kib_poolset { int ps_pool_size; /* new pool size */ int ps_cpt; /* CPT id */ - kib_ps_pool_create_t ps_pool_create; /* create a new pool */ + kib_ps_pool_create_t ps_pool_create; /* create a new pool */ kib_ps_pool_destroy_t ps_pool_destroy; /* destroy a pool */ - kib_ps_node_init_t ps_node_init; /* initialize new allocated node */ - kib_ps_node_fini_t ps_node_fini; /* finalize node */ + kib_ps_node_init_t ps_node_init; /* initialize new allocated node */ + kib_ps_node_fini_t ps_node_fini; /* finalize node */ }; struct kib_pool { @@ -329,32 +329,48 @@ struct kib_sched_info { }; struct kib_data { - int kib_init; /* initialisation state */ - int kib_shutdown; /* shut down? */ - struct list_head kib_devs; /* IB devices extant */ - struct list_head kib_failed_devs; /* list head of failed devices */ - wait_queue_head_t kib_failover_waitq; /* schedulers sleep here */ - atomic_t kib_nthreads; /* # live threads */ - rwlock_t kib_global_lock; /* stabilize net/dev/peer_ni/conn ops */ - struct list_head *kib_peers; /* hash table of all my known peers */ - int kib_peer_hash_size; /* size of kib_peers */ - void *kib_connd; /* the connd task (serialisation assertions) */ - struct list_head kib_connd_conns; /* connections to setup/teardown */ - struct list_head kib_connd_zombies; /* connections with zero refcount */ + /* initialisation state */ + int kib_init; + /* shut down? */ + int kib_shutdown; + /* IB devices extant */ + struct list_head kib_devs; + /* list head of failed devices */ + struct list_head kib_failed_devs; + /* schedulers sleep here */ + wait_queue_head_t kib_failover_waitq; + /* # live threads */ + atomic_t kib_nthreads; + /* stabilize net/dev/peer_ni/conn ops */ + rwlock_t kib_global_lock; + /* hash table of all my known peers */ + struct list_head *kib_peers; + /* size of kib_peers */ + int kib_peer_hash_size; + /* the connd task (serialisation assertions) */ + void *kib_connd; + /* connections to setup/teardown */ + struct list_head kib_connd_conns; + /* connections with zero refcount */ + struct list_head kib_connd_zombies; /* connections to reconnect */ struct list_head kib_reconn_list; /* peers wait for reconnection */ struct list_head kib_reconn_wait; - /** + /* * The second that peers are pulled out from @kib_reconn_wait * for reconnection. */ time64_t kib_reconn_sec; - wait_queue_head_t kib_connd_waitq; /* connection daemon sleeps here */ - spinlock_t kib_connd_lock; /* serialise */ - struct ib_qp_attr kib_error_qpa; /* QP->ERROR */ - struct kib_sched_info **kib_scheds; /* percpt data for schedulers */ + /* connection daemon sleeps here */ + wait_queue_head_t kib_connd_waitq; + /* serialise */ + spinlock_t kib_connd_lock; + /* QP->ERROR */ + struct ib_qp_attr kib_error_qpa; + /* percpt data for schedulers */ + struct kib_sched_info **kib_scheds; }; #define IBLND_INIT_NOTHING 0 @@ -373,8 +389,8 @@ struct kib_connparams { } __packed; struct kib_immediate_msg { - struct lnet_hdr ibim_hdr; /* portals header */ - char ibim_payload[0];/* piggy-backed payload */ + struct lnet_hdr ibim_hdr; /* portals header */ + char ibim_payload[0]; /* piggy-backed payload */ } __packed; struct kib_rdma_frag { @@ -573,12 +589,12 @@ struct kib_conn { struct kib_connvars *ibc_connvars; /* in-progress connection state */ }; -#define IBLND_CONN_INIT 0 /* being initialised */ -#define IBLND_CONN_ACTIVE_CONNECT 1 /* active sending req */ -#define IBLND_CONN_PASSIVE_WAIT 2 /* passive waiting for rtu */ -#define IBLND_CONN_ESTABLISHED 3 /* connection established */ -#define IBLND_CONN_CLOSING 4 /* being closed */ -#define IBLND_CONN_DISCONNECTED 5 /* disconnected */ +#define IBLND_CONN_INIT 0 /* being initialised */ +#define IBLND_CONN_ACTIVE_CONNECT 1 /* active sending req */ +#define IBLND_CONN_PASSIVE_WAIT 2 /* passive waiting for rtu */ +#define IBLND_CONN_ESTABLISHED 3 /* connection established */ +#define IBLND_CONN_CLOSING 4 /* being closed */ +#define IBLND_CONN_DISCONNECTED 5 /* disconnected */ struct kib_peer_ni { struct list_head ibp_list; /* stash on global peer_ni list */ @@ -774,11 +790,11 @@ struct kib_peer_ni { if (!list_empty(&conn->ibc_tx_noops) || /* NOOP already queued */ !list_empty(&conn->ibc_tx_queue_nocred) || /* piggyback NOOP */ - !conn->ibc_credits) /* no credit */ + !conn->ibc_credits) /* no credit */ return 0; - if (conn->ibc_credits == 1 && /* last credit reserved for */ - !conn->ibc_outstanding_credits) /* giving back credits */ + if (conn->ibc_credits == 1 && /* last credit reserved for */ + !conn->ibc_outstanding_credits) /* giving back credits */ return 0; /* No tx to piggyback NOOP onto or no credit to send a tx */ diff --git a/net/lnet/klnds/o2iblnd/o2iblnd_cb.c b/net/lnet/klnds/o2iblnd/o2iblnd_cb.c index a702654..c2ce3b9 100644 --- a/net/lnet/klnds/o2iblnd/o2iblnd_cb.c +++ b/net/lnet/klnds/o2iblnd/o2iblnd_cb.c @@ -177,10 +177,10 @@ static int kiblnd_init_rdma(struct kib_conn *conn, struct kib_tx *tx, int type, rx->rx_wrq.wr_id = kiblnd_ptr2wreqid(rx, IBLND_WID_RX); LASSERT(conn->ibc_state >= IBLND_CONN_INIT); - LASSERT(rx->rx_nob >= 0); /* not posted */ + LASSERT(rx->rx_nob >= 0); /* not posted */ if (conn->ibc_state > IBLND_CONN_ESTABLISHED) { - kiblnd_drop_rx(rx); /* No more posts for this rx */ + kiblnd_drop_rx(rx); /* No more posts for this rx */ return 0; } @@ -202,7 +202,7 @@ static int kiblnd_init_rdma(struct kib_conn *conn, struct kib_tx *tx, int type, if (unlikely(rc)) { kiblnd_close_conn(conn, rc); - kiblnd_drop_rx(rx); /* No more posts for this rx */ + kiblnd_drop_rx(rx); /* No more posts for this rx */ goto out; } @@ -264,7 +264,7 @@ static int kiblnd_init_rdma(struct kib_conn *conn, struct kib_tx *tx, int type, return; } - if (!tx->tx_status) { /* success so far */ + if (!tx->tx_status) { /* success so far */ if (status < 0) /* failed? */ tx->tx_status = status; else if (txtype == IBLND_MSG_GET_REQ) @@ -363,9 +363,9 @@ static int kiblnd_init_rdma(struct kib_conn *conn, struct kib_tx *tx, int type, break; } - if (credits) /* credit already posted */ + if (credits) /* credit already posted */ post_credit = IBLND_POSTRX_NO_CREDIT; - else /* a keepalive NOOP */ + else /* a keepalive NOOP */ post_credit = IBLND_POSTRX_PEER_CREDIT; break; @@ -373,7 +373,7 @@ static int kiblnd_init_rdma(struct kib_conn *conn, struct kib_tx *tx, int type, post_credit = IBLND_POSTRX_DONT_POST; rc = lnet_parse(ni, &msg->ibm_u.immediate.ibim_hdr, msg->ibm_srcnid, rx, 0); - if (rc < 0) /* repost on error */ + if (rc < 0) /* repost on error */ post_credit = IBLND_POSTRX_PEER_CREDIT; break; @@ -381,7 +381,7 @@ static int kiblnd_init_rdma(struct kib_conn *conn, struct kib_tx *tx, int type, post_credit = IBLND_POSTRX_DONT_POST; rc = lnet_parse(ni, &msg->ibm_u.putreq.ibprm_hdr, msg->ibm_srcnid, rx, 1); - if (rc < 0) /* repost on error */ + if (rc < 0) /* repost on error */ post_credit = IBLND_POSTRX_PEER_CREDIT; break; @@ -445,7 +445,7 @@ static int kiblnd_init_rdma(struct kib_conn *conn, struct kib_tx *tx, int type, post_credit = IBLND_POSTRX_DONT_POST; rc = lnet_parse(ni, &msg->ibm_u.get.ibgm_hdr, msg->ibm_srcnid, rx, 1); - if (rc < 0) /* repost on error */ + if (rc < 0) /* repost on error */ post_credit = IBLND_POSTRX_PEER_CREDIT; break; @@ -457,7 +457,7 @@ static int kiblnd_init_rdma(struct kib_conn *conn, struct kib_tx *tx, int type, break; } - if (rc < 0) /* protocol error */ + if (rc < 0) /* protocol error */ kiblnd_close_conn(conn, rc); if (post_credit != IBLND_POSTRX_DONT_POST) @@ -475,8 +475,8 @@ static int kiblnd_init_rdma(struct kib_conn *conn, struct kib_tx *tx, int type, int err = -EIO; LASSERT(net); - LASSERT(rx->rx_nob < 0); /* was posted */ - rx->rx_nob = 0; /* isn't now */ + LASSERT(rx->rx_nob < 0); /* was posted */ + rx->rx_nob = 0; /* isn't now */ if (conn->ibc_state > IBLND_CONN_ESTABLISHED) goto ignore; @@ -532,7 +532,7 @@ static int kiblnd_init_rdma(struct kib_conn *conn, struct kib_tx *tx, int type, CDEBUG(D_NET, "rx %p conn %p\n", rx, conn); kiblnd_close_conn(conn, err); ignore: - kiblnd_drop_rx(rx); /* Don't re-post rx. */ + kiblnd_drop_rx(rx); /* Don't re-post rx. */ } static int @@ -1036,13 +1036,13 @@ static int kiblnd_map_tx(struct lnet_ni *ni, struct kib_tx *tx, conn->ibc_noops_posted--; if (failed) { - tx->tx_waiting = 0; /* don't wait for peer_ni */ + tx->tx_waiting = 0; /* don't wait for peer_ni */ tx->tx_status = -EIO; } - idle = !tx->tx_sending && /* This is the final callback */ - !tx->tx_waiting && /* Not waiting for peer_ni */ - !tx->tx_queued; /* Not re-queued (PUT_DONE) */ + idle = !tx->tx_sending && /* This is the final callback */ + !tx->tx_waiting && /* Not waiting for peer_ni */ + !tx->tx_queued; /* Not re-queued (PUT_DONE) */ if (idle) list_del(&tx->tx_list); @@ -1181,8 +1181,8 @@ static int kiblnd_map_tx(struct lnet_ni *ni, struct kib_tx *tx, struct list_head *q; s64 timeout_ns; - LASSERT(tx->tx_nwrq > 0); /* work items set up */ - LASSERT(!tx->tx_queued); /* not queued for sending already */ + LASSERT(tx->tx_nwrq > 0); /* work items set up */ + LASSERT(!tx->tx_queued); /* not queued for sending already */ LASSERT(conn->ibc_state >= IBLND_CONN_ESTABLISHED); timeout_ns = *kiblnd_tunables.kib_timeout * NSEC_PER_SEC; @@ -1309,7 +1309,7 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, dstaddr.sin_port = htons(*kiblnd_tunables.kib_service); dstaddr.sin_addr.s_addr = htonl(LNET_NIDADDR(peer_ni->ibp_nid)); - kiblnd_peer_addref(peer_ni); /* cmid's ref */ + kiblnd_peer_addref(peer_ni); /* cmid's ref */ if (*kiblnd_tunables.kib_use_priv_port) { rc = kiblnd_resolve_addr(cmid, &srcaddr, &dstaddr, @@ -1331,7 +1331,7 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, failed2: kiblnd_peer_connect_failed(peer_ni, 1, rc); - kiblnd_peer_decref(peer_ni); /* cmid's ref */ + kiblnd_peer_decref(peer_ni); /* cmid's ref */ rdma_destroy_id(cmid); return; failed: @@ -1569,12 +1569,12 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, case LNET_MSG_GET: if (routing || target_is_router) - break; /* send IMMEDIATE */ + break; /* send IMMEDIATE */ /* is the REPLY message too small for RDMA? */ nob = offsetof(struct kib_msg, ibm_u.immediate.ibim_payload[lntmsg->msg_md->md_length]); if (nob <= IBLND_MSG_SIZE) - break; /* send IMMEDIATE */ + break; /* send IMMEDIATE */ tx = kiblnd_get_idle_tx(ni, target.nid); if (!tx) { @@ -1616,8 +1616,8 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, return -EIO; } - tx->tx_lntmsg[0] = lntmsg; /* finalise lntmsg[0,1] on completion */ - tx->tx_waiting = 1; /* waiting for GET_DONE */ + tx->tx_lntmsg[0] = lntmsg; /* finalise lntmsg[0,1] on completion */ + tx->tx_waiting = 1; /* waiting for GET_DONE */ kiblnd_launch_tx(ni, tx, target.nid); return 0; @@ -1626,7 +1626,7 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, /* Is the payload small enough not to need RDMA? */ nob = offsetof(struct kib_msg, ibm_u.immediate.ibim_payload[payload_nob]); if (nob <= IBLND_MSG_SIZE) - break; /* send IMMEDIATE */ + break; /* send IMMEDIATE */ tx = kiblnd_get_idle_tx(ni, target.nid); if (!tx) { @@ -1656,8 +1656,8 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, ibmsg->ibm_u.putreq.ibprm_cookie = tx->tx_cookie; kiblnd_init_tx_msg(ni, tx, IBLND_MSG_PUT_REQ, sizeof(struct kib_putreq_msg)); - tx->tx_lntmsg[0] = lntmsg; /* finalise lntmsg on completion */ - tx->tx_waiting = 1; /* waiting for PUT_{ACK,NAK} */ + tx->tx_lntmsg[0] = lntmsg; /* finalise lntmsg on completion */ + tx->tx_waiting = 1; /* waiting for PUT_{ACK,NAK} */ kiblnd_launch_tx(ni, tx, target.nid); return 0; } @@ -1687,7 +1687,7 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, nob = offsetof(struct kib_immediate_msg, ibim_payload[payload_nob]); kiblnd_init_tx_msg(ni, tx, IBLND_MSG_IMMEDIATE, nob); - tx->tx_lntmsg[0] = lntmsg; /* finalise lntmsg on completion */ + tx->tx_lntmsg[0] = lntmsg; /* finalise lntmsg on completion */ kiblnd_launch_tx(ni, tx, target.nid); return 0; } @@ -1843,8 +1843,8 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, kiblnd_init_tx_msg(ni, tx, IBLND_MSG_PUT_ACK, nob); - tx->tx_lntmsg[0] = lntmsg; /* finalise lntmsg on completion */ - tx->tx_waiting = 1; /* waiting for PUT_DONE */ + tx->tx_lntmsg[0] = lntmsg; /* finalise lntmsg on completion */ + tx->tx_waiting = 1; /* waiting for PUT_DONE */ kiblnd_queue_tx(tx, conn); /* reposted buffer reserved for PUT_DONE */ @@ -1965,8 +1965,8 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, list_del(&conn->ibc_list); /* connd (see below) takes over ibc_list's ref */ - if (list_empty(&peer_ni->ibp_conns) && /* no more conns */ - kiblnd_peer_active(peer_ni)) { /* still in peer_ni table */ + if (list_empty(&peer_ni->ibp_conns) && /* no more conns */ + kiblnd_peer_active(peer_ni)) { /* still in peer_ni table */ kiblnd_unlink_peer_locked(peer_ni); /* set/clear error on last conn */ @@ -2183,7 +2183,7 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, * Add conn to peer_ni's list and nuke any dangling conns from * a different peer_ni instance... */ - kiblnd_conn_addref(conn); /* +1 ref for ibc_list */ + kiblnd_conn_addref(conn); /* +1 ref for ibc_list */ list_add(&conn->ibc_list, &peer_ni->ibp_conns); peer_ni->ibp_reconnected = 0; if (active) @@ -2341,9 +2341,9 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, rej.ibr_incarnation = net->ibn_incarnation; } - if (!ni || /* no matching net */ - ni->ni_nid != reqmsg->ibm_dstnid || /* right NET, wrong NID! */ - net->ibn_dev != ibdev) { /* wrong device */ + if (!ni || /* no matching net */ + ni->ni_nid != reqmsg->ibm_dstnid || /* right NET, wrong NID! */ + net->ibn_dev != ibdev) { /* wrong device */ CERROR("Can't accept conn from %s on %s (%s:%d:%pI4h): bad dst nid %s\n", libcfs_nid2str(nid), !ni ? "NA" : libcfs_nid2str(ni->ni_nid), @@ -2551,7 +2551,7 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, memset(&cp, 0, sizeof(cp)); cp.private_data = ackmsg; cp.private_data_len = ackmsg->ibm_nob; - cp.responder_resources = 0; /* No atomic ops or RDMA reads */ + cp.responder_resources = 0; /* No atomic ops or RDMA reads */ cp.initiator_depth = 0; cp.flow_control = 1; cp.retry_count = *kiblnd_tunables.kib_retry_count; @@ -2985,7 +2985,7 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, memset(&cp, 0, sizeof(cp)); cp.private_data = msg; cp.private_data_len = msg->ibm_nob; - cp.responder_resources = 0; /* No atomic ops or RDMA reads */ + cp.responder_resources = 0; /* No atomic ops or RDMA reads */ cp.initiator_depth = 0; cp.flow_control = 1; cp.retry_count = *kiblnd_tunables.kib_retry_count; @@ -3067,7 +3067,7 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, } kiblnd_peer_connect_failed(peer_ni, 1, rc); kiblnd_peer_decref(peer_ni); - return rc; /* rc destroys cmid */ + return rc; /* rc destroys cmid */ case RDMA_CM_EVENT_ROUTE_ERROR: peer_ni = (struct kib_peer_ni *)cmid->context; @@ -3075,7 +3075,7 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, libcfs_nid2str(peer_ni->ibp_nid), event->status); kiblnd_peer_connect_failed(peer_ni, 1, -EHOSTUNREACH); kiblnd_peer_decref(peer_ni); - return -EHOSTUNREACH; /* rc destroys cmid */ + return -EHOSTUNREACH; /* rc destroys cmid */ case RDMA_CM_EVENT_ROUTE_RESOLVED: peer_ni = (struct kib_peer_ni *)cmid->context; @@ -3089,7 +3089,7 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, libcfs_nid2str(peer_ni->ibp_nid), event->status); kiblnd_peer_connect_failed(peer_ni, 1, event->status); kiblnd_peer_decref(peer_ni); - return event->status; /* rc destroys cmid */ + return event->status; /* rc destroys cmid */ case RDMA_CM_EVENT_UNREACHABLE: conn = (struct kib_conn *)cmid->context; diff --git a/net/lnet/klnds/socklnd/socklnd.c b/net/lnet/klnds/socklnd/socklnd.c index d55b00f..03fa706 100644 --- a/net/lnet/klnds/socklnd/socklnd.c +++ b/net/lnet/klnds/socklnd/socklnd.c @@ -74,7 +74,7 @@ atomic_set(&route->ksnr_refcount, 1); route->ksnr_peer = NULL; - route->ksnr_retry_interval = 0; /* OK to connect at any time */ + route->ksnr_retry_interval = 0; /* OK to connect at any time */ route->ksnr_ipaddr = ipaddr; route->ksnr_port = port; route->ksnr_scheduled = 0; @@ -427,7 +427,7 @@ struct ksock_peer * route->ksnr_deleted = 1; list_del(&route->ksnr_list); - ksocknal_route_decref(route); /* drop peer_ni's ref */ + ksocknal_route_decref(route); /* drop peer_ni's ref */ if (list_empty(&peer_ni->ksnp_routes) && list_empty(&peer_ni->ksnp_conns)) { @@ -601,7 +601,7 @@ struct ksock_peer * ksocknal_peer_decref(peer_ni); /* ...till here */ - rc = 0; /* matched! */ + rc = 0; /* matched! */ } } @@ -974,7 +974,7 @@ struct ksock_peer * int peer_port; rc = lnet_sock_getaddr(sock, 1, &peer_ip, &peer_port); - LASSERT(!rc); /* we succeeded before */ + LASSERT(!rc); /* we succeeded before */ cr = kzalloc(sizeof(*cr), GFP_NOFS); if (!cr) { @@ -2564,7 +2564,7 @@ static int ksocknal_push(struct lnet_ni *ni, struct lnet_process_id id) LASSERT(ksocknal_data.ksnd_nnets > 0); spin_lock_bh(&net->ksnn_lock); - net->ksnn_shutdown = 1; /* prevent new peers */ + net->ksnn_shutdown = 1; /* prevent new peers */ spin_unlock_bh(&net->ksnn_lock); /* Delete all peers */ diff --git a/net/lnet/klnds/socklnd/socklnd_cb.c b/net/lnet/klnds/socklnd/socklnd_cb.c index 4861dcf..5b75ea6 100644 --- a/net/lnet/klnds/socklnd/socklnd_cb.c +++ b/net/lnet/klnds/socklnd/socklnd_cb.c @@ -119,7 +119,7 @@ struct ksock_tx * /* Never touch tx->tx_iov inside ksocknal_lib_send_iov() */ rc = ksocknal_lib_send_iov(conn, tx); - if (rc <= 0) /* sent nothing? */ + if (rc <= 0) /* sent nothing? */ return rc; nob = rc; @@ -157,7 +157,7 @@ struct ksock_tx * /* Never touch tx->tx_kiov inside ksocknal_lib_send_kiov() */ rc = ksocknal_lib_send_kiov(conn, tx); - if (rc <= 0) /* sent nothing? */ + if (rc <= 0) /* sent nothing? */ return rc; nob = rc; @@ -212,7 +212,7 @@ struct ksock_tx * } bufnob = conn->ksnc_sock->sk->sk_wmem_queued; - if (rc > 0) /* sent something? */ + if (rc > 0) /* sent something? */ conn->ksnc_tx_bufnob += rc; /* account it */ if (bufnob < conn->ksnc_tx_bufnob) { @@ -268,7 +268,7 @@ struct ksock_tx * conn->ksnc_peer->ksnp_last_alive = ktime_get_seconds(); conn->ksnc_rx_deadline = ktime_get_seconds() + *ksocknal_tunables.ksnd_timeout; - mb(); /* order with setting rx_started */ + mb(); /* order with setting rx_started */ conn->ksnc_rx_started = 1; conn->ksnc_rx_nob_left -= nob; @@ -522,8 +522,8 @@ struct ksock_tx * LASSERT(!route->ksnr_connecting); LASSERT(ksocknal_route_mask() & ~route->ksnr_connected); - route->ksnr_scheduled = 1; /* scheduling conn for connd */ - ksocknal_route_addref(route); /* extra ref for connd */ + route->ksnr_scheduled = 1; /* scheduling conn for connd */ + ksocknal_route_addref(route); /* extra ref for connd */ spin_lock_bh(&ksocknal_data.ksnd_connd_lock); @@ -906,7 +906,7 @@ struct ksock_route * return -ENOMEM; } - tx->tx_conn = NULL; /* set when assigned a conn */ + tx->tx_conn = NULL; /* set when assigned a conn */ tx->tx_lnetmsg = lntmsg; if (payload_iov) { @@ -984,9 +984,9 @@ struct ksock_route * ksocknal_lib_eager_ack(conn); } - if (!nob_to_skip) { /* right at next packet boundary now */ + if (!nob_to_skip) { /* right at next packet boundary now */ conn->ksnc_rx_started = 0; - mb(); /* racing with timeout thread */ + mb(); /* racing with timeout thread */ switch (conn->ksnc_proto->pro_version) { case KSOCK_PROTO_V2: @@ -1231,8 +1231,8 @@ struct ksock_route * case SOCKNAL_RX_SLOP: /* starting new packet? */ if (ksocknal_new_packet(conn, conn->ksnc_rx_nob_left)) - return 0; /* come back later */ - goto again; /* try to finish reading slop now */ + return 0; /* come back later */ + goto again; /* try to finish reading slop now */ default: break; @@ -1240,7 +1240,7 @@ struct ksock_route * /* Not Reached */ LBUG(); - return -EINVAL; /* keep gcc happy */ + return -EINVAL; /* keep gcc happy */ } int @@ -1449,7 +1449,7 @@ int ksocknal_scheduler(void *arg) did_something = 1; } - if (!did_something || /* nothing to do */ + if (!did_something || /* nothing to do */ ++nloops == SOCKNAL_RESCHED) { /* hogging CPU? */ spin_unlock_bh(&sched->kss_lock); diff --git a/net/lnet/klnds/socklnd/socklnd_lib.c b/net/lnet/klnds/socklnd/socklnd_lib.c index 3ac5f2e..251c28f 100644 --- a/net/lnet/klnds/socklnd/socklnd_lib.c +++ b/net/lnet/klnds/socklnd/socklnd_lib.c @@ -381,7 +381,7 @@ static int lustre_csum(struct kvec *v, void *context) int rc; rc = ksocknal_connsock_addref(conn); - if (rc) /* being shut down */ + if (rc) /* being shut down */ return; sk = conn->ksnc_sock->sk; @@ -416,7 +416,7 @@ static int lustre_csum(struct kvec *v, void *context) read_lock(&ksocknal_data.ksnd_global_lock); conn = sk->sk_user_data; - if (!conn) { /* raced with ksocknal_terminate_conn */ + if (!conn) { /* raced with ksocknal_terminate_conn */ LASSERT(sk->sk_data_ready != &ksocknal_data_ready); sk->sk_data_ready(sk); } else { @@ -450,7 +450,7 @@ static int lustre_csum(struct kvec *v, void *context) !conn ? "" : (list_empty(&conn->ksnc_tx_queue) ? " empty" : " queued")); - if (!conn) { /* raced with ksocknal_terminate_conn */ + if (!conn) { /* raced with ksocknal_terminate_conn */ LASSERT(sk->sk_write_space != &ksocknal_write_space); sk->sk_write_space(sk); @@ -458,7 +458,7 @@ static int lustre_csum(struct kvec *v, void *context) return; } - if (wspace >= min_wpace) { /* got enough space */ + if (wspace >= min_wpace) { /* got enough space */ ksocknal_write_callback(conn); /* diff --git a/net/lnet/libcfs/debug.c b/net/lnet/libcfs/debug.c index e779d6b..88c4c36 100644 --- a/net/lnet/libcfs/debug.c +++ b/net/lnet/libcfs/debug.c @@ -277,10 +277,10 @@ static const char *libcfs_debug_dbg2str(int debug) continue; token = fn(i); - if (!token) /* unused bit */ + if (!token) /* unused bit */ continue; - if (len > 0) { /* separator? */ + if (len > 0) { /* separator? */ if (len < size) str[len] = ' '; len++; diff --git a/net/lnet/libcfs/tracefile.c b/net/lnet/libcfs/tracefile.c index 5f51ec7..3b29116 100644 --- a/net/lnet/libcfs/tracefile.c +++ b/net/lnet/libcfs/tracefile.c @@ -892,17 +892,17 @@ int cfs_trace_copyin_string(char *knl_buffer, int knl_buffer_nob, return -EFAULT; nob = strnlen(knl_buffer, usr_buffer_nob); - while (--nob >= 0) /* strip trailing whitespace */ + while (--nob >= 0) /* strip trailing whitespace */ if (!isspace(knl_buffer[nob])) break; - if (nob < 0) /* empty string */ + if (nob < 0) /* empty string */ return -EINVAL; - if (nob == knl_buffer_nob) /* no space to terminate */ + if (nob == knl_buffer_nob) /* no space to terminate */ return -EOVERFLOW; - knl_buffer[nob + 1] = 0; /* terminate */ + knl_buffer[nob + 1] = 0; /* terminate */ return 0; } EXPORT_SYMBOL(cfs_trace_copyin_string); diff --git a/net/lnet/lnet/acceptor.c b/net/lnet/lnet/acceptor.c index d91a38c..1854347 100644 --- a/net/lnet/lnet/acceptor.c +++ b/net/lnet/lnet/acceptor.c @@ -150,7 +150,7 @@ int port; int fatal; - BUILD_BUG_ON(sizeof(cr) > 16); /* too big to be on the stack */ + BUILD_BUG_ON(sizeof(cr) > 16); /* too big to be on the stack */ for (port = LNET_ACCEPTOR_MAX_RESERVED_PORT; port >= LNET_ACCEPTOR_MIN_RESERVED_PORT; @@ -215,10 +215,10 @@ struct lnet_ni *ni; char *str; - LASSERT(sizeof(cr) <= 16); /* not too big for the stack */ + LASSERT(sizeof(cr) <= 16); /* not too big for the stack */ rc = lnet_sock_getaddr(sock, 1, &peer_ip, &peer_port); - LASSERT(!rc); /* we succeeded before */ + LASSERT(!rc); /* we succeeded before */ if (!lnet_accept_magic(magic, LNET_PROTO_ACCEPTOR_MAGIC)) { if (lnet_accept_magic(magic, LNET_PROTO_MAGIC)) { diff --git a/net/lnet/lnet/config.c b/net/lnet/lnet/config.c index ef1faa0..0560215 100644 --- a/net/lnet/lnet/config.c +++ b/net/lnet/lnet/config.c @@ -1142,7 +1142,7 @@ struct lnet_ni * tmp2 = &nets; /* expanding nets */ } else if (ntokens == 2 && lnet_parse_hops(token, &hops)) { - got_hops = 1; /* got a hop count */ + got_hops = 1; /* got a hop count */ continue; } else { tmp2 = &gateways; /* expanding gateways */ @@ -1347,7 +1347,7 @@ struct lnet_ni * if (!matched) return 0; - strcpy(net_entry, net); /* replace with matched net */ + strcpy(net_entry, net); /* replace with matched net */ return 1; } @@ -1491,7 +1491,7 @@ struct lnet_ni * list_del(&tb->ltb_list); - if (!rc) { /* no match */ + if (!rc) { /* no match */ lnet_free_text_buf(tb); continue; } diff --git a/net/lnet/lnet/lib-move.c b/net/lnet/lnet/lib-move.c index eee4510a..cab830a 100644 --- a/net/lnet/lnet/lib-move.c +++ b/net/lnet/lnet/lib-move.c @@ -362,11 +362,11 @@ void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats, unsigned int frag_len; unsigned int niov; - if (!len) /* no data => */ - return 0; /* no frags */ + if (!len) /* no data => */ + return 0; /* no frags */ LASSERT(src_niov > 0); - while (offset >= src->iov_len) { /* skip initial frags */ + while (offset >= src->iov_len) { /* skip initial frags */ offset -= src->iov_len; src_niov--; src++; @@ -424,11 +424,11 @@ void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats, unsigned int frag_len; unsigned int niov; - if (!len) /* no data => */ - return 0; /* no frags */ + if (!len) /* no data => */ + return 0; /* no frags */ LASSERT(src_niov > 0); - while (offset >= src->bv_len) { /* skip initial frags */ + while (offset >= src->bv_len) { /* skip initial frags */ offset -= src->bv_len; src_niov--; src++; @@ -2507,7 +2507,7 @@ void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats, * call back lnd_recv() come what may... */ if (!list_empty(&the_lnet.ln_test_peers) && /* normally we don't */ - fail_peer(src_nid, 0)) { /* shall we now? */ + fail_peer(src_nid, 0)) { /* shall we now? */ CERROR("%s, src %s: Dropping %s to simulate failure\n", libcfs_nid2str(from_nid), libcfs_nid2str(src_nid), lnet_msgtyp2str(type)); @@ -2976,7 +2976,7 @@ struct lnet_msg * LASSERT(the_lnet.ln_refcount > 0); if (!list_empty(&the_lnet.ln_test_peers) && /* normally we don't */ - fail_peer(target.nid, 1)) { /* shall we now? */ + fail_peer(target.nid, 1)) { /* shall we now? */ CERROR("Dropping GET to %s: simulated failure\n", libcfs_id2str(target)); return -EIO; diff --git a/net/lnet/lnet/lo.c b/net/lnet/lnet/lo.c index 2563243..350495f 100644 --- a/net/lnet/lnet/lo.c +++ b/net/lnet/lnet/lo.c @@ -50,7 +50,7 @@ { struct lnet_msg *sendmsg = private; - if (lntmsg) { /* not discarding */ + if (lntmsg) { /* not discarding */ if (sendmsg->msg_iov) lnet_copy_iov2iter(to, sendmsg->msg_niov, diff --git a/net/lnet/lnet/router.c b/net/lnet/lnet/router.c index 4b0c57a..278807d 100644 --- a/net/lnet/lnet/router.c +++ b/net/lnet/lnet/router.c @@ -117,10 +117,10 @@ spin_lock(&lp->lpni_lock); lp->lpni_timestamp = when; /* update timestamp */ - lp->lpni_ping_deadline = 0; /* disable ping timeout */ + lp->lpni_ping_deadline = 0; /* disable ping timeout */ - if (lp->lpni_alive_count && /* got old news */ - (!lp->lpni_alive) == (!alive)) { /* new date for old news */ + if (lp->lpni_alive_count && /* got old news */ + (!lp->lpni_alive) == (!alive)) { /* new date for old news */ spin_unlock(&lp->lpni_lock); CDEBUG(D_NET, "Old news\n"); return; @@ -337,7 +337,7 @@ static void lnet_shuffle_seed(void) (hops != LNET_UNDEFINED_HOPS && (hops < 1 || hops > 255))) return -EINVAL; - if (lnet_islocalnet(net)) /* it's a local network */ + if (lnet_islocalnet(net)) /* it's a local network */ return -EEXIST; /* Assume net, route, all new */ diff --git a/net/lnet/selftest/conrpc.c b/net/lnet/selftest/conrpc.c index af0d2aa..112f545 100644 --- a/net/lnet/selftest/conrpc.c +++ b/net/lnet/selftest/conrpc.c @@ -294,7 +294,7 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *, spin_lock(&rpc->crpc_lock); if (!crpc->crp_posted || /* not posted */ - crpc->crp_stamp_ns) { /* rpc done or aborted already */ + crpc->crp_stamp_ns) { /* rpc done or aborted already */ if (!crpc->crp_stamp_ns) { crpc->crp_stamp_ns = ktime_get_ns(); crpc->crp_status = -EINTR; diff --git a/net/lnet/selftest/conrpc.h b/net/lnet/selftest/conrpc.h index dfc31b8..fd75c50 100644 --- a/net/lnet/selftest/conrpc.h +++ b/net/lnet/selftest/conrpc.h @@ -74,14 +74,14 @@ struct lstcon_rpc { }; struct lstcon_rpc_trans { - struct list_head tas_olink; /* link chain on owner list */ - struct list_head tas_link; /* link chain on global list */ - int tas_opc; /* operation code of transaction */ - unsigned int tas_feats_updated; /* features mask is uptodate */ - unsigned int tas_features; /* test features mask */ - wait_queue_head_t tas_waitq; /* wait queue head */ - atomic_t tas_remaining; /* # of un-scheduled rpcs */ - struct list_head tas_rpcs_list; /* queued requests */ + struct list_head tas_olink; /* link chain on owner list */ + struct list_head tas_link; /* link chain on global list */ + int tas_opc; /* operation code of transaction */ + unsigned int tas_feats_updated; /* features mask is uptodate */ + unsigned int tas_features; /* test features mask */ + wait_queue_head_t tas_waitq; /* wait queue head */ + atomic_t tas_remaining; /* # of un-scheduled rpcs */ + struct list_head tas_rpcs_list; /* queued requests */ }; #define LST_TRANS_PRIVATE 0x1000 diff --git a/net/lnet/selftest/console.h b/net/lnet/selftest/console.h index b5709a4..a463276 100644 --- a/net/lnet/selftest/console.h +++ b/net/lnet/selftest/console.h @@ -64,8 +64,7 @@ struct lstcon_ndlink { /* (alias of nodes) group descriptor */ struct lstcon_group { - struct list_head grp_link; /* chain on global group list - */ + struct list_head grp_link; /* chain on global group list */ int grp_ref; /* reference count */ int grp_userland; /* has userland nodes */ int grp_nnode; /* # of nodes */ @@ -76,17 +75,17 @@ struct lstcon_group { struct list_head grp_ndl_hash[0]; /* hash table for nodes */ }; -#define LST_BATCH_IDLE 0xB0 /* idle batch */ -#define LST_BATCH_RUNNING 0xB1 /* running batch */ +#define LST_BATCH_IDLE 0xB0 /* idle batch */ +#define LST_BATCH_RUNNING 0xB1 /* running batch */ struct lstcon_tsb_hdr { - struct lst_bid tsb_id; /* batch ID */ - int tsb_index; /* test index */ + struct lst_bid tsb_id; /* batch ID */ + int tsb_index; /* test index */ }; /* (tests ) batch descriptor */ struct lstcon_batch { - struct lstcon_tsb_hdr bat_hdr; /* test_batch header */ + struct lstcon_tsb_hdr bat_hdr; /* test_batch header */ struct list_head bat_link; /* chain on session's batches list */ int bat_ntest; /* # of test */ int bat_state; /* state of the batch */ @@ -95,22 +94,23 @@ struct lstcon_batch { */ char bat_name[LST_NAME_SIZE];/* name of batch */ - struct list_head bat_test_list; /* list head of tests (struct lstcon_test) + struct list_head bat_test_list; /* list head of tests + * (struct lstcon_test) */ struct list_head bat_trans_list; /* list head of transaction */ struct list_head bat_cli_list; /* list head of client nodes * (struct lstcon_node) */ - struct list_head *bat_cli_hash; /* hash table of client nodes */ + struct list_head *bat_cli_hash; /* hash table of client nodes */ struct list_head bat_srv_list; /* list head of server nodes */ - struct list_head *bat_srv_hash; /* hash table of server nodes */ + struct list_head *bat_srv_hash; /* hash table of server nodes */ }; /* a single test descriptor */ struct lstcon_test { struct lstcon_tsb_hdr tes_hdr; /* test batch header */ - struct list_head tes_link; /* chain on batch's tests list */ - struct lstcon_batch *tes_batch; /* pointer to batch */ + struct list_head tes_link; /* chain on batch's tests list */ + struct lstcon_batch *tes_batch; /* pointer to batch */ int tes_type; /* type of the test, i.e: bulk, ping */ int tes_stop_onerr; /* stop on error */ @@ -129,48 +129,48 @@ struct lstcon_test { char tes_param[0]; /* test parameter */ }; -#define LST_GLOBAL_HASHSIZE 503 /* global nodes hash table size */ -#define LST_NODE_HASHSIZE 239 /* node hash table (for batch or group) */ +#define LST_GLOBAL_HASHSIZE 503 /* global nodes hash table size */ +#define LST_NODE_HASHSIZE 239 /* node hash table (for batch or group) */ -#define LST_SESSION_NONE 0x0 /* no session */ -#define LST_SESSION_ACTIVE 0x1 /* working session */ +#define LST_SESSION_NONE 0x0 /* no session */ +#define LST_SESSION_ACTIVE 0x1 /* working session */ -#define LST_CONSOLE_TIMEOUT 300 /* default console timeout */ +#define LST_CONSOLE_TIMEOUT 300 /* default console timeout */ struct lstcon_session { - struct mutex ses_mutex; /* only 1 thread in session */ - struct lst_sid ses_id; /* global session id */ - int ses_key; /* local session key */ - int ses_state; /* state of session */ - int ses_timeout; /* timeout in seconds */ - time64_t ses_laststamp; /* last operation stamp (seconds) - */ - unsigned int ses_features; /* tests features of the session - */ - unsigned int ses_feats_updated:1; /* features are synced with - * remote test nodes - */ - unsigned int ses_force:1; /* force creating */ - unsigned int ses_shutdown:1; /* session is shutting down */ - unsigned int ses_expired:1; /* console is timedout */ - u64 ses_id_cookie; /* batch id cookie */ - char ses_name[LST_NAME_SIZE];/* session name */ - struct lstcon_rpc_trans *ses_ping; /* session pinger */ - struct stt_timer ses_ping_timer; /* timer for pinger */ - struct lstcon_trans_stat ses_trans_stat; /* transaction stats */ - - struct list_head ses_trans_list; /* global list of transaction */ - struct list_head ses_grp_list; /* global list of groups */ - struct list_head ses_bat_list; /* global list of batches */ - struct list_head ses_ndl_list; /* global list of nodes */ - struct list_head *ses_ndl_hash; /* hash table of nodes */ - - spinlock_t ses_rpc_lock; /* serialize */ - atomic_t ses_rpc_counter; /* # of initialized RPCs */ - struct list_head ses_rpc_freelist; /* idle console rpc */ + struct mutex ses_mutex; /* only 1 thread in session */ + struct lst_sid ses_id; /* global session id */ + int ses_key; /* local session key */ + int ses_state; /* state of session */ + int ses_timeout; /* timeout in seconds */ + time64_t ses_laststamp; /* last operation stamp (secs) */ + unsigned int ses_features; /* tests features of the session */ + unsigned int ses_feats_updated:1; /* features are synced with + * remote test nodes + */ + unsigned int ses_force:1; /* force creating */ + unsigned int ses_shutdown:1; /* session is shutting down */ + unsigned int ses_expired:1; /* console is timedout */ + u64 ses_id_cookie; /* batch id cookie */ + char ses_name[LST_NAME_SIZE];/* session name */ + struct lstcon_rpc_trans + *ses_ping; /* session pinger */ + struct stt_timer ses_ping_timer; /* timer for pinger */ + struct lstcon_trans_stat + ses_trans_stat; /* transaction stats */ + + struct list_head ses_trans_list; /* global list of transaction */ + struct list_head ses_grp_list; /* global list of groups */ + struct list_head ses_bat_list; /* global list of batches */ + struct list_head ses_ndl_list; /* global list of nodes */ + struct list_head *ses_ndl_hash; /* hash table of nodes */ + + spinlock_t ses_rpc_lock; /* serialize */ + atomic_t ses_rpc_counter; /* # of initialized RPCs */ + struct list_head ses_rpc_freelist; /* idle console rpc */ }; /* session descriptor */ -extern struct lstcon_session console_session; +extern struct lstcon_session console_session; static inline struct lstcon_trans_stat * lstcon_trans_stat(void) diff --git a/net/lnet/selftest/framework.c b/net/lnet/selftest/framework.c index 714ceec..c8c42b9 100644 --- a/net/lnet/selftest/framework.c +++ b/net/lnet/selftest/framework.c @@ -101,13 +101,13 @@ #define sfw_batch_active(b) (atomic_read(&(b)->bat_nactive)) static struct smoketest_framework { - struct list_head fw_zombie_rpcs; /* RPCs to be recycled */ - struct list_head fw_zombie_sessions; /* stopping sessions */ - struct list_head fw_tests; /* registered test cases */ - atomic_t fw_nzombies; /* # zombie sessions */ - spinlock_t fw_lock; /* serialise */ - struct sfw_session *fw_session; /* _the_ session */ - int fw_shuttingdown; /* shutdown in progress */ + struct list_head fw_zombie_rpcs; /* RPCs to be recycled */ + struct list_head fw_zombie_sessions; /* stopping sessions */ + struct list_head fw_tests; /* registered test cases */ + atomic_t fw_nzombies; /* # zombie sessions */ + spinlock_t fw_lock; /* serialise */ + struct sfw_session *fw_session; /* _the_ session */ + int fw_shuttingdown; /* shutdown in progress */ struct srpc_server_rpc *fw_active_srpc;/* running RPC */ } sfw_data; diff --git a/net/lnet/selftest/rpc.h b/net/lnet/selftest/rpc.h index ae1c07f..8ccae3a 100644 --- a/net/lnet/selftest/rpc.h +++ b/net/lnet/selftest/rpc.h @@ -213,29 +213,29 @@ struct srpc_test_reply { /* TEST RPCs */ struct srpc_ping_reqst { - u64 pnr_rpyid; - u32 pnr_magic; - u32 pnr_seq; - u64 pnr_time_sec; - u64 pnr_time_usec; + u64 pnr_rpyid; + u32 pnr_magic; + u32 pnr_seq; + u64 pnr_time_sec; + u64 pnr_time_usec; } __packed; struct srpc_ping_reply { - u32 pnr_status; - u32 pnr_magic; - u32 pnr_seq; + u32 pnr_status; + u32 pnr_magic; + u32 pnr_seq; } __packed; struct srpc_brw_reqst { - u64 brw_rpyid; /* reply buffer matchbits */ - u64 brw_bulkid; /* bulk buffer matchbits */ - u32 brw_rw; /* read or write */ - u32 brw_len; /* bulk data len */ - u32 brw_flags; /* bulk data patterns */ + u64 brw_rpyid; /* reply buffer matchbits */ + u64 brw_bulkid; /* bulk buffer matchbits */ + u32 brw_rw; /* read or write */ + u32 brw_len; /* bulk data len */ + u32 brw_flags; /* bulk data patterns */ } __packed; /* bulk r/w request */ struct srpc_brw_reply { - u32 brw_status; + u32 brw_status; } __packed; /* bulk r/w reply */ #define SRPC_MSG_MAGIC 0xeeb0f00d diff --git a/net/lnet/selftest/selftest.h b/net/lnet/selftest/selftest.h index fb16143..0fe8c4f 100644 --- a/net/lnet/selftest/selftest.h +++ b/net/lnet/selftest/selftest.h @@ -198,23 +198,23 @@ struct srpc_server_rpc { /* client-side state of a RPC */ struct srpc_client_rpc { - struct list_head crpc_list; /* chain on user's lists */ - spinlock_t crpc_lock; /* serialize */ - int crpc_service; - atomic_t crpc_refcount; - int crpc_timeout; /* # seconds to wait for reply */ - struct stt_timer crpc_timer; + struct list_head crpc_list; /* chain on user's lists */ + spinlock_t crpc_lock; /* serialize */ + int crpc_service; + atomic_t crpc_refcount; + int crpc_timeout; /* # seconds to wait for reply */ + struct stt_timer crpc_timer; struct swi_workitem crpc_wi; struct lnet_process_id crpc_dest; - void (*crpc_done)(struct srpc_client_rpc *); - void (*crpc_fini)(struct srpc_client_rpc *); - int crpc_status; /* completion status */ - void *crpc_priv; /* caller data */ + void (*crpc_done)(struct srpc_client_rpc *); + void (*crpc_fini)(struct srpc_client_rpc *); + int crpc_status; /* completion status */ + void *crpc_priv; /* caller data */ /* state flags */ - unsigned int crpc_aborted:1; /* being given up */ - unsigned int crpc_closed:1; /* completed */ + unsigned int crpc_aborted:1; /* being given up */ + unsigned int crpc_closed:1; /* completed */ /* RPC events */ struct srpc_event crpc_bulkev; /* bulk event */ @@ -400,17 +400,17 @@ struct sfw_test_instance { #define sfw_id_pages(n) DIV_ROUND_UP(n, SFW_ID_PER_PAGE) struct sfw_test_unit { - struct list_head tsu_list; /* chain on lst_test_instance */ - struct lnet_process_id tsu_dest; /* id of dest node */ - int tsu_loop; /* loop count of the test */ - struct sfw_test_instance *tsu_instance; /* pointer to test instance */ - void *tsu_private; /* private data */ + struct list_head tsu_list; /* chain on lst_test_instance */ + struct lnet_process_id tsu_dest; /* id of dest node */ + int tsu_loop; /* loop count of the test */ + struct sfw_test_instance *tsu_instance; /* pointer to test instance */ + void *tsu_private; /* private data */ struct swi_workitem tsu_worker; /* workitem of the test unit */ }; struct sfw_test_case { - struct list_head tsc_list; /* chain on fw_tests */ - struct srpc_service *tsc_srv_service; /* test service */ + struct list_head tsc_list; /* chain on fw_tests */ + struct srpc_service *tsc_srv_service; /* test service */ struct sfw_test_client_ops *tsc_cli_ops; /* ops of test client */ };