From patchwork Sun Sep 18 05:21:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979311 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 19794C32771 for ; Sun, 18 Sep 2022 05:22:32 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbm53QsQz1yD0; Sat, 17 Sep 2022 22:22:29 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVblv32ZQz1wLC for ; Sat, 17 Sep 2022 22:22:19 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id D1067367; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id C3AF2F64; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:21:51 -0400 Message-Id: <1663478534-19917-2-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 01/24] lustre: dne: add crush2 hash type X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Andreas Dilger The original "crush" hash type has a significant error with files that have all-number suffixes, or suffixes that have non-alpha characters in them. These files will all be placed on the same MDT as the base filename, which causes MDT imbalance. Add a "crush2" hash type that has more stringent checks for the suffix, so that it doesn't consider all-digit suffixes, or files that only have a '.' at the right offset, as temporary files. Test that the "broken" all-digit or extra-'.' filenames are hashed properly with "crush2". We also need to confirm that the old "crush" hash has not changed (for name lookup compatibility) and still has the original "bad hashing" bug that puts all files on the same MDT. Fix handling of types beyond MDT_HASH_TYPE_CRUSH when creating dirs. Fix debug layout printing of hash_type in more parts of the code. Don't flood console if hash type is unrecognized in the future. Fixes: d956d88c463f ("lustre: dne: introduce new directory hash type 'crush'") WC-bug-id: https://jira.whamcloud.com/browse/LU-15720 Lustre-commit: 1ac4b9598ad6e2f94 ("LU-15720 dne: add crush2 hash type") Signed-off-by: Andreas Dilger Reviewed-on: https://review.whamcloud.com/47015 Tested-by: Shuichi Ihara Reviewed-by: Lai Siyao Reviewed-by: Yingjin Qian Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lu_object.h | 64 ++++++++++++++++++++++++++++----- fs/lustre/include/lustre_lmv.h | 60 +++++++++++++++++-------------- fs/lustre/include/obd_support.h | 2 +- fs/lustre/llite/dir.c | 9 +++-- fs/lustre/llite/namei.c | 2 -- fs/lustre/lmv/lmv_obd.c | 4 +-- include/uapi/linux/lustre/lustre_user.h | 15 ++++---- 7 files changed, 105 insertions(+), 51 deletions(-) diff --git a/fs/lustre/include/lu_object.h b/fs/lustre/include/lu_object.h index e4dd287c5..5c7f439 100644 --- a/fs/lustre/include/lu_object.h +++ b/fs/lustre/include/lu_object.h @@ -1290,8 +1290,29 @@ static inline bool lu_name_is_dot_or_dotdot(const struct lu_name *lname) return name_is_dot_or_dotdot(lname->ln_name, lname->ln_namelen); } +/** + * Determine if filename should be considered a "temporary" name. + * + * For temporary names, use only the main part of the filename and ignore + * the suffix, so that the filename will hash to the same MDT after it is + * renamed. That avoids creating spurious remote entries for rsync, dcp, + * vi, and other tools that create a temporary name before renaming the file. + * + * The "CRUSH" and "CRUSH2" hash types are slightly different, and should + * not be modified without introducing a new hash type. The hash algorithm + * forms an important part of the network protocol for striped directories, + * so if the hash function were "fixed" in any way it would prevent clients + * from looking up a filename on the right MDT. LU-15692. + * + * @name filename + * @namelen length of @name + * @dot_prefix if @name needs a leading '.' to be temporary + * @suffixlen number of characters after '.' in @name to check + * @crush2 whether CRUSH or CRUSH2 heuristic should be used + */ static inline bool lu_name_is_temp_file(const char *name, int namelen, - bool dot_prefix, int suffixlen) + bool dot_prefix, int suffixlen, + bool crush2) { int lower = 0; int upper = 0; @@ -1305,21 +1326,46 @@ static inline bool lu_name_is_temp_file(const char *name, int namelen, name[namelen - suffixlen - 1] != '.') return false; + /* Any non-alphanumeric chars in the suffix for CRUSH2 mean the + * filename is *not* temporary. The original CRUSH was incorrectly + * matching if a '.' happens to be in the right place, for example + * file.mdtest.12.12345 or output.6334.log, which is bad. LU-15692 + */ while (len) { - lower += islower(name[namelen - len]); - upper += isupper(name[namelen - len]); - digit += isdigit(name[namelen - len]); + if (islower(name[namelen - len])) + lower++; + else if (isupper(name[namelen - len])) + upper++; + else if (isdigit(name[namelen - len])) + digit++; + else if (crush2) + return false; len--; } - /* mktemp() filename suffixes will have a mix of upper- and lower-case - * letters and/or numbers, not all numbers, or all upper or lower-case. - * About 0.07% of randomly-generated names will slip through, + + /* mktemp() suffixes normally have a mix of upper- and lower-case + * letters and/or digits, rarely all upper- or lower-case or digits. + * Random all-digit suffixes are rare (1/45k for suffixlen=6), but + * common in normal usage (incrementing versions, dates, ranks, etc), + * so are considered non-temporary even if 1 or 2 non-numeric chars. + * + * About 0.07% of randomly-generated names will slip through, which + * only means that they may be renamed to a different MDT (slowdown), * but this avoids 99.93% of cross-MDT renames for those files. */ - if ((digit >= suffixlen - 1 && !isdigit(name[namelen - suffixlen])) || - upper == suffixlen || lower == suffixlen) + if (upper == suffixlen || lower == suffixlen) return false; + if (crush2) { + if (digit >= suffixlen - 1 && + isdigit(name[namelen - suffixlen])) + return false; + } else { /* old crush incorrectly returns "true" for all-digit suffix */ + if (digit >= suffixlen - 1 && + !isdigit(name[namelen - suffixlen])) + return false; + } + return true; } diff --git a/fs/lustre/include/lustre_lmv.h b/fs/lustre/include/lustre_lmv.h index 3720a97..a2ef550 100644 --- a/fs/lustre/include/lustre_lmv.h +++ b/fs/lustre/include/lustre_lmv.h @@ -121,21 +121,20 @@ static inline bool lmv_dir_bad_hash(const struct lmv_stripe_md *lsm) static inline void lsm_md_dump(int mask, const struct lmv_stripe_md *lsm) { - bool valid_hash = lmv_dir_bad_hash(lsm); int i; - /* If lsm_md_magic == LMV_MAGIC_FOREIGN pool_name may not be a null - * terminated string so only print LOV_MAXPOOLNAME bytes. - */ - CDEBUG(mask, - "magic %#x stripe count %d master mdt %d hash type %s:%#x max-inherit %hhu max-inherit-rr %hhu version %d migrate offset %d migrate hash %#x pool %.*s\n", + CDEBUG_LIMIT(mask, + "dump LMV: magic=%#x count=%u index=%u hash=%s:%#x max_inherit=%hhu max_inherit_rr=%hhu version=%u migrate_offset=%u migrate_hash=%s:%x pool=%.*s\n", lsm->lsm_md_magic, lsm->lsm_md_stripe_count, lsm->lsm_md_master_mdt_index, - valid_hash ? "invalid hash" : - mdt_hash_name[lsm->lsm_md_hash_type & (LMV_HASH_TYPE_MAX - 1)], - lsm->lsm_md_hash_type, lsm->lsm_md_max_inherit, - lsm->lsm_md_max_inherit_rr, lsm->lsm_md_layout_version, - lsm->lsm_md_migrate_offset, lsm->lsm_md_migrate_hash, + lmv_is_known_hash_type(lsm->lsm_md_hash_type) ? + mdt_hash_name[lsm->lsm_md_hash_type & LMV_HASH_TYPE_MASK] : + "invalid", lsm->lsm_md_hash_type, + lsm->lsm_md_max_inherit, lsm->lsm_md_max_inherit_rr, + lsm->lsm_md_layout_version, lsm->lsm_md_migrate_offset, + lmv_is_known_hash_type(lsm->lsm_md_migrate_hash) ? + mdt_hash_name[lsm->lsm_md_migrate_hash & LMV_HASH_TYPE_MASK] : + "invalid", lsm->lsm_md_migrate_hash, LOV_MAXPOOLNAME, lsm->lsm_md_pool_name); if (!lmv_dir_striped(lsm)) @@ -245,7 +244,7 @@ static inline u32 crush_hash(u32 a, u32 b) * algorithm. */ static inline unsigned int -lmv_hash_crush(unsigned int count, const char *name, int namelen) +lmv_hash_crush(unsigned int count, const char *name, int namelen, bool crush2) { unsigned long long straw; unsigned long long highest_straw = 0; @@ -258,10 +257,10 @@ static inline u32 crush_hash(u32 a, u32 b) * 1. rsync: ..XXXXXX * 2. dstripe: .XXXXXXXX */ - if (lu_name_is_temp_file(name, namelen, true, 6)) { + if (lu_name_is_temp_file(name, namelen, true, 6, crush2)) { name++; namelen -= 8; - } else if (lu_name_is_temp_file(name, namelen, false, 8)) { + } else if (lu_name_is_temp_file(name, namelen, false, 8, crush2)) { namelen -= 9; } else if (lu_name_is_backup_file(name, namelen, &i)) { LASSERT(i < namelen); @@ -340,7 +339,11 @@ static inline u32 crush_hash(u32 a, u32 b) break; case LMV_HASH_TYPE_CRUSH: stripe_index = lmv_hash_crush(stripe_count, name, - namelen); + namelen, false); + break; + case LMV_HASH_TYPE_CRUSH2: + stripe_index = lmv_hash_crush(stripe_count, name, + namelen, true); break; default: return -EBADFD; @@ -414,16 +417,19 @@ static inline bool lmv_user_magic_supported(u32 lum_magic) lum_magic == LMV_MAGIC_FOREIGN; } -#define LMV_DEBUG(mask, lmv, msg) \ - CDEBUG(mask, \ - "%s LMV: magic=%#x count=%u index=%u hash=%s:%#x version=%u migrate offset=%u migrate hash=%s:%u.\n",\ - msg, (lmv)->lmv_magic, (lmv)->lmv_stripe_count, \ - (lmv)->lmv_master_mdt_index, \ - mdt_hash_name[(lmv)->lmv_hash_type & (LMV_HASH_TYPE_MAX - 1)],\ - (lmv)->lmv_hash_type, (lmv)->lmv_layout_version, \ - (lmv)->lmv_migrate_offset, \ - mdt_hash_name[(lmv)->lmv_migrate_hash & (LMV_HASH_TYPE_MAX - 1)],\ - (lmv)->lmv_migrate_hash) +#define LMV_DEBUG(mask, lmv, msg) \ + CDEBUG_LIMIT(mask, \ + "%s LMV: magic=%#x count=%u index=%u hash=%s:%#x version=%u migrate_offset=%u migrate_hash=%s:%x pool=%.*s\n",\ + msg, (lmv)->lmv_magic, (lmv)->lmv_stripe_count, \ + (lmv)->lmv_master_mdt_index, \ + lmv_is_known_hash_type((lmv)->lmv_hash_type) ? \ + mdt_hash_name[(lmv)->lmv_hash_type & LMV_HASH_TYPE_MASK] : \ + "invalid", (lmv)->lmv_hash_type, \ + (lmv)->lmv_layout_version, (lmv)->lmv_migrate_offset, \ + lmv_is_known_hash_type((lmv)->lmv_migrate_hash) ? \ + mdt_hash_name[(lmv)->lmv_migrate_hash & LMV_HASH_TYPE_MASK] : \ + "invalid", (lmv)->lmv_migrate_hash, \ + LOV_MAXPOOLNAME, lmv->lmv_pool_name) /* master LMV is sane */ static inline bool lmv_is_sane(const struct lmv_mds_md_v1 *lmv) @@ -442,7 +448,7 @@ static inline bool lmv_is_sane(const struct lmv_mds_md_v1 *lmv) return true; insane: - LMV_DEBUG(D_ERROR, lmv, "insane"); + LMV_DEBUG(D_ERROR, lmv, "unknown layout"); return false; } @@ -464,7 +470,7 @@ static inline bool lmv_is_sane2(const struct lmv_mds_md_v1 *lmv) return true; insane: - LMV_DEBUG(D_ERROR, lmv, "insane"); + LMV_DEBUG(D_ERROR, lmv, "unknown layout"); return false; } diff --git a/fs/lustre/include/obd_support.h b/fs/lustre/include/obd_support.h index e25d4ed..0909351 100644 --- a/fs/lustre/include/obd_support.h +++ b/fs/lustre/include/obd_support.h @@ -513,7 +513,7 @@ #define OBD_FAIL_UPDATE_OBJ_NET_REP 0x1701 /* LMV */ -#define OBD_FAIL_UNKNOWN_LMV_STRIPE 0x1901 +#define OBD_FAIL_LMV_UNKNOWN_STRIPE 0x1901 /* FLR */ #define OBD_FAIL_FLR_LV_DELAY 0x1A01 diff --git a/fs/lustre/llite/dir.c b/fs/lustre/llite/dir.c index 9e7812d..451bd0e 100644 --- a/fs/lustre/llite/dir.c +++ b/fs/lustre/llite/dir.c @@ -442,9 +442,10 @@ static int ll_dir_setdirstripe(struct dentry *dparent, struct lmv_user_md *lump, if (lump->lum_magic != LMV_MAGIC_FOREIGN) { CDEBUG(D_VFSTRACE, - "VFS Op:inode=" DFID "(%p) name %s stripe_offset %d, stripe_count: %u\n", + "VFS Op:inode=" DFID "(%p) name %s stripe_offset %d stripe_count: %u, hash_type=%x\n", PFID(ll_inode2fid(parent)), parent, dirname, - (int)lump->lum_stripe_offset, lump->lum_stripe_count); + (int)lump->lum_stripe_offset, lump->lum_stripe_count, + lump->lum_hash_type); } else { struct lmv_foreign_md *lfm = (struct lmv_foreign_md *)lump; @@ -465,7 +466,9 @@ static int ll_dir_setdirstripe(struct dentry *dparent, struct lmv_user_md *lump, /* MDS < 2.14 doesn't support 'crush' hash type, and cannot handle * unknown hash if client doesn't set a valid one. switch to fnv_1a_64. */ - if (!(exp_connect_flags2(sbi->ll_md_exp) & OBD_CONNECT2_CRUSH)) { + if (CFS_FAIL_CHECK(OBD_FAIL_LMV_UNKNOWN_STRIPE)) { + lump->lum_hash_type = cfs_fail_val; + } else if (!(exp_connect_flags2(sbi->ll_md_exp) & OBD_CONNECT2_CRUSH)) { enum lmv_hash_type type = lump->lum_hash_type & LMV_HASH_TYPE_MASK; diff --git a/fs/lustre/llite/namei.c b/fs/lustre/llite/namei.c index d382554..8b21effc 100644 --- a/fs/lustre/llite/namei.c +++ b/fs/lustre/llite/namei.c @@ -1637,7 +1637,6 @@ static int ll_new_node(struct inode *dir, struct dentry *dchild, from_kuid(&init_user_ns, current_fsuid()), from_kgid(&init_user_ns, current_fsgid()), current_cap(), rdev, &request); -#if OBD_OCD_VERSION(2, 14, 58, 0) < LUSTRE_VERSION_CODE /* * server < 2.12.58 doesn't pack default LMV in intent_getattr reply, * fetch default LMV here. @@ -1704,7 +1703,6 @@ static int ll_new_node(struct inode *dir, struct dentry *dchild, request = NULL; goto again; } -#endif if (err < 0) goto err_exit; diff --git a/fs/lustre/lmv/lmv_obd.c b/fs/lustre/lmv/lmv_obd.c index e10d1bf..84d583e 100644 --- a/fs/lustre/lmv/lmv_obd.c +++ b/fs/lustre/lmv/lmv_obd.c @@ -3307,8 +3307,8 @@ static int lmv_unpack_md_v1(struct obd_export *exp, struct lmv_stripe_md *lsm, lsm->lsm_md_magic = le32_to_cpu(lmm1->lmv_magic); lsm->lsm_md_stripe_count = le32_to_cpu(lmm1->lmv_stripe_count); lsm->lsm_md_master_mdt_index = le32_to_cpu(lmm1->lmv_master_mdt_index); - if (OBD_FAIL_CHECK(OBD_FAIL_UNKNOWN_LMV_STRIPE)) - lsm->lsm_md_hash_type = LMV_HASH_TYPE_UNKNOWN; + if (CFS_FAIL_CHECK(OBD_FAIL_LMV_UNKNOWN_STRIPE)) + lsm->lsm_md_hash_type = cfs_fail_val ?: LMV_HASH_TYPE_UNKNOWN; else lsm->lsm_md_hash_type = le32_to_cpu(lmm1->lmv_hash_type); lsm->lsm_md_layout_version = le32_to_cpu(lmm1->lmv_layout_version); diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index 7b79604..8cfee7f 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -709,10 +709,12 @@ struct lmv_user_mds_data { enum lmv_hash_type { LMV_HASH_TYPE_UNKNOWN = 0, /* 0 is reserved for testing purpose */ - LMV_HASH_TYPE_ALL_CHARS = 1, - LMV_HASH_TYPE_FNV_1A_64 = 2, - LMV_HASH_TYPE_CRUSH = 3, + LMV_HASH_TYPE_ALL_CHARS = 1, /* simple sum of characters */ + LMV_HASH_TYPE_FNV_1A_64 = 2, /* reasonable non-cryptographic hash */ + LMV_HASH_TYPE_CRUSH = 3, /* double-hash to optimize migration */ + LMV_HASH_TYPE_CRUSH2 = 4, /* CRUSH with small fixes, LU-15692 */ LMV_HASH_TYPE_MAX, + LMV_HASH_TYPE_DEFAULT = LMV_HASH_TYPE_FNV_1A_64 }; static __attribute__((unused)) const char *mdt_hash_name[] = { @@ -720,9 +722,9 @@ static __attribute__((unused)) const char *mdt_hash_name[] = { "all_char", "fnv_1a_64", "crush", + "crush2", }; -#define LMV_HASH_TYPE_DEFAULT LMV_HASH_TYPE_FNV_1A_64 /* Right now only the lower part(0-16bits) of lmv_hash_type is being used, * and the higher part will be the flag to indicate the status of object, @@ -733,9 +735,8 @@ static __attribute__((unused)) const char *mdt_hash_name[] = { static inline bool lmv_is_known_hash_type(__u32 type) { - return (type & LMV_HASH_TYPE_MASK) == LMV_HASH_TYPE_FNV_1A_64 || - (type & LMV_HASH_TYPE_MASK) == LMV_HASH_TYPE_ALL_CHARS || - (type & LMV_HASH_TYPE_MASK) == LMV_HASH_TYPE_CRUSH; + return (type & LMV_HASH_TYPE_MASK) > LMV_HASH_TYPE_UNKNOWN && + (type & LMV_HASH_TYPE_MASK) < LMV_HASH_TYPE_MAX; } /* fixed layout, such directories won't split automatically */ From patchwork Sun Sep 18 05:21:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979307 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 074F8C32771 for ; Sun, 18 Sep 2022 05:22:26 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbm13GvZz1y81; Sat, 17 Sep 2022 22:22:25 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVblw1DHFz1wLC for ; Sat, 17 Sep 2022 22:22:20 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id D3F828AB4; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id C5C16E8B93; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:21:52 -0400 Message-Id: <1663478534-19917-3-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 02/24] lustre: ptlrpc: change rq_self to struct lnet_nid X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mr NeilBrown rq_self in struct ptlrpc_request can now store largs NIDs. ptlrpc_connection_get() is also changed to received a 'struct lnet_nid'. WC-bug-id: https://jira.whamcloud.com/browse/LU-10391 Lustre-commit: 37ba07f338293d249 ("LU-10391 ptlrpc: change rq_self to struct lnet_nid") Signed-off-by: Mr NeilBrown Reviewed-on: https://review.whamcloud.com/44636 Reviewed-by: Frank Sehr Reviewed-by: James Simmons Reviewed-by: Chris Horn Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_net.h | 7 ++++--- fs/lustre/ptlrpc/client.c | 4 ++-- fs/lustre/ptlrpc/connection.c | 4 ++-- fs/lustre/ptlrpc/events.c | 10 ++++++---- fs/lustre/ptlrpc/lproc_ptlrpc.c | 4 ++-- fs/lustre/ptlrpc/niobuf.c | 5 +++-- 6 files changed, 19 insertions(+), 15 deletions(-) diff --git a/fs/lustre/include/lustre_net.h b/fs/lustre/include/lustre_net.h index f70cc7c..3a35b03 100644 --- a/fs/lustre/include/lustre_net.h +++ b/fs/lustre/include/lustre_net.h @@ -941,7 +941,7 @@ struct ptlrpc_request { /** import where request is being sent */ struct obd_import *rq_import; /** our LNet NID */ - lnet_nid_t rq_self; + struct lnet_nid rq_self; /** Peer description (the other side) */ struct lnet_process_id rq_peer; /** Descriptor for the NID from which the peer sent the request. */ @@ -1706,7 +1706,8 @@ static inline bool nrs_policy_compat_one(const struct ptlrpc_service *svc, /* ptlrpc/events.c */ int ptlrpc_uuid_to_peer(struct obd_uuid *uuid, - struct lnet_process_id *peer, lnet_nid_t *self); + struct lnet_process_id *peer, + struct lnet_nid *self); /** * These callbacks are invoked by LNet when something happened to * underlying buffer @@ -1721,7 +1722,7 @@ int ptlrpc_uuid_to_peer(struct obd_uuid *uuid, /* ptlrpc/connection.c */ struct ptlrpc_connection *ptlrpc_connection_get(struct lnet_process_id peer, - lnet_nid_t self, + struct lnet_nid *self, struct obd_uuid *uuid); static inline void ptlrpc_connection_put(struct ptlrpc_connection *conn) diff --git a/fs/lustre/ptlrpc/client.c b/fs/lustre/ptlrpc/client.c index 685d6e2..7a4a9c9 100644 --- a/fs/lustre/ptlrpc/client.c +++ b/fs/lustre/ptlrpc/client.c @@ -105,7 +105,7 @@ struct ptlrpc_connection *ptlrpc_uuid_to_connection(struct obd_uuid *uuid, lnet_nid_t nid4refnet) { struct ptlrpc_connection *c; - lnet_nid_t self; + struct lnet_nid self; struct lnet_process_id peer; int err; @@ -121,7 +121,7 @@ struct ptlrpc_connection *ptlrpc_uuid_to_connection(struct obd_uuid *uuid, return NULL; } - c = ptlrpc_connection_get(peer, self, uuid); + c = ptlrpc_connection_get(peer, &self, uuid); if (c) { memcpy(c->c_remote_uuid.uuid, uuid->uuid, sizeof(c->c_remote_uuid.uuid)); diff --git a/fs/lustre/ptlrpc/connection.c b/fs/lustre/ptlrpc/connection.c index 58161fe..74a1c6a 100644 --- a/fs/lustre/ptlrpc/connection.c +++ b/fs/lustre/ptlrpc/connection.c @@ -76,7 +76,7 @@ static int lnet_process_id_cmp(struct rhashtable_compare_arg *arg, }; struct ptlrpc_connection * -ptlrpc_connection_get(struct lnet_process_id peer4, lnet_nid_t self, +ptlrpc_connection_get(struct lnet_process_id peer4, struct lnet_nid *self, struct obd_uuid *uuid) { struct ptlrpc_connection *conn, *conn2; @@ -95,7 +95,7 @@ struct ptlrpc_connection * return NULL; conn->c_peer = peer; - lnet_nid4_to_nid(self, &conn->c_self); + conn->c_self = *self; atomic_set(&conn->c_refcount, 1); if (uuid) obd_str2uuid(&conn->c_remote_uuid, uuid->uuid); diff --git a/fs/lustre/ptlrpc/events.c b/fs/lustre/ptlrpc/events.c index 140ea85..231eace 100644 --- a/fs/lustre/ptlrpc/events.c +++ b/fs/lustre/ptlrpc/events.c @@ -348,7 +348,7 @@ void request_in_callback(struct lnet_event *ev) /* Multi-Rail: keep track of both initiator and source NID. */ req->rq_peer = lnet_pid_to_pid4(&ev->initiator); req->rq_source = lnet_pid_to_pid4(&ev->source); - req->rq_self = lnet_nid_to_nid4(&ev->target.nid); + req->rq_self = ev->target.nid; req->rq_rqbd = rqbd; req->rq_phase = RQ_PHASE_NEW; if (ev->type == LNET_EVENT_PUT) @@ -468,7 +468,8 @@ static void ptlrpc_master_callback(struct lnet_event *ev) } int ptlrpc_uuid_to_peer(struct obd_uuid *uuid, - struct lnet_process_id *peer, lnet_nid_t *self) + struct lnet_process_id *peer, + struct lnet_nid *self) { int best_dist = 0; u32 best_order = 0; @@ -492,7 +493,8 @@ int ptlrpc_uuid_to_peer(struct obd_uuid *uuid, continue; if (dist == 0) { /* local! use loopback LND */ - peer->nid = *self = LNET_NID_LO_0; + peer->nid = LNET_NID_LO_0; + lnet_nid4_to_nid(peer->nid, self); rc = 0; break; } @@ -504,7 +506,7 @@ int ptlrpc_uuid_to_peer(struct obd_uuid *uuid, best_order = order; peer->nid = lnet_nid_to_nid4(&dst_nid); - *self = lnet_nid_to_nid4(&src_nid); + *self = src_nid; rc = 0; } } diff --git a/fs/lustre/ptlrpc/lproc_ptlrpc.c b/fs/lustre/ptlrpc/lproc_ptlrpc.c index 52010cb..bb17a69 100644 --- a/fs/lustre/ptlrpc/lproc_ptlrpc.c +++ b/fs/lustre/ptlrpc/lproc_ptlrpc.c @@ -1007,8 +1007,8 @@ static int ptlrpc_lprocfs_svc_req_history_show(struct seq_file *s, void *iter) req->rq_history_seq, req->rq_export && req->rq_export->exp_obd ? req->rq_export->exp_obd->obd_name : - libcfs_nid2str_r(req->rq_self, nidstr, - sizeof(nidstr)), + libcfs_nidstr_r(&req->rq_self, nidstr, + sizeof(nidstr)), libcfs_id2str(req->rq_peer), req->rq_xid, req->rq_reqlen, ptlrpc_rqphase2str(req), (s64)req->rq_arrival_time.tv_sec, diff --git a/fs/lustre/ptlrpc/niobuf.c b/fs/lustre/ptlrpc/niobuf.c index be1811a..3a10411 100644 --- a/fs/lustre/ptlrpc/niobuf.c +++ b/fs/lustre/ptlrpc/niobuf.c @@ -416,7 +416,7 @@ int ptlrpc_send_reply(struct ptlrpc_request *req, int flags) ptlrpc_at_set_reply(req, flags); if (!req->rq_export || !req->rq_export->exp_connection) - conn = ptlrpc_connection_get(req->rq_peer, req->rq_self, NULL); + conn = ptlrpc_connection_get(req->rq_peer, &req->rq_self, NULL); else conn = ptlrpc_connection_addref(req->rq_export->exp_connection); @@ -435,7 +435,8 @@ int ptlrpc_send_reply(struct ptlrpc_request *req, int flags) rc = ptl_send_buf(&rs->rs_md_h, rs->rs_repbuf, rs->rs_repdata_len, (rs->rs_difficult && !rs->rs_no_ack) ? LNET_ACK_REQ : LNET_NOACK_REQ, - &rs->rs_cb_id, req->rq_self, req->rq_source, + &rs->rs_cb_id, lnet_nid_to_nid4(&req->rq_self), + req->rq_source, ptlrpc_req2svc(req)->srv_rep_portal, req->rq_rep_mbits ? req->rq_rep_mbits : req->rq_xid, req->rq_reply_off, NULL); From patchwork Sun Sep 18 05:21:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979313 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BAFA4C32771 for ; Sun, 18 Sep 2022 05:22:36 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbmB0pj4z1yDs; Sat, 17 Sep 2022 22:22:34 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVblw6ns4z1wLC for ; Sat, 17 Sep 2022 22:22:20 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id D59B58ABE; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id CBE6527E1; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:21:53 -0400 Message-Id: <1663478534-19917-4-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 03/24] lustre: ptlrpc: pass net num to ptlrpc_uuid_to_connection X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mr NeilBrown Rather than passing a nid to indicate which net to choose, pass just the net number. This will make it easier to convert to 'struct lnet_nid'. Also change ptlrpc_uuid_to_peer() to take the refnet as an explicit argument, rather than embedding in in the peer pid. This makes the refnet test more obvious, and removes the (strange) need to test the address part against zero. WC-bug-id: https://jira.whamcloud.com/browse/LU-10391 Lustre-commit: a0bdb01ad05b24d24 ("LU-10391 ptlrpc: pass net num to ptlrpc_uuid_to_connection") Signed-off-by: Mr NeilBrown Reviewed-on: https://review.whamcloud.com/44637 Reviewed-by: Alex Zhuravlev Reviewed-by: Andreas Dilger Reviewed-by: Frank Sehr Reviewed-by: James Simmons Reviewed-by: Chris Horn Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_net.h | 5 +++-- fs/lustre/ldlm/ldlm_lib.c | 10 +++++----- fs/lustre/ptlrpc/client.c | 5 ++--- fs/lustre/ptlrpc/events.c | 7 ++++--- 4 files changed, 14 insertions(+), 13 deletions(-) diff --git a/fs/lustre/include/lustre_net.h b/fs/lustre/include/lustre_net.h index 3a35b03..d564e28 100644 --- a/fs/lustre/include/lustre_net.h +++ b/fs/lustre/include/lustre_net.h @@ -1707,7 +1707,8 @@ static inline bool nrs_policy_compat_one(const struct ptlrpc_service *svc, /* ptlrpc/events.c */ int ptlrpc_uuid_to_peer(struct obd_uuid *uuid, struct lnet_process_id *peer, - struct lnet_nid *self); + struct lnet_nid *self, + u32 refnet); /** * These callbacks are invoked by LNet when something happened to * underlying buffer @@ -1811,7 +1812,7 @@ static inline int ptlrpc_client_bulk_active(struct ptlrpc_request *req) void ptlrpc_init_client(int req_portal, int rep_portal, const char *name, struct ptlrpc_client *); struct ptlrpc_connection *ptlrpc_uuid_to_connection(struct obd_uuid *uuid, - lnet_nid_t nid4refnet); + u32 refnet); int ptlrpc_queue_wait(struct ptlrpc_request *req); int ptlrpc_replay_req(struct ptlrpc_request *req); diff --git a/fs/lustre/ldlm/ldlm_lib.c b/fs/lustre/ldlm/ldlm_lib.c index 02d1eea..804bb9c 100644 --- a/fs/lustre/ldlm/ldlm_lib.c +++ b/fs/lustre/ldlm/ldlm_lib.c @@ -56,7 +56,7 @@ static int import_set_conn(struct obd_import *imp, struct obd_uuid *uuid, { struct ptlrpc_connection *ptlrpc_conn; struct obd_import_conn *imp_conn = NULL, *item; - lnet_nid_t nid4refnet = LNET_NID_ANY; + u32 refnet = LNET_NET_ANY; int rc = 0; if (!create && !priority) { @@ -66,10 +66,10 @@ static int import_set_conn(struct obd_import *imp, struct obd_uuid *uuid, if (imp->imp_connection && imp->imp_connection->c_remote_uuid.uuid[0] == 0) - /* nid4refnet is used to restrict network connections */ - nid4refnet = lnet_nid_to_nid4(&imp->imp_connection->c_self); + /* refnet is used to restrict network connections */ + refnet = LNET_NID_NET(&imp->imp_connection->c_self); - ptlrpc_conn = ptlrpc_uuid_to_connection(uuid, nid4refnet); + ptlrpc_conn = ptlrpc_uuid_to_connection(uuid, refnet); if (!ptlrpc_conn) { CDEBUG(D_HA, "can't find connection %s\n", uuid->uuid); return -ENOENT; @@ -145,7 +145,7 @@ int client_import_dyn_add_conn(struct obd_import *imp, struct obd_uuid *uuid, struct ptlrpc_connection *ptlrpc_conn; int rc; - ptlrpc_conn = ptlrpc_uuid_to_connection(uuid, prim_nid); + ptlrpc_conn = ptlrpc_uuid_to_connection(uuid, LNET_NIDNET(prim_nid)); if (!ptlrpc_conn) { const char *str_uuid = obd_uuid2str(uuid); diff --git a/fs/lustre/ptlrpc/client.c b/fs/lustre/ptlrpc/client.c index 7a4a9c9..7ca7d35 100644 --- a/fs/lustre/ptlrpc/client.c +++ b/fs/lustre/ptlrpc/client.c @@ -102,7 +102,7 @@ void ptlrpc_init_client(int req_portal, int rep_portal, const char *name, * Return PortalRPC connection for remote uud @uuid */ struct ptlrpc_connection *ptlrpc_uuid_to_connection(struct obd_uuid *uuid, - lnet_nid_t nid4refnet) + u32 refnet) { struct ptlrpc_connection *c; struct lnet_nid self; @@ -114,8 +114,7 @@ struct ptlrpc_connection *ptlrpc_uuid_to_connection(struct obd_uuid *uuid, * before accessing its values. * coverity[uninit_use_in_call] */ - peer.nid = nid4refnet; - err = ptlrpc_uuid_to_peer(uuid, &peer, &self); + err = ptlrpc_uuid_to_peer(uuid, &peer, &self, refnet); if (err != 0) { CNETERR("cannot find peer %s!\n", uuid->uuid); return NULL; diff --git a/fs/lustre/ptlrpc/events.c b/fs/lustre/ptlrpc/events.c index 231eace..793c078 100644 --- a/fs/lustre/ptlrpc/events.c +++ b/fs/lustre/ptlrpc/events.c @@ -469,7 +469,8 @@ static void ptlrpc_master_callback(struct lnet_event *ev) int ptlrpc_uuid_to_peer(struct obd_uuid *uuid, struct lnet_process_id *peer, - struct lnet_nid *self) + struct lnet_nid *self, + u32 refnet) { int best_dist = 0; u32 best_order = 0; @@ -484,8 +485,8 @@ int ptlrpc_uuid_to_peer(struct obd_uuid *uuid, /* Choose the matching UUID that's closest */ while (lustre_uuid_to_peer(uuid->uuid, &dst_nid, count++) == 0) { - if (peer->nid != LNET_NID_ANY && LNET_NIDADDR(peer->nid) == 0 && - LNET_NID_NET(&dst_nid) != LNET_NIDNET(peer->nid)) + if (refnet != LNET_NET_ANY && + LNET_NID_NET(&dst_nid) != refnet) continue; dist = LNetDist(&dst_nid, &src_nid, &order); From patchwork Sun Sep 18 05:21:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979309 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4BA92C6FA90 for ; Sun, 18 Sep 2022 05:22:27 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbm14xCWz1yBX; Sat, 17 Sep 2022 22:22:25 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVblx5BCSz1wLC for ; Sat, 17 Sep 2022 22:22:21 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id D7B598F02; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id CDF2027E2; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:21:54 -0400 Message-Id: <1663478534-19917-5-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 04/24] lustre: ptlrpc: change rq_peer to struct lnet_nid X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mr NeilBrown rq_peer in struct ptlrpc_request can now store large NIDs. ptlrpc_connection_get() and others now take a struct lnet_processid WC-bug-id: https://jira.whamcloud.com/browse/LU-10391 Lustre-commit: afb9714c097fc6ec5 ("LU-10391 ptlrpc: change rq_peer to struct lnet_nid") Signed-off-by: Mr NeilBrown Reviewed-on: https://review.whamcloud.com/44638 Reviewed-by: Frank Sehr Reviewed-by: James Simmons Reviewed-by: Chris Horn Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_net.h | 6 +++--- fs/lustre/ldlm/ldlm_lockd.c | 6 +++--- fs/lustre/ldlm/ldlm_request.c | 2 +- fs/lustre/ptlrpc/client.c | 4 ++-- fs/lustre/ptlrpc/connection.c | 5 ++--- fs/lustre/ptlrpc/events.c | 14 +++++++------- fs/lustre/ptlrpc/lproc_ptlrpc.c | 2 +- fs/lustre/ptlrpc/niobuf.c | 3 ++- fs/lustre/ptlrpc/nrs_delay.c | 2 +- fs/lustre/ptlrpc/nrs_fifo.c | 4 ++-- fs/lustre/ptlrpc/sec.c | 2 +- fs/lustre/ptlrpc/service.c | 12 ++++++------ 12 files changed, 31 insertions(+), 31 deletions(-) diff --git a/fs/lustre/include/lustre_net.h b/fs/lustre/include/lustre_net.h index d564e28..da90d4f 100644 --- a/fs/lustre/include/lustre_net.h +++ b/fs/lustre/include/lustre_net.h @@ -943,7 +943,7 @@ struct ptlrpc_request { /** our LNet NID */ struct lnet_nid rq_self; /** Peer description (the other side) */ - struct lnet_process_id rq_peer; + struct lnet_processid rq_peer; /** Descriptor for the NID from which the peer sent the request. */ struct lnet_process_id rq_source; /** @@ -1706,7 +1706,7 @@ static inline bool nrs_policy_compat_one(const struct ptlrpc_service *svc, /* ptlrpc/events.c */ int ptlrpc_uuid_to_peer(struct obd_uuid *uuid, - struct lnet_process_id *peer, + struct lnet_processid *peer, struct lnet_nid *self, u32 refnet); /** @@ -1722,7 +1722,7 @@ int ptlrpc_uuid_to_peer(struct obd_uuid *uuid, /** @} */ /* ptlrpc/connection.c */ -struct ptlrpc_connection *ptlrpc_connection_get(struct lnet_process_id peer, +struct ptlrpc_connection *ptlrpc_connection_get(struct lnet_processid *peer, struct lnet_nid *self, struct obd_uuid *uuid); diff --git a/fs/lustre/ldlm/ldlm_lockd.c b/fs/lustre/ldlm/ldlm_lockd.c index 9f89766..0ff4e3a 100644 --- a/fs/lustre/ldlm/ldlm_lockd.c +++ b/fs/lustre/ldlm/ldlm_lockd.c @@ -584,9 +584,9 @@ static inline void ldlm_callback_errmsg(struct ptlrpc_request *req, const struct lustre_handle *handle) { DEBUG_REQ((req->rq_no_reply || rc) ? D_WARNING : D_DLMTRACE, req, - "%s: [nid %s] [rc %d] [lock %#llx]", - msg, libcfs_id2str(req->rq_peer), rc, - handle ? handle->cookie : 0); + "%s: NID=%s lock=%#llx: rc = %d", + msg, libcfs_idstr(&req->rq_peer), + handle ? handle->cookie : 0, rc); if (req->rq_no_reply) CWARN("No reply was sent, maybe cause bug 21636.\n"); else if (rc) diff --git a/fs/lustre/ldlm/ldlm_request.c b/fs/lustre/ldlm/ldlm_request.c index f59778e..56ae9b1 100644 --- a/fs/lustre/ldlm/ldlm_request.c +++ b/fs/lustre/ldlm/ldlm_request.c @@ -2100,7 +2100,7 @@ static int replay_lock_interpret(const struct lu_env *env, CERROR("received replay ack for unknown local cookie %#llx remote cookie %#llx from server %s id %s\n", aa->lock_handle.cookie, reply->lock_handle.cookie, req->rq_export->exp_client_uuid.uuid, - libcfs_id2str(req->rq_peer)); + libcfs_idstr(&req->rq_peer)); rc = -ESTALE; goto out; } diff --git a/fs/lustre/ptlrpc/client.c b/fs/lustre/ptlrpc/client.c index 7ca7d35..069ffdf 100644 --- a/fs/lustre/ptlrpc/client.c +++ b/fs/lustre/ptlrpc/client.c @@ -106,7 +106,7 @@ struct ptlrpc_connection *ptlrpc_uuid_to_connection(struct obd_uuid *uuid, { struct ptlrpc_connection *c; struct lnet_nid self; - struct lnet_process_id peer; + struct lnet_processid peer; int err; /* @@ -120,7 +120,7 @@ struct ptlrpc_connection *ptlrpc_uuid_to_connection(struct obd_uuid *uuid, return NULL; } - c = ptlrpc_connection_get(peer, &self, uuid); + c = ptlrpc_connection_get(&peer, &self, uuid); if (c) { memcpy(c->c_remote_uuid.uuid, uuid->uuid, sizeof(c->c_remote_uuid.uuid)); diff --git a/fs/lustre/ptlrpc/connection.c b/fs/lustre/ptlrpc/connection.c index 74a1c6a..dad6641 100644 --- a/fs/lustre/ptlrpc/connection.c +++ b/fs/lustre/ptlrpc/connection.c @@ -76,13 +76,12 @@ static int lnet_process_id_cmp(struct rhashtable_compare_arg *arg, }; struct ptlrpc_connection * -ptlrpc_connection_get(struct lnet_process_id peer4, struct lnet_nid *self, +ptlrpc_connection_get(struct lnet_processid *peer_orig, struct lnet_nid *self, struct obd_uuid *uuid) { struct ptlrpc_connection *conn, *conn2; - struct lnet_processid peer; + struct lnet_processid peer = *peer_orig; - lnet_pid4_to_pid(peer4, &peer); LNetPrimaryNID(&peer.nid); conn = rhashtable_lookup_fast(&conn_hash, &peer, conn_hash_params); if (conn) { diff --git a/fs/lustre/ptlrpc/events.c b/fs/lustre/ptlrpc/events.c index 793c078..6b6fd97 100644 --- a/fs/lustre/ptlrpc/events.c +++ b/fs/lustre/ptlrpc/events.c @@ -346,7 +346,7 @@ void request_in_callback(struct lnet_event *ev) req->rq_reqdata_len = ev->mlength; ktime_get_real_ts64(&req->rq_arrival_time); /* Multi-Rail: keep track of both initiator and source NID. */ - req->rq_peer = lnet_pid_to_pid4(&ev->initiator); + req->rq_peer = ev->initiator; req->rq_source = lnet_pid_to_pid4(&ev->source); req->rq_self = ev->target.nid; req->rq_rqbd = rqbd; @@ -356,7 +356,7 @@ void request_in_callback(struct lnet_event *ev) req, req->rq_xid, ev->mlength); CDEBUG(D_RPCTRACE, "peer: %s (source: %s)\n", - libcfs_id2str(req->rq_peer), libcfs_id2str(req->rq_source)); + libcfs_idstr(&req->rq_peer), libcfs_id2str(req->rq_source)); spin_lock(&svcpt->scp_lock); @@ -468,7 +468,7 @@ static void ptlrpc_master_callback(struct lnet_event *ev) } int ptlrpc_uuid_to_peer(struct obd_uuid *uuid, - struct lnet_process_id *peer, + struct lnet_processid *peer, struct lnet_nid *self, u32 refnet) { @@ -494,8 +494,8 @@ int ptlrpc_uuid_to_peer(struct obd_uuid *uuid, continue; if (dist == 0) { /* local! use loopback LND */ - peer->nid = LNET_NID_LO_0; - lnet_nid4_to_nid(peer->nid, self); + lnet_nid4_to_nid(LNET_NID_LO_0, self); + peer->nid = *self; rc = 0; break; } @@ -506,13 +506,13 @@ int ptlrpc_uuid_to_peer(struct obd_uuid *uuid, best_dist = dist; best_order = order; - peer->nid = lnet_nid_to_nid4(&dst_nid); + peer->nid = dst_nid; *self = src_nid; rc = 0; } } - CDEBUG(D_NET, "%s->%s\n", uuid->uuid, libcfs_id2str(*peer)); + CDEBUG(D_NET, "%s->%s\n", uuid->uuid, libcfs_idstr(peer)); return rc; } diff --git a/fs/lustre/ptlrpc/lproc_ptlrpc.c b/fs/lustre/ptlrpc/lproc_ptlrpc.c index bb17a69..603259d 100644 --- a/fs/lustre/ptlrpc/lproc_ptlrpc.c +++ b/fs/lustre/ptlrpc/lproc_ptlrpc.c @@ -1009,7 +1009,7 @@ static int ptlrpc_lprocfs_svc_req_history_show(struct seq_file *s, void *iter) req->rq_export->exp_obd->obd_name : libcfs_nidstr_r(&req->rq_self, nidstr, sizeof(nidstr)), - libcfs_id2str(req->rq_peer), req->rq_xid, + libcfs_idstr(&req->rq_peer), req->rq_xid, req->rq_reqlen, ptlrpc_rqphase2str(req), (s64)req->rq_arrival_time.tv_sec, (s64)req->rq_arrival_time.tv_nsec / NSEC_PER_USEC, diff --git a/fs/lustre/ptlrpc/niobuf.c b/fs/lustre/ptlrpc/niobuf.c index 3a10411..099f105 100644 --- a/fs/lustre/ptlrpc/niobuf.c +++ b/fs/lustre/ptlrpc/niobuf.c @@ -416,7 +416,8 @@ int ptlrpc_send_reply(struct ptlrpc_request *req, int flags) ptlrpc_at_set_reply(req, flags); if (!req->rq_export || !req->rq_export->exp_connection) - conn = ptlrpc_connection_get(req->rq_peer, &req->rq_self, NULL); + conn = ptlrpc_connection_get(&req->rq_peer, &req->rq_self, + NULL); else conn = ptlrpc_connection_addref(req->rq_export->exp_connection); diff --git a/fs/lustre/ptlrpc/nrs_delay.c b/fs/lustre/ptlrpc/nrs_delay.c index 5b4c2a9..127f00c 100644 --- a/fs/lustre/ptlrpc/nrs_delay.c +++ b/fs/lustre/ptlrpc/nrs_delay.c @@ -296,7 +296,7 @@ static void nrs_delay_req_stop(struct ptlrpc_nrs_policy *policy, DEBUG_REQ(D_RPCTRACE, req, "NRS: finished delayed request from %s after %llds", - libcfs_id2str(req->rq_peer), + libcfs_idstr(&req->rq_peer), (s64)(nrq->nr_u.delay.req_start_time - req->rq_srv.sr_arrival_time.tv_sec)); } diff --git a/fs/lustre/ptlrpc/nrs_fifo.c b/fs/lustre/ptlrpc/nrs_fifo.c index d0eaebc..1689616 100644 --- a/fs/lustre/ptlrpc/nrs_fifo.c +++ b/fs/lustre/ptlrpc/nrs_fifo.c @@ -175,7 +175,7 @@ struct ptlrpc_nrs_request *nrs_fifo_req_get(struct ptlrpc_nrs_policy *policy, list_del_init(&nrq->nr_u.fifo.fr_list); CDEBUG(D_RPCTRACE, "NRS start %s request from %s, seq: %llu\n", - policy->pol_desc->pd_name, libcfs_id2str(req->rq_peer), + policy->pol_desc->pd_name, libcfs_idstr(&req->rq_peer), nrq->nr_u.fifo.fr_sequence); } @@ -237,7 +237,7 @@ static void nrs_fifo_req_stop(struct ptlrpc_nrs_policy *policy, rq_nrq); CDEBUG(D_RPCTRACE, "NRS stop %s request from %s, seq: %llu\n", - policy->pol_desc->pd_name, libcfs_id2str(req->rq_peer), + policy->pol_desc->pd_name, libcfs_idstr(&req->rq_peer), nrq->nr_u.fifo.fr_sequence); } diff --git a/fs/lustre/ptlrpc/sec.c b/fs/lustre/ptlrpc/sec.c index f2d0340..976df0b 100644 --- a/fs/lustre/ptlrpc/sec.c +++ b/fs/lustre/ptlrpc/sec.c @@ -2097,7 +2097,7 @@ int sptlrpc_svc_unwrap_request(struct ptlrpc_request *req) break; default: CERROR("error unpacking request from %s x%llu\n", - libcfs_id2str(req->rq_peer), req->rq_xid); + libcfs_idstr(&req->rq_peer), req->rq_xid); return SECSVC_DROP; } diff --git a/fs/lustre/ptlrpc/service.c b/fs/lustre/ptlrpc/service.c index 7db6f52..277fbdb 100644 --- a/fs/lustre/ptlrpc/service.c +++ b/fs/lustre/ptlrpc/service.c @@ -1595,7 +1595,7 @@ static int ptlrpc_server_handle_req_in(struct ptlrpc_service_part *svcpt, rc = ptlrpc_unpack_req_msg(req, req->rq_reqlen); if (rc != 0) { CERROR("error unpacking request: ptl %d from %s x%llu\n", - svc->srv_req_portal, libcfs_id2str(req->rq_peer), + svc->srv_req_portal, libcfs_idstr(&req->rq_peer), req->rq_xid); goto err_req; } @@ -1604,7 +1604,7 @@ static int ptlrpc_server_handle_req_in(struct ptlrpc_service_part *svcpt, rc = lustre_unpack_req_ptlrpc_body(req, MSG_PTLRPC_BODY_OFF); if (rc) { CERROR("error unpacking ptlrpc body: ptl %d from %s x %llu\n", - svc->srv_req_portal, libcfs_id2str(req->rq_peer), + svc->srv_req_portal, libcfs_idstr(&req->rq_peer), req->rq_xid); goto err_req; } @@ -1621,7 +1621,7 @@ static int ptlrpc_server_handle_req_in(struct ptlrpc_service_part *svcpt, if (lustre_msg_get_type(req->rq_reqmsg) != PTL_RPC_MSG_REQUEST) { CERROR("wrong packet type received (type=%u) from %s\n", lustre_msg_get_type(req->rq_reqmsg), - libcfs_id2str(req->rq_peer)); + libcfs_idstr(&req->rq_peer)); goto err_req; } @@ -1779,7 +1779,7 @@ static int ptlrpc_server_handle_request(struct ptlrpc_service_part *svcpt, if (ktime_get_real_seconds() > request->rq_deadline) { DEBUG_REQ(D_ERROR, request, "Dropping timed-out request from %s: deadline %lld/%llds ago", - libcfs_id2str(request->rq_peer), + libcfs_idstr(&request->rq_peer), request->rq_deadline - request->rq_arrival_time.tv_sec, ktime_get_real_seconds() - request->rq_deadline); @@ -1794,7 +1794,7 @@ static int ptlrpc_server_handle_request(struct ptlrpc_service_part *svcpt, (request->rq_export ? refcount_read(&request->rq_export->exp_handle.h_ref) : -99), lustre_msg_get_status(request->rq_reqmsg), request->rq_xid, - libcfs_id2str(request->rq_peer), + libcfs_idstr(&request->rq_peer), lustre_msg_get_opc(request->rq_reqmsg), lustre_msg_get_jobid(request->rq_reqmsg) ?: ""); @@ -1836,7 +1836,7 @@ static int ptlrpc_server_handle_request(struct ptlrpc_service_part *svcpt, refcount_read(&request->rq_export->exp_handle.h_ref) : -99), lustre_msg_get_status(request->rq_reqmsg), request->rq_xid, - libcfs_id2str(request->rq_peer), + libcfs_idstr(&request->rq_peer), lustre_msg_get_opc(request->rq_reqmsg), lustre_msg_get_jobid(request->rq_reqmsg) ?: "", timediff_usecs, From patchwork Sun Sep 18 05:21:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979312 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 476DCC6FA86 for ; Sun, 18 Sep 2022 05:22:36 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbm951yTz1yDm; Sat, 17 Sep 2022 22:22:33 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVbly3Lbpz1y6l for ; Sat, 17 Sep 2022 22:22:22 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id DB6298F03; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id D338027E5; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:21:55 -0400 Message-Id: <1663478534-19917-6-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 05/24] lustre: ptlrpc: change rq_source to struct lnet_nid X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mr NeilBrown rq_source in struct ptlrpc_request can now store large NIDs. ptl_send_buf() now takes a struct lnet_processid for the peer. WC-bug-id: https://jira.whamcloud.com/browse/LU-10391 Lustre-commit: 1c250cbfa127a1517 ("LU-10391 ptlrpc: change rq_source to struct lnet_nid") Signed-off-by: Mr NeilBrown Reviewed-on: https://review.whamcloud.com/44639 Reviewed-by: Frank Sehr Reviewed-by: James Simmons Reviewed-by: Chris Horn Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_net.h | 2 +- fs/lustre/ptlrpc/events.c | 4 ++-- fs/lustre/ptlrpc/niobuf.c | 14 ++++++-------- 3 files changed, 9 insertions(+), 11 deletions(-) diff --git a/fs/lustre/include/lustre_net.h b/fs/lustre/include/lustre_net.h index da90d4f..d9aa530 100644 --- a/fs/lustre/include/lustre_net.h +++ b/fs/lustre/include/lustre_net.h @@ -945,7 +945,7 @@ struct ptlrpc_request { /** Peer description (the other side) */ struct lnet_processid rq_peer; /** Descriptor for the NID from which the peer sent the request. */ - struct lnet_process_id rq_source; + struct lnet_processid rq_source; /** * service time estimate (secs) * If the request is not served by this time, it is marked as timed out. diff --git a/fs/lustre/ptlrpc/events.c b/fs/lustre/ptlrpc/events.c index 6b6fd97..e6313af 100644 --- a/fs/lustre/ptlrpc/events.c +++ b/fs/lustre/ptlrpc/events.c @@ -347,7 +347,7 @@ void request_in_callback(struct lnet_event *ev) ktime_get_real_ts64(&req->rq_arrival_time); /* Multi-Rail: keep track of both initiator and source NID. */ req->rq_peer = ev->initiator; - req->rq_source = lnet_pid_to_pid4(&ev->source); + req->rq_source = ev->source; req->rq_self = ev->target.nid; req->rq_rqbd = rqbd; req->rq_phase = RQ_PHASE_NEW; @@ -356,7 +356,7 @@ void request_in_callback(struct lnet_event *ev) req, req->rq_xid, ev->mlength); CDEBUG(D_RPCTRACE, "peer: %s (source: %s)\n", - libcfs_idstr(&req->rq_peer), libcfs_id2str(req->rq_source)); + libcfs_idstr(&req->rq_peer), libcfs_idstr(&req->rq_source)); spin_lock(&svcpt->scp_lock); diff --git a/fs/lustre/ptlrpc/niobuf.c b/fs/lustre/ptlrpc/niobuf.c index 099f105..8f19e92 100644 --- a/fs/lustre/ptlrpc/niobuf.c +++ b/fs/lustre/ptlrpc/niobuf.c @@ -46,20 +46,18 @@ */ static int ptl_send_buf(struct lnet_handle_md *mdh, void *base, int len, enum lnet_ack_req ack, struct ptlrpc_cb_id *cbid, - lnet_nid_t self4, struct lnet_process_id peer_id4, + lnet_nid_t self4, struct lnet_processid *peer_id, int portal, u64 xid, unsigned int offset, struct lnet_handle_md *bulk_cookie) { int rc; struct lnet_md md; struct lnet_nid self; - struct lnet_processid peer_id; lnet_nid4_to_nid(self4, &self); - lnet_pid4_to_pid(peer_id4, &peer_id); LASSERT(portal != 0); - CDEBUG(D_INFO, "peer_id %s\n", libcfs_id2str(peer_id4)); + CDEBUG(D_INFO, "peer_id %s\n", libcfs_idstr(peer_id)); md.start = base; md.length = len; md.threshold = (ack == LNET_ACK_REQ) ? 2 : 1; @@ -91,7 +89,7 @@ static int ptl_send_buf(struct lnet_handle_md *mdh, void *base, int len, percpu_ref_get(&ptlrpc_pending); rc = LNetPut(&self, *mdh, ack, - &peer_id, portal, xid, offset, 0); + peer_id, portal, xid, offset, 0); if (unlikely(rc != 0)) { int rc2; /* We're going to get an UNLINK event when I unlink below, @@ -99,7 +97,7 @@ static int ptl_send_buf(struct lnet_handle_md *mdh, void *base, int len, * I fall through and return success here! */ CERROR("LNetPut(%s, %d, %lld) failed: %d\n", - libcfs_id2str(peer_id4), portal, xid, rc); + libcfs_idstr(peer_id), portal, xid, rc); rc2 = LNetMDUnlink(*mdh); LASSERTF(rc2 == 0, "rc2 = %d\n", rc2); } @@ -437,7 +435,7 @@ int ptlrpc_send_reply(struct ptlrpc_request *req, int flags) (rs->rs_difficult && !rs->rs_no_ack) ? LNET_ACK_REQ : LNET_NOACK_REQ, &rs->rs_cb_id, lnet_nid_to_nid4(&req->rq_self), - req->rq_source, + &req->rq_source, ptlrpc_req2svc(req)->srv_rep_portal, req->rq_rep_mbits ? req->rq_rep_mbits : req->rq_xid, req->rq_reply_off, NULL); @@ -729,7 +727,7 @@ int ptl_send_rpc(struct ptlrpc_request *request, int noreply) request->rq_reqbuf, request->rq_reqdata_len, LNET_NOACK_REQ, &request->rq_req_cbid, LNET_NID_ANY, - lnet_pid_to_pid4(&connection->c_peer), + &connection->c_peer, request->rq_request_portal, request->rq_xid, 0, &bulk_cookie); if (likely(rc == 0)) From patchwork Sun Sep 18 05:21:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979310 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 33D0AC6FA8B for ; Sun, 18 Sep 2022 05:22:32 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbm54BNJz1yBh; Sat, 17 Sep 2022 22:22:29 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVblz1XpGz1y78 for ; Sat, 17 Sep 2022 22:22:23 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id DF50E8F04; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id D658DE8B94; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:21:56 -0400 Message-Id: <1663478534-19917-7-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 06/24] lustre: ptlrpc: change bd_sender in ptlrpc_bulk_frag_ops X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mr NeilBrown bd_sender in struct ptlrpc_bulk_frag_ops is now 'struct lnet_nid'. WC-bug-id: https://jira.whamcloud.com/browse/LU-10391 Lustre-commit: d096ca267bb74fa3e1 ("LU-10391 ptlrpc: change bd_sender in ptlrpc_bulk_frag_ops") Signed-off-by: Mr NeilBrown Reviewed-on: https://review.whamcloud.com/44640 Reviewed-by: Frank Sehr Reviewed-by: James Simmons Reviewed-by: Chris Horn Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_net.h | 2 +- fs/lustre/osc/osc_request.c | 4 ++-- fs/lustre/ptlrpc/events.c | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/fs/lustre/include/lustre_net.h b/fs/lustre/include/lustre_net.h index d9aa530..f8d2861 100644 --- a/fs/lustre/include/lustre_net.h +++ b/fs/lustre/include/lustre_net.h @@ -1211,7 +1211,7 @@ struct ptlrpc_bulk_desc { u64 bd_last_mbits; struct ptlrpc_cb_id bd_cbid; /* network callback info */ - lnet_nid_t bd_sender; /* stash event::sender */ + struct lnet_nid bd_sender; /* stash event::sender */ int bd_md_count; /* # valid entries in bd_mds */ int bd_md_max_brw; /* max entries in bd_mds */ diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index d66185b..7577fad 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -2253,9 +2253,9 @@ static int osc_brw_fini_request(struct ptlrpc_request *req, int rc) goto out; if (req->rq_bulk && - lnet_nid_to_nid4(&peer->nid) != req->rq_bulk->bd_sender) { + !nid_same(&peer->nid, &req->rq_bulk->bd_sender)) { via = " via "; - router = libcfs_nid2str(req->rq_bulk->bd_sender); + router = libcfs_nidstr(&req->rq_bulk->bd_sender); } if (server_cksum != client_cksum) { diff --git a/fs/lustre/ptlrpc/events.c b/fs/lustre/ptlrpc/events.c index e6313af..17ef775 100644 --- a/fs/lustre/ptlrpc/events.c +++ b/fs/lustre/ptlrpc/events.c @@ -213,7 +213,7 @@ void client_bulk_callback(struct lnet_event *ev) if (ev->type != LNET_EVENT_UNLINK && ev->status == 0) { desc->bd_nob_transferred += ev->mlength; - desc->bd_sender = lnet_nid_to_nid4(&ev->sender); + desc->bd_sender = ev->sender; } else { /* start reconnect and resend if network error hit */ spin_lock(&req->rq_lock); From patchwork Sun Sep 18 05:21:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979316 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7A6CCC6FA8B for ; Sun, 18 Sep 2022 05:22:43 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbmK5RPtz1yFN; Sat, 17 Sep 2022 22:22:41 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVbm127sxz1y7v for ; Sat, 17 Sep 2022 22:22:25 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id E15618F08; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id D9B6DE8BA5; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:21:57 -0400 Message-Id: <1663478534-19917-8-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 07/24] lustre: ptlrpc: pass lnet_nid for self to ptl_send_buf() X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mr NeilBrown The 'self' arg to ptl_send_buf() is now a pointer to a 'struct lnet_nid', and can be NULL meaning "ANY NID". LNetPut() already accepts NULL as the self pointer. WC-bug-id: https://jira.whamcloud.com/browse/LU-10391 Lustre-commit: 3f7e728d91dd9f113 ("LU-10391 ptlrpc: pass lnet_nid for self to ptl_send_buf()") Signed-off-by: Mr NeilBrown Reviewed-on: https://review.whamcloud.com/44641 Reviewed-by: Frank Sehr Reviewed-by: James Simmons Reviewed-by: Chris Horn Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/ptlrpc/niobuf.c | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/fs/lustre/ptlrpc/niobuf.c b/fs/lustre/ptlrpc/niobuf.c index 8f19e92..670bfb0 100644 --- a/fs/lustre/ptlrpc/niobuf.c +++ b/fs/lustre/ptlrpc/niobuf.c @@ -46,15 +46,12 @@ */ static int ptl_send_buf(struct lnet_handle_md *mdh, void *base, int len, enum lnet_ack_req ack, struct ptlrpc_cb_id *cbid, - lnet_nid_t self4, struct lnet_processid *peer_id, + struct lnet_nid *self, struct lnet_processid *peer_id, int portal, u64 xid, unsigned int offset, struct lnet_handle_md *bulk_cookie) { int rc; struct lnet_md md; - struct lnet_nid self; - - lnet_nid4_to_nid(self4, &self); LASSERT(portal != 0); CDEBUG(D_INFO, "peer_id %s\n", libcfs_idstr(peer_id)); @@ -88,7 +85,7 @@ static int ptl_send_buf(struct lnet_handle_md *mdh, void *base, int len, percpu_ref_get(&ptlrpc_pending); - rc = LNetPut(&self, *mdh, ack, + rc = LNetPut(self, *mdh, ack, peer_id, portal, xid, offset, 0); if (unlikely(rc != 0)) { int rc2; @@ -434,7 +431,7 @@ int ptlrpc_send_reply(struct ptlrpc_request *req, int flags) rc = ptl_send_buf(&rs->rs_md_h, rs->rs_repbuf, rs->rs_repdata_len, (rs->rs_difficult && !rs->rs_no_ack) ? LNET_ACK_REQ : LNET_NOACK_REQ, - &rs->rs_cb_id, lnet_nid_to_nid4(&req->rq_self), + &rs->rs_cb_id, &req->rq_self, &req->rq_source, ptlrpc_req2svc(req)->srv_rep_portal, req->rq_rep_mbits ? req->rq_rep_mbits : req->rq_xid, @@ -726,7 +723,7 @@ int ptl_send_rpc(struct ptlrpc_request *request, int noreply) rc = ptl_send_buf(&request->rq_req_md_h, request->rq_reqbuf, request->rq_reqdata_len, LNET_NOACK_REQ, &request->rq_req_cbid, - LNET_NID_ANY, + NULL, &connection->c_peer, request->rq_request_portal, request->rq_xid, 0, &bulk_cookie); From patchwork Sun Sep 18 05:21:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979314 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 53E1CC6FA8B for ; Sun, 18 Sep 2022 05:22:40 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbmF2scXz1y6q; Sat, 17 Sep 2022 22:22:37 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVblz6J6rz1y7H for ; Sat, 17 Sep 2022 22:22:23 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id E0ED08F07; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id DCDFEE8BA9; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:21:58 -0400 Message-Id: <1663478534-19917-9-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 08/24] lustre: llite: don't use a kms if it invalid. X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexey Lyashkov , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Alexey Lyashkov Lockless DIO don't update a KMS as other IO type does, it caused a situation when next read don't known a real file size to be read. Lets avoid using an invalid KMS. Fixes: bf18998820 ("lustre: clio: turn on lockless for some kind of IO") WC-bug-id: https://jira.whamcloud.com/browse/LU-15829 Lustre-commit: dc907414db16d99e7 ("LU-15829 llite: don't use a kms if it invalid.") Signed-off-by: Alexey Lyashkov Reviewed-on: https://review.whamcloud.com/47395 Reviewed-by: Shaun Tancheff Reviewed-by: Bobi Jam Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/cl_object.h | 2 ++ fs/lustre/llite/vvp_io.c | 2 +- fs/lustre/lov/lov_internal.h | 2 +- fs/lustre/lov/lov_merge.c | 46 ++++++++++++++++++++++++------------------- fs/lustre/lov/lov_object.c | 23 +++------------------- 5 files changed, 33 insertions(+), 42 deletions(-) diff --git a/fs/lustre/include/cl_object.h b/fs/lustre/include/cl_object.h index 3253f1c..492c704 100644 --- a/fs/lustre/include/cl_object.h +++ b/fs/lustre/include/cl_object.h @@ -138,6 +138,8 @@ struct cl_device { struct cl_attr { /** Object size, in bytes */ loff_t cat_size; + + unsigned int cat_kms_valid:1; /** * Known minimal size, in bytes. * diff --git a/fs/lustre/llite/vvp_io.c b/fs/lustre/llite/vvp_io.c index 75c5224..6fd2e2dd 100644 --- a/fs/lustre/llite/vvp_io.c +++ b/fs/lustre/llite/vvp_io.c @@ -151,7 +151,7 @@ static int vvp_prep_size(const struct lu_env *env, struct cl_object *obj, result = cl_object_attr_get(env, obj, attr); if (result == 0) { kms = attr->cat_kms; - if (pos > kms) { + if (pos > kms || !attr->cat_kms_valid) { /* * A glimpse is necessary to determine whether we * return a short read (B) or some zeroes at the end diff --git a/fs/lustre/lov/lov_internal.h b/fs/lustre/lov/lov_internal.h index 8c0b6fa..cd6bc7a 100644 --- a/fs/lustre/lov/lov_internal.h +++ b/fs/lustre/lov/lov_internal.h @@ -271,7 +271,7 @@ struct lov_request_set { /* lov_merge.c */ int lov_merge_lvb_kms(struct lov_stripe_md *lsm, int index, - struct ost_lvb *lvb, u64 *kms_place); + struct cl_attr *attr); /* lov_offset.c */ u64 stripe_width(struct lov_stripe_md *lsm, unsigned int index); diff --git a/fs/lustre/lov/lov_merge.c b/fs/lustre/lov/lov_merge.c index 8800c5f..5cc5928 100644 --- a/fs/lustre/lov/lov_merge.c +++ b/fs/lustre/lov/lov_merge.c @@ -32,6 +32,7 @@ #define DEBUG_SUBSYSTEM S_LOV +#include #include #include "lov_internal.h" @@ -41,26 +42,24 @@ * uptodate time on the local client. */ int lov_merge_lvb_kms(struct lov_stripe_md *lsm, int index, - struct ost_lvb *lvb, u64 *kms_place) + struct cl_attr *attr) { struct lov_stripe_md_entry *lse = lsm->lsm_entries[index]; u64 size = 0; u64 kms = 0; u64 blocks = 0; - s64 current_mtime = lvb->lvb_mtime; - s64 current_atime = lvb->lvb_atime; - s64 current_ctime = lvb->lvb_ctime; + /* XXX: timestamps can be negative by sanity:test_39m, + * how can it be? + */ + s64 current_mtime = LLONG_MIN; + s64 current_atime = LLONG_MIN; + s64 current_ctime = LLONG_MIN; int i; int rc = 0; assert_spin_locked(&lsm->lsm_lock); LASSERT(lsm->lsm_lock_owner == current->pid); - CDEBUG(D_INODE, - "MDT ID " DOSTID " initial value: s=%llu m=%llu a=%llu c=%llu b=%llu\n", - POSTID(&lsm->lsm_oi), lvb->lvb_size, lvb->lvb_mtime, - lvb->lvb_atime, lvb->lvb_ctime, lvb->lvb_blocks); - for (i = 0; i < lse->lsme_stripe_count; i++) { struct lov_oinfo *loi = lse->lsme_oinfo[i]; u64 lov_size, tmpsize; @@ -70,7 +69,12 @@ int lov_merge_lvb_kms(struct lov_stripe_md *lsm, int index, continue; } - tmpsize = loi->loi_kms; + if (loi->loi_kms_valid) { + attr->cat_kms_valid = 1; + tmpsize = loi->loi_kms; + } else { + tmpsize = 0; + } lov_size = lov_stripe_size(lsm, index, tmpsize, i); if (lov_size > kms) kms = lov_size; @@ -91,18 +95,20 @@ int lov_merge_lvb_kms(struct lov_stripe_md *lsm, int index, current_ctime = loi->loi_lvb.lvb_ctime; CDEBUG(D_INODE, - "MDT ID " DOSTID " on OST[%u]: s=%llu m=%llu a=%llu c=%llu b=%llu\n", + "MDT ID " DOSTID " on OST[%u]: s=%llu (%d) m=%llu a=%llu c=%llu b=%llu\n", POSTID(&lsm->lsm_oi), loi->loi_ost_idx, - loi->loi_lvb.lvb_size, loi->loi_lvb.lvb_mtime, - loi->loi_lvb.lvb_atime, loi->loi_lvb.lvb_ctime, - loi->loi_lvb.lvb_blocks); + loi->loi_lvb.lvb_size, loi->loi_kms_valid, + loi->loi_lvb.lvb_mtime, loi->loi_lvb.lvb_atime, + loi->loi_lvb.lvb_ctime, loi->loi_lvb.lvb_blocks); } - *kms_place = kms; - lvb->lvb_size = size; - lvb->lvb_blocks = blocks; - lvb->lvb_mtime = current_mtime; - lvb->lvb_atime = current_atime; - lvb->lvb_ctime = current_ctime; + if (!rc) { + attr->cat_kms = kms; + attr->cat_size = size; + attr->cat_mtime = current_mtime; + attr->cat_atime = current_atime; + attr->cat_ctime = current_ctime; + attr->cat_blocks = blocks; + } return rc; } diff --git a/fs/lustre/lov/lov_object.c b/fs/lustre/lov/lov_object.c index 3934a98..064764c 100644 --- a/fs/lustre/lov/lov_object.c +++ b/fs/lustre/lov/lov_object.c @@ -373,9 +373,7 @@ static int lov_attr_get_raid0(const struct lu_env *env, struct lov_object *lov, { struct lov_layout_raid0 *r0 = &lle->lle_raid0; struct lov_stripe_md *lsm = lov->lo_lsm; - struct ost_lvb *lvb = &lov_env_info(env)->lti_lvb; struct cl_attr *attr = &r0->lo_attr; - u64 kms = 0; int result = 0; if (r0->lo_attr_valid) { @@ -383,21 +381,6 @@ static int lov_attr_get_raid0(const struct lu_env *env, struct lov_object *lov, return 0; } - memset(lvb, 0, sizeof(*lvb)); - - /* XXX: timestamps can be negative by sanity:test_39m, - * how can it be? - */ - lvb->lvb_atime = LLONG_MIN; - lvb->lvb_ctime = LLONG_MIN; - lvb->lvb_mtime = LLONG_MIN; - - /* - * XXX that should be replaced with a loop over sub-objects, - * doing cl_object_attr_get() on them. But for now, let's - * reuse old lov code. - */ - /* * XXX take lsm spin-lock to keep lov_merge_lvb_kms() * happy. It's not needed, because new code uses @@ -405,11 +388,9 @@ static int lov_attr_get_raid0(const struct lu_env *env, struct lov_object *lov, * sub-object attributes. */ lov_stripe_lock(lsm); - result = lov_merge_lvb_kms(lsm, index, lvb, &kms); + result = lov_merge_lvb_kms(lsm, index, attr); lov_stripe_unlock(lsm); if (result == 0) { - cl_lvb2attr(attr, lvb); - attr->cat_kms = kms; r0->lo_attr_valid = 1; *lov_attr = attr; } @@ -1042,6 +1023,8 @@ static int lov_attr_get_composite(const struct lu_env *env, lov_attr->cat_ctime, lov_attr->cat_blocks); /* merge results */ + if (lov_attr->cat_kms_valid) + attr->cat_kms_valid = 1; attr->cat_blocks += lov_attr->cat_blocks; if (attr->cat_size < lov_attr->cat_size) attr->cat_size = lov_attr->cat_size; From patchwork Sun Sep 18 05:21:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979315 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 51A81C6FA86 for ; Sun, 18 Sep 2022 05:22:40 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbmF292Yz1yFs; Sat, 17 Sep 2022 22:22:37 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVbm04PFlz1y6n for ; Sat, 17 Sep 2022 22:22:24 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id E2B2C8F09; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id E00BFE8B93; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:21:59 -0400 Message-Id: <1663478534-19917-10-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 09/24] lustre: mdc: check/grab import before access X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Alex Zhuravlev to ensure the import doesn't disappear while being accessed via sysfs. WC-bug-id: https://jira.whamcloud.com/browse/LU-14441 Lustre-commit: b8416320b381ae8a6 ("LU-14441 mdc: check/grab import before access") Signed-off-by: Alex Zhuravlev Reviewed-on: https://review.whamcloud.com/41681 Reviewed-by: Andreas Dilger Reviewed-by: Neil Brown Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/mdc/lproc_mdc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/fs/lustre/mdc/lproc_mdc.c b/fs/lustre/mdc/lproc_mdc.c index 3de6533..cb3744a 100644 --- a/fs/lustre/mdc/lproc_mdc.c +++ b/fs/lustre/mdc/lproc_mdc.c @@ -340,6 +340,7 @@ static ssize_t max_rpcs_in_flight_store(struct kobject *kobj, { struct obd_device *obd = container_of(kobj, struct obd_device, obd_kset.kobj); + struct obd_import *imp; unsigned int val; int rc; @@ -347,11 +348,10 @@ static ssize_t max_rpcs_in_flight_store(struct kobject *kobj, if (rc) return rc; - rc = obd_set_max_rpcs_in_flight(&obd->u.cli, val); - if (rc) - count = rc; + with_imp_locked(obd, imp, rc) + rc = obd_set_max_rpcs_in_flight(&obd->u.cli, val); - return count; + return rc ? rc : count; } LUSTRE_RW_ATTR(max_rpcs_in_flight); @@ -372,6 +372,7 @@ static ssize_t max_mod_rpcs_in_flight_store(struct kobject *kobj, { struct obd_device *obd = container_of(kobj, struct obd_device, obd_kset.kobj); + struct obd_import *imp; u16 val; int rc; @@ -379,11 +380,10 @@ static ssize_t max_mod_rpcs_in_flight_store(struct kobject *kobj, if (rc) return rc; - rc = obd_set_max_mod_rpcs_in_flight(&obd->u.cli, val); - if (rc) - count = rc; + with_imp_locked(obd, imp, rc) + rc = obd_set_max_mod_rpcs_in_flight(&obd->u.cli, val); - return count; + return rc ? rc : count; } LUSTRE_RW_ATTR(max_mod_rpcs_in_flight); From patchwork Sun Sep 18 05:22:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979317 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CBA52C32771 for ; Sun, 18 Sep 2022 05:22:43 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbmL12n8z1y7K; Sat, 17 Sep 2022 22:22:42 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVbm16yMMz1yBg for ; Sat, 17 Sep 2022 22:22:25 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id E61708F0A; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id E3C3837C; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:22:00 -0400 Message-Id: <1663478534-19917-11-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 10/24] lustre: llog: handle -EBADR for catalog processing X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mikhail Pershin , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mikhail Pershin Llog catalog processing might retry to get the last llog block to check for new records if any. That might return -EBADR code which should be considered as valid. Previously -EIO was returned in all cases. WC-bug-id: https://jira.whamcloud.com/browse/LU-16052 Lustre-commit: e260f751f2a21fa12 ("LU-16052 llog: handle -EBADR for catalog processing") Signed-off-by: Mikhail Pershin Reviewed-on: https://review.whamcloud.com/48070 Reviewed-by: Andreas Dilger Reviewed-by: Etienne AUJAMES Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/obdclass/llog.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/lustre/obdclass/llog.c b/fs/lustre/obdclass/llog.c index 0cc64ce..38904b6 100644 --- a/fs/lustre/obdclass/llog.c +++ b/fs/lustre/obdclass/llog.c @@ -341,8 +341,8 @@ static int llog_process_thread(void *arg) /* we`ve tried to reread the chunk, but there is no * new records */ - if (rc == -EIO && repeated && (chunk_offset + buf_offset) == - cur_offset) { + if (repeated && (chunk_offset + buf_offset) == cur_offset && + (rc == -EBADR || rc == -EIO)) { rc = 0; goto out; } From patchwork Sun Sep 18 05:22:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979319 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 269A9C32771 for ; Sun, 18 Sep 2022 05:22:47 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbmQ63Wqz1yDy; Sat, 17 Sep 2022 22:22:46 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVbm24grWz1yBr for ; Sat, 17 Sep 2022 22:22:26 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id EA5928F0B; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id E6EE2F64; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:22:01 -0400 Message-Id: <1663478534-19917-12-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 11/24] lnet: Always use ping reply to set route lr_alive X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Chris Horn , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Chris Horn We currently process discovery ping replies in different ways depending on whether the gateway has discovery enabled or disabled (or the local peer doing the processing has discovery enabled or disabled). When DD is disabled we process the ping reply to set the lr_alive field of lnet_route because the peer objects for non-MR routers do not contain all the information needed to calculate the route aliveness when a message is being sent. When DD is enabled then we don't do any special processing of the ping reply. We simply let discovery update the NI status for the GW's peer NIs and then we calculate the route aliveness on every send. We issue discovery pings to routers every alive_router_check_interval seconds (default 60), but we calculate route aliveness on every send to a remote network (1000s of times per seconds). Thus, it is better to slightly duplicate the effort expended when we receive a discovery reply so that we can avoid calculating route aliveness on every send. Since both lr_alive and hop type are being set on each ping reply, for both DD enabled and disabled cases, we can remove the code for updating lr_alive and hop type from lnet_router_discovery_complete(). If discover encounters a fatal error, we still set the status of each peer NI, as well as all routes, to down in lnet_router_discovery_complete(). WC-bug-id: https://jira.whamcloud.com/browse/LU-15595 Lustre-commit: 1ea6c87d415144522 ("LU-15595 lnet: Always use ping reply to set route lr_alive") Signed-off-by: Chris Horn Reviewed-on: https://review.whamcloud.com/46624 Reviewed-by: Cyril Bordage Reviewed-by: Andriy Skulysh Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- include/linux/lnet/lib-lnet.h | 3 +- net/lnet/lnet/peer.c | 14 ++++----- net/lnet/lnet/router.c | 71 +++++++++---------------------------------- 3 files changed, 23 insertions(+), 65 deletions(-) diff --git a/include/linux/lnet/lib-lnet.h b/include/linux/lnet/lib-lnet.h index 1d9b8c7..fc086da 100644 --- a/include/linux/lnet/lib-lnet.h +++ b/include/linux/lnet/lib-lnet.h @@ -848,7 +848,8 @@ struct socket *lnet_sock_connect(int interface, int local_port, void lnet_consolidate_routes_locked(struct lnet_peer *orig_lp, struct lnet_peer *new_lp); void lnet_router_discovery_complete(struct lnet_peer *lp); -void lnet_router_discovery_ping_reply(struct lnet_peer *lp); +void lnet_router_discovery_ping_reply(struct lnet_peer *lp, + struct lnet_ping_buffer *pbuf); int lnet_monitor_thr_start(void); void lnet_monitor_thr_stop(void); diff --git a/net/lnet/lnet/peer.c b/net/lnet/lnet/peer.c index 8d81a7d..e7c3c83 100644 --- a/net/lnet/lnet/peer.c +++ b/net/lnet/lnet/peer.c @@ -2745,14 +2745,6 @@ static void lnet_peer_clear_discovery_error(struct lnet_peer *lp) out: lp->lp_state &= ~LNET_PEER_PING_SENT; spin_unlock(&lp->lp_lock); - - lnet_net_lock(LNET_LOCK_EX); - /* If this peer is a gateway, call the routing callback to - * handle the ping reply - */ - if (lp->lp_rtr_refcount > 0) - lnet_router_discovery_ping_reply(lp); - lnet_net_unlock(LNET_LOCK_EX); } /* @@ -3052,6 +3044,12 @@ static int lnet_peer_merge_data(struct lnet_peer *lp, */ rc = 0; out: + /* If this peer is a gateway, invoke the routing callback to update + * the associated route status + */ + if (lp->lp_rtr_refcount > 0) + lnet_router_discovery_ping_reply(lp, pbuf); + kfree(curnis); kfree(addnis); kfree(delnis); diff --git a/net/lnet/lnet/router.c b/net/lnet/lnet/router.c index 146647c..5d1e5a05a 100644 --- a/net/lnet/lnet/router.c +++ b/net/lnet/lnet/router.c @@ -153,6 +153,7 @@ static void lnet_del_route_from_rnet(struct lnet_nid *gw_nid, void lnet_move_route(struct lnet_route *route, struct lnet_peer *lp, struct list_head *rt_list) +__must_hold(&the_lnet.ln_api_mutex) { struct lnet_remotenet *rnet; struct list_head zombies; @@ -378,61 +379,31 @@ bool lnet_is_route_alive(struct lnet_route *route) } } -static void -lnet_set_route_hop_type(struct lnet_peer *gw, struct lnet_route *route) -{ - struct lnet_peer_net *lpn; - bool single_hop = false; - - list_for_each_entry(lpn, &gw->lp_peer_nets, lpn_peer_nets) { - if (route->lr_net == lpn->lpn_net_id) { - single_hop = true; - break; - } - } - route->lr_single_hop = single_hop; - lnet_check_route_inconsistency(route); -} - -/* Must hold net_lock/EX */ +/* Routes are added and removed under both ln_api_mutex and net_lock/EX + * Since we are not modifying anything we simply require the ln_api_mutex be + * held so that things are not modified underneath us + */ void -lnet_router_discovery_ping_reply(struct lnet_peer *lp) +lnet_router_discovery_ping_reply(struct lnet_peer *lp, + struct lnet_ping_buffer *pbuf) +__must_hold(&the_lnet.ln_api_mutex) { - struct lnet_ping_buffer *pbuf = lp->lp_data; struct lnet_peer_net *llpn; struct lnet_route *route; bool single_hop = false; bool net_up = false; - unsigned int lp_state; u32 net; int i; - spin_lock(&lp->lp_lock); - lp_state = lp->lp_state; - - /* only handle replies if discovery is disabled. */ - if (!lnet_is_discovery_disabled_locked(lp)) { - spin_unlock(&lp->lp_lock); - return; - } - - spin_unlock(&lp->lp_lock); - - if (lp_state & LNET_PEER_PING_FAILED || - pbuf->pb_info.pi_features & LNET_PING_FEAT_RTE_DISABLED) { - CDEBUG(D_NET, "Set routes down for gw %s because %s %d\n", - libcfs_nidstr(&lp->lp_primary_nid), - lp_state & LNET_PEER_PING_FAILED ? "ping failed" : - "route feature is disabled", lp->lp_ping_error); - /* If the ping failed or the peer has routing disabled then - * mark the routes served by this peer down - */ + if (pbuf->pb_info.pi_features & LNET_PING_FEAT_RTE_DISABLED) { + CERROR("Peer %s is being used as a gateway but routing feature is not turned on\n", + libcfs_nidstr(&lp->lp_primary_nid)); list_for_each_entry(route, &lp->lp_routes, lr_gwlist) lnet_set_route_aliveness(route, false); return; } - CDEBUG(D_NET, "Discovery is disabled. Processing reply for gw: %s:%d\n", + CDEBUG(D_NET, "Processing reply for gw: %s nnis %d\n", libcfs_nidstr(&lp->lp_primary_nid), pbuf->pb_info.pi_nnis); /* examine the ping response to determine if the routes on that @@ -495,22 +466,8 @@ bool lnet_is_route_alive(struct lnet_route *route) lp->lp_alive = lp->lp_dc_error == 0; spin_unlock(&lp->lp_lock); - if (!lp->lp_dc_error) { - /* ping replies are being handled when discovery is disabled */ - if (lnet_is_discovery_disabled_locked(lp)) - return; - - /* mark single-hop routes. If the remote net is not configured - * on the gateway we assume this is intentional and we mark the - * gateway as multi-hop - */ - list_for_each_entry(route, &lp->lp_routes, lr_gwlist) { - lnet_set_route_aliveness(route, true); - lnet_set_route_hop_type(lp, route); - } - + if (!lp->lp_dc_error) return; - } /* We do not send messages directly to the remote interfaces * of an LNet router. As such, we rely on the PING response @@ -642,6 +599,7 @@ static void lnet_shuffle_seed(void) int lnet_add_route(u32 net, u32 hops, struct lnet_nid *gateway, u32 priority, u32 sensitivity) +__must_hold(&the_lnet.ln_api_mutex) { struct list_head *route_entry; struct lnet_remotenet *rnet; @@ -821,6 +779,7 @@ static void lnet_shuffle_seed(void) int lnet_del_route(u32 net, struct lnet_nid *gw) +__must_hold(&the_lnet.ln_api_mutex) { LIST_HEAD(rnet_zombies); struct lnet_remotenet *rnet; From patchwork Sun Sep 18 05:22:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979318 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B2F7DC6FA86 for ; Sun, 18 Sep 2022 05:22:46 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbmQ2jJ2z1yBt; Sat, 17 Sep 2022 22:22:46 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVbm32rYNz1yC9 for ; Sat, 17 Sep 2022 22:22:27 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id ED15E8F0C; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id E9EEC27E1; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:22:02 -0400 Message-Id: <1663478534-19917-13-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 12/24] lustre: clio: remove vvp_page_print() X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: "John L. Hammond" Remove vvp_page_print() by placing equivalent code in cl_page_print(). WC-bug-id: https://jira.whamcloud.com/browse/LU-10994 Lustre-commit: bf1d1b0e41ff245f5 ("LU-10994 clio: remove vvp_page_print()") Signed-off-by: John L. Hammond Reviewed-on: https://review.whamcloud.com/47398 Reviewed-by: Patrick Farrell Reviewed-by: Bobi Jam Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/vvp_page.c | 24 ------------------------ fs/lustre/obdclass/cl_page.c | 23 +++++++++++++++++++---- 2 files changed, 19 insertions(+), 28 deletions(-) diff --git a/fs/lustre/llite/vvp_page.c b/fs/lustre/llite/vvp_page.c index db1cd7c1..f27bbfa 100644 --- a/fs/lustre/llite/vvp_page.c +++ b/fs/lustre/llite/vvp_page.c @@ -214,28 +214,6 @@ static int vvp_page_make_ready(const struct lu_env *env, return result; } -static int vvp_page_print(const struct lu_env *env, - const struct cl_page_slice *slice, - void *cookie, lu_printer_t printer) -{ - struct vvp_page *vpg = cl2vvp_page(slice); - struct page *vmpage = vpg->vpg_page; - - (*printer)(env, cookie, - LUSTRE_VVP_NAME"-page@%p vm@%p ", vpg, vmpage); - if (vmpage) { - (*printer)(env, cookie, "%lx %d:%d %lx %lu %slru", - (long)vmpage->flags, page_count(vmpage), - page_mapcount(vmpage), vmpage->private, - vmpage->index, - list_empty(&vmpage->lru) ? "not-" : ""); - } - - (*printer)(env, cookie, "\n"); - - return 0; -} - static int vvp_page_fail(const struct lu_env *env, const struct cl_page_slice *slice) { @@ -249,7 +227,6 @@ static int vvp_page_fail(const struct lu_env *env, static const struct cl_page_operations vvp_page_ops = { .cpo_discard = vvp_page_discard, - .cpo_print = vvp_page_print, .io = { [CRT_READ] = { .cpo_prep = vvp_page_prep_read, @@ -265,7 +242,6 @@ static int vvp_page_fail(const struct lu_env *env, }; static const struct cl_page_operations vvp_transient_page_ops = { - .cpo_print = vvp_page_print, }; int vvp_page_init(const struct lu_env *env, struct cl_object *obj, diff --git a/fs/lustre/obdclass/cl_page.c b/fs/lustre/obdclass/cl_page.c index 6319c3d..6fbbfa8 100644 --- a/fs/lustre/obdclass/cl_page.c +++ b/fs/lustre/obdclass/cl_page.c @@ -1017,21 +1017,36 @@ void cl_page_header_print(const struct lu_env *env, void *cookie, * Prints human readable representation of @cl_page to the @f. */ void cl_page_print(const struct lu_env *env, void *cookie, - lu_printer_t printer, const struct cl_page *cl_page) + lu_printer_t printer, const struct cl_page *cp) { + struct page *vmpage = cp->cp_vmpage; const struct cl_page_slice *slice; int result = 0; int i; - cl_page_header_print(env, cookie, printer, cl_page); - cl_page_slice_for_each(cl_page, slice, i) { + cl_page_header_print(env, cookie, printer, cp); + + (*printer)(env, cookie, "vmpage @%p", vmpage); + + if (vmpage) { + (*printer)(env, cookie, " %lx %d:%d %lx %lu %slru", + (long)vmpage->flags, page_count(vmpage), + page_mapcount(vmpage), vmpage->private, + page_index(vmpage), + list_empty(&vmpage->lru) ? "not-" : ""); + } + + (*printer)(env, cookie, "\n"); + + cl_page_slice_for_each(cp, slice, i) { if (slice->cpl_ops->cpo_print) result = (*slice->cpl_ops->cpo_print)(env, slice, cookie, printer); if (result != 0) break; } - (*printer)(env, cookie, "end page@%p\n", cl_page); + + (*printer)(env, cookie, "end page@%p\n", cp); } EXPORT_SYMBOL(cl_page_print); From patchwork Sun Sep 18 05:22:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979331 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BB1CBC6FA86 for ; Sun, 18 Sep 2022 05:23:19 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbn33447z1yDm; Sat, 17 Sep 2022 22:23:19 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVbm413d5z1yCV for ; Sat, 17 Sep 2022 22:22:28 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id F02CD8F0D; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id ED238E8B94; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:22:03 -0400 Message-Id: <1663478534-19917-14-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 13/24] lustre: clio: remove cpo_prep and cpo_make_ready X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: "John L. Hammond" Remove the cpo_prep and cpo_make_ready methods from struct cl_page_operations. These methods were only implemented by the vvp layer and so they can be easily inlined into cl_page_prep() and cl_page_make_ready(). WC-bug-id: https://jira.whamcloud.com/browse/LU-10994 Lustre-commit: ca161cfbcbf04bffb ("LU-10994 clio: remove cpo_prep and cpo_make_ready") Signed-off-by: John L. Hammond Reviewed-on: https://review.whamcloud.com/47399 Reviewed-by: James Simmons Reviewed-by: Andreas Dilger Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/cl_object.h | 28 +---------- fs/lustre/llite/vvp_page.c | 83 -------------------------------- fs/lustre/obdclass/cl_page.c | 109 ++++++++++++++++++++++-------------------- 3 files changed, 59 insertions(+), 161 deletions(-) diff --git a/fs/lustre/include/cl_object.h b/fs/lustre/include/cl_object.h index 492c704..b19d521 100644 --- a/fs/lustre/include/cl_object.h +++ b/fs/lustre/include/cl_object.h @@ -895,22 +895,8 @@ struct cl_page_operations { */ struct { /** - * Called when a page is submitted for a transfer as a part of - * cl_page_list. - * - * Return: 0 if page is eligible for submission; - * -EALREADY skip this page; - * -ve if error. - * - * \see cl_page_prep() - */ - int (*cpo_prep)(const struct lu_env *env, - const struct cl_page_slice *slice, - struct cl_io *io); - /** * Completion handler. This is guaranteed to be eventually - * fired after cl_page_operations::cpo_prep() or - * cl_page_operations::cpo_make_ready() call. + * fired after cl_page_prep() or cl_page_make_ready() call. * * This method can be called in a non-blocking context. It is * guaranteed however, that the page involved and its object @@ -922,18 +908,6 @@ struct cl_page_operations { void (*cpo_completion)(const struct lu_env *env, const struct cl_page_slice *slice, int ioret); - /** - * Called when cached page is about to be added to the - * ptlrpc request as a part of req formation. - * - * Return 0 proceed with this page; - * -EAGAIN skip this page; - * -ve error. - * - * \see cl_page_make_ready() - */ - int (*cpo_make_ready)(const struct lu_env *env, - const struct cl_page_slice *slice); } io[CRT_NR]; /** * Tell transfer engine that only [to, from] part of a page should be diff --git a/fs/lustre/llite/vvp_page.c b/fs/lustre/llite/vvp_page.c index f27bbfa..7c80908 100644 --- a/fs/lustre/llite/vvp_page.c +++ b/fs/lustre/llite/vvp_page.c @@ -63,33 +63,6 @@ static void vvp_page_discard(const struct lu_env *env, ll_ra_stats_inc(vmpage->mapping->host, RA_STAT_DISCARDED); } -static int vvp_page_prep_read(const struct lu_env *env, - const struct cl_page_slice *slice, - struct cl_io *unused) -{ - /* Skip the page already marked as PG_uptodate. */ - return PageUptodate(cl2vm_page(slice)) ? -EALREADY : 0; -} - -static int vvp_page_prep_write(const struct lu_env *env, - const struct cl_page_slice *slice, - struct cl_io *unused) -{ - struct page *vmpage = cl2vm_page(slice); - struct cl_page *pg = slice->cpl_page; - - LASSERT(PageLocked(vmpage)); - LASSERT(!PageDirty(vmpage)); - - /* ll_writepage path is not a sync write, so need to set page writeback - * flag - */ - if (!pg->cp_sync_io) - set_page_writeback(vmpage); - - return 0; -} - /** * Handles page transfer errors at VM level. * @@ -173,70 +146,14 @@ static void vvp_page_completion_write(const struct lu_env *env, } } -/** - * Implements cl_page_operations::cpo_make_ready() method. - * - * This is called to yank a page from the transfer cache and to send it out as - * a part of transfer. This function try-locks the page. If try-lock failed, - * page is owned by some concurrent IO, and should be skipped (this is bad, - * but hopefully rare situation, as it usually results in transfer being - * shorter than possible). - * - * Return: 0 success, page can be placed into transfer - * - * -EAGAIN page is either used by concurrent IO has been - * truncated. Skip it. - */ -static int vvp_page_make_ready(const struct lu_env *env, - const struct cl_page_slice *slice) -{ - struct page *vmpage = cl2vm_page(slice); - struct cl_page *pg = slice->cpl_page; - int result = 0; - - lock_page(vmpage); - if (clear_page_dirty_for_io(vmpage)) { - LASSERT(pg->cp_state == CPS_CACHED); - /* This actually clears the dirty bit in the radix tree. */ - set_page_writeback(vmpage); - CL_PAGE_HEADER(D_PAGE, env, pg, "readied\n"); - } else if (pg->cp_state == CPS_PAGEOUT) { - /* is it possible for osc_flush_async_page() to already - * make it ready? - */ - result = -EALREADY; - } else { - CL_PAGE_DEBUG(D_ERROR, env, pg, "Unexpecting page state %d.\n", - pg->cp_state); - LBUG(); - } - unlock_page(vmpage); - return result; -} - -static int vvp_page_fail(const struct lu_env *env, - const struct cl_page_slice *slice) -{ - /* - * Cached read? - */ - LBUG(); - - return 0; -} - static const struct cl_page_operations vvp_page_ops = { .cpo_discard = vvp_page_discard, .io = { [CRT_READ] = { - .cpo_prep = vvp_page_prep_read, .cpo_completion = vvp_page_completion_read, - .cpo_make_ready = vvp_page_fail, }, [CRT_WRITE] = { - .cpo_prep = vvp_page_prep_write, .cpo_completion = vvp_page_completion_write, - .cpo_make_ready = vvp_page_make_ready, }, }, }; diff --git a/fs/lustre/obdclass/cl_page.c b/fs/lustre/obdclass/cl_page.c index 6fbbfa8..2e3374c 100644 --- a/fs/lustre/obdclass/cl_page.c +++ b/fs/lustre/obdclass/cl_page.c @@ -826,45 +826,38 @@ static void cl_page_io_start(const struct lu_env *env, } /** - * Prepares page for immediate transfer. cl_page_operations::cpo_prep() is - * called top-to-bottom. Every layer either agrees to submit this page (by - * returning 0), or requests to omit this page (by returning -EALREADY). Layer - * handling interactions with the VM also has to inform VM that page is under - * transfer now. + * Prepares page for immediate transfer. Return -EALREADY if this page + * should be omitted from transfer. */ int cl_page_prep(const struct lu_env *env, struct cl_io *io, - struct cl_page *cl_page, enum cl_req_type crt) + struct cl_page *cp, enum cl_req_type crt) { - const struct cl_page_slice *slice; - int result = 0; - int i; + struct page *vmpage = cp->cp_vmpage; + int rc; - /* - * this has to be called bottom-to-top, so that llite can set up - * PG_writeback without risking other layers deciding to skip this - * page. - */ - if (crt >= CRT_NR) - return -EINVAL; - - if (cl_page->cp_type != CPT_TRANSIENT) { - cl_page_slice_for_each(cl_page, slice, i) { - if (slice->cpl_ops->io[crt].cpo_prep) - result = (*slice->cpl_ops->io[crt].cpo_prep)(env, - slice, - io); - if (result != 0) - break; + if (cp->cp_type == CPT_TRANSIENT) { + /* Nothing to do. */ + } else if (crt == CRT_READ) { + if (PageUptodate(vmpage)) { + rc = -EALREADY; + goto out; } - } + } else { + LASSERT(PageLocked(vmpage)); + LASSERT(!PageDirty(vmpage)); - if (result >= 0) { - result = 0; - cl_page_io_start(env, cl_page, crt); + /* ll_writepage path is not a sync write, so need to + * set page writeback flag + */ + if (!cp->cp_sync_io) + set_page_writeback(vmpage); } + cl_page_io_start(env, cp, crt); + rc = 0; +out: + CL_PAGE_HEADER(D_TRACE, env, cp, "%d %d\n", crt, rc); - CL_PAGE_HEADER(D_TRACE, env, cl_page, "%d %d\n", crt, result); - return result; + return rc; } EXPORT_SYMBOL(cl_page_prep); @@ -919,35 +912,49 @@ void cl_page_completion(const struct lu_env *env, * * \pre cl_page->cp_state == CPS_CACHED * \post cl_page->cp_state == CPS_PAGEIN || cl_page->cp_state == CPS_PAGEOUT - * - * \see cl_page_operations::cpo_make_ready() */ -int cl_page_make_ready(const struct lu_env *env, struct cl_page *cl_page, +int cl_page_make_ready(const struct lu_env *env, struct cl_page *cp, enum cl_req_type crt) { - const struct cl_page_slice *slice; - int result = 0; - int i; + struct page *vmpage = cp->cp_vmpage; + int rc = 0; - if (crt >= CRT_NR) - return -EINVAL; + PASSERT(env, cp, crt == CRT_WRITE); - cl_page_slice_for_each(cl_page, slice, i) { - if (slice->cpl_ops->io[crt].cpo_make_ready) - result = (*slice->cpl_ops->io[crt].cpo_make_ready)(env, - slice); - if (result != 0) - break; + if (cp->cp_type == CPT_TRANSIENT) + goto out; + + lock_page(vmpage); + + if (clear_page_dirty_for_io(vmpage)) { + LASSERT(cp->cp_state == CPS_CACHED); + /* This actually clears the dirty bit in the + * radix tree. + */ + set_page_writeback(vmpage); + CL_PAGE_HEADER(D_PAGE, env, cp, "readied\n"); + } else if (cp->cp_state == CPS_PAGEOUT) { + /* is it possible for osc_flush_async_page() + * to already make it ready? + */ + rc = -EALREADY; + } else { + CL_PAGE_DEBUG(D_ERROR, env, cp, + "unexpecting page state %d\n", + cp->cp_state); + LBUG(); } - if (result >= 0) { - PASSERT(env, cl_page, cl_page->cp_state == CPS_CACHED); - cl_page_io_start(env, cl_page, crt); - result = 0; + unlock_page(vmpage); +out: + if (rc == 0) { + PASSERT(env, cp, cp->cp_state == CPS_CACHED); + cl_page_io_start(env, cp, crt); } - CL_PAGE_HEADER(D_TRACE, env, cl_page, "%d %d\n", crt, result); - return result; + CL_PAGE_HEADER(D_TRACE, env, cp, "%d %d\n", crt, rc); + + return rc; } EXPORT_SYMBOL(cl_page_make_ready); From patchwork Sun Sep 18 05:22:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979321 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 12679C6FA8B for ; Sun, 18 Sep 2022 05:22:54 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbmY5JxHz1yC4; Sat, 17 Sep 2022 22:22:53 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVbm46NVwz1yBh for ; Sat, 17 Sep 2022 22:22:28 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id F21428F0E; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id F0392E8B93; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:22:04 -0400 Message-Id: <1663478534-19917-15-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 14/24] lustre: clio: remove struct vvp_page X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: "John L. Hammond" Remove struct vvp_page and use struct cl_page_slice in its place. Use cp_vmpage in place of vpg_page and cl_page_index() in place of vvp_index(). WC-bug-id: https://jira.whamcloud.com/browse/LU-10994 Lustre-commit: 127570b3e5a2ff018 ("LU-10994 clio: remove struct vvp_page") Signed-off-by: John L. Hammond Reviewed-on: https://review.whamcloud.com/47400 Reviewed-by: James Simmons Reviewed-by: Andreas Dilger Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/cl_object.h | 7 ++++++- fs/lustre/llite/rw.c | 24 ++++++++++-------------- fs/lustre/llite/rw26.c | 5 ++--- fs/lustre/llite/vvp_dev.c | 7 +++---- fs/lustre/llite/vvp_internal.h | 19 ++----------------- fs/lustre/llite/vvp_io.c | 9 +++------ fs/lustre/llite/vvp_object.c | 2 +- fs/lustre/llite/vvp_page.c | 20 ++++++++------------ 8 files changed, 35 insertions(+), 58 deletions(-) diff --git a/fs/lustre/include/cl_object.h b/fs/lustre/include/cl_object.h index b19d521..c816b1d 100644 --- a/fs/lustre/include/cl_object.h +++ b/fs/lustre/include/cl_object.h @@ -963,12 +963,17 @@ struct cl_page_operations { } \ } while (0) -static inline struct page *cl_page_vmpage(struct cl_page *page) +static inline struct page *cl_page_vmpage(const struct cl_page *page) { LASSERT(page->cp_vmpage); return page->cp_vmpage; } +static inline pgoff_t cl_page_index(const struct cl_page *cp) +{ + return cl_page_vmpage(cp)->index; +} + /** * Check if a cl_page is in use. * diff --git a/fs/lustre/llite/rw.c b/fs/lustre/llite/rw.c index 7c4b8ec..2290b31 100644 --- a/fs/lustre/llite/rw.c +++ b/fs/lustre/llite/rw.c @@ -1635,7 +1635,6 @@ int ll_io_read_page(const struct lu_env *env, struct cl_io *io, pgoff_t io_start_index; pgoff_t io_end_index; int rc = 0, rc2 = 0; - struct vvp_page *vpg; if (file) { fd = file->private_data; @@ -1648,7 +1647,6 @@ int ll_io_read_page(const struct lu_env *env, struct cl_io *io, if (page->cp_vmpage && PagePrivate2(page->cp_vmpage)) unlockpage = false; - vpg = cl2vvp_page(cl_object_page_slice(page->cp_obj, page)); uptodate = page->cp_defer_uptodate; if (ll_readahead_enabled(sbi) && !page->cp_ra_updated && ras) { @@ -1658,7 +1656,7 @@ int ll_io_read_page(const struct lu_env *env, struct cl_io *io, flags |= LL_RAS_HIT; if (mmap) flags |= LL_RAS_MMAP; - ras_update(sbi, inode, ras, vvp_index(vpg), flags, io); + ras_update(sbi, inode, ras, cl_page_index(page), flags, io); } cl_2queue_init(queue); @@ -1680,15 +1678,15 @@ int ll_io_read_page(const struct lu_env *env, struct cl_io *io, io_end_index = cl_index(io->ci_obj, io->u.ci_rw.crw_pos + io->u.ci_rw.crw_count - 1); } else { - io_start_index = vvp_index(vpg); - io_end_index = vvp_index(vpg); + io_start_index = cl_page_index(page); + io_end_index = cl_page_index(page); } if (ll_readahead_enabled(sbi) && ras && !io->ci_rand_read) { pgoff_t skip_index = 0; - if (ras->ras_next_readahead_idx < vvp_index(vpg)) - skip_index = vvp_index(vpg); + if (ras->ras_next_readahead_idx < cl_page_index(page)) + skip_index = cl_page_index(page); rc2 = ll_readahead(env, io, &queue->c2_qin, ras, uptodate, file, skip_index, &ra_start_index); @@ -1698,15 +1696,15 @@ int ll_io_read_page(const struct lu_env *env, struct cl_io *io, CDEBUG(D_READA | (rc2 ? D_IOTRACE : 0), DFID " %d pages read ahead at %lu, triggered by user read at %lu, stride offset %lld, stride length %lld, stride bytes %lld\n", PFID(ll_inode2fid(inode)), rc2, ra_start_index, - vvp_index(vpg), ras->ras_stride_offset, + cl_page_index(page), ras->ras_stride_offset, ras->ras_stride_length, ras->ras_stride_bytes); - } else if (vvp_index(vpg) == io_start_index && + } else if (cl_page_index(page) == io_start_index && io_end_index - io_start_index > 0) { rc2 = ll_readpages(env, io, &queue->c2_qin, io_start_index + 1, io_end_index); CDEBUG(D_READA, DFID " %d pages read at %lu\n", - PFID(ll_inode2fid(inode)), rc2, vvp_index(vpg)); + PFID(ll_inode2fid(inode)), rc2, cl_page_index(page)); } if (queue->c2_qin.pl_nr > 0) { @@ -1865,7 +1863,6 @@ int ll_readpage(struct file *file, struct page *vmpage) struct ll_file_data *fd = file->private_data; struct ll_readahead_state *ras = &fd->fd_ras; struct lu_env *local_env = NULL; - struct vvp_page *vpg; result = -ENODATA; @@ -1880,7 +1877,6 @@ int ll_readpage(struct file *file, struct page *vmpage) return result; } - vpg = cl2vvp_page(cl_object_page_slice(page->cp_obj, page)); if (page->cp_defer_uptodate) { enum ras_update_flags flags = LL_RAS_HIT; @@ -1892,11 +1888,11 @@ int ll_readpage(struct file *file, struct page *vmpage) * if the page is hit in cache because non cache page * case will be handled by slow read later. */ - ras_update(sbi, inode, ras, vvp_index(vpg), flags, io); + ras_update(sbi, inode, ras, cl_page_index(page), flags, io); /* avoid duplicate ras_update() call */ page->cp_ra_updated = 1; - if (ll_use_fast_io(file, ras, vvp_index(vpg))) + if (ll_use_fast_io(file, ras, cl_page_index(page))) result = 0; } diff --git a/fs/lustre/llite/rw26.c b/fs/lustre/llite/rw26.c index 4f2e68e..cadded4 100644 --- a/fs/lustre/llite/rw26.c +++ b/fs/lustre/llite/rw26.c @@ -452,8 +452,7 @@ static int ll_prepare_partial_page(const struct lu_env *env, struct cl_io *io, { struct cl_attr *attr = vvp_env_thread_attr(env); struct cl_object *obj = io->ci_obj; - struct vvp_page *vpg = cl_object_page_slice(obj, pg); - loff_t offset = cl_offset(obj, vvp_index(vpg)); + loff_t offset = cl_offset(obj, cl_page_index(pg)); int result; cl_object_attr_lock(obj); @@ -470,7 +469,7 @@ static int ll_prepare_partial_page(const struct lu_env *env, struct cl_io *io, * purposes here we can treat it like i_size. */ if (attr->cat_kms <= offset) { - char *kaddr = kmap_atomic(vpg->vpg_page); + char *kaddr = kmap_atomic(pg->cp_vmpage); memset(kaddr, 0, cl_page_size(obj)); kunmap_atomic(kaddr); diff --git a/fs/lustre/llite/vvp_dev.c b/fs/lustre/llite/vvp_dev.c index 99335bd..298720c 100644 --- a/fs/lustre/llite/vvp_dev.c +++ b/fs/lustre/llite/vvp_dev.c @@ -429,15 +429,14 @@ static struct page *vvp_pgcache_current(struct vvp_seq_private *priv) static void vvp_pgcache_page_show(const struct lu_env *env, struct seq_file *seq, struct cl_page *page) { - struct vvp_page *vpg; struct page *vmpage; int has_flags; - vpg = cl2vvp_page(cl_page_at(page, &vvp_device_type)); - vmpage = vpg->vpg_page; + vmpage = page->cp_vmpage; seq_printf(seq, " %5i | %p %p %s %s | %p " DFID "(%p) %lu %u [", 0 /* gen */, - vpg, page, + NULL, /* was vvp_page */ + page, "none", PageWriteback(vmpage) ? "wb" : "-", vmpage, PFID(ll_inode2fid(vmpage->mapping->host)), diff --git a/fs/lustre/llite/vvp_internal.h b/fs/lustre/llite/vvp_internal.h index 0e0da76..17feae6 100644 --- a/fs/lustre/llite/vvp_internal.h +++ b/fs/lustre/llite/vvp_internal.h @@ -209,23 +209,8 @@ struct vvp_object { }; /** - * VVP-private page state. + * Their is no VVP-private page state. */ -struct vvp_page { - struct cl_page_slice vpg_cl; - /** VM page */ - struct page *vpg_page; -}; - -static inline struct vvp_page *cl2vvp_page(const struct cl_page_slice *slice) -{ - return container_of(slice, struct vvp_page, vpg_cl); -} - -static inline pgoff_t vvp_index(struct vvp_page *vvp) -{ - return vvp->vpg_page->index; -} struct vvp_device { struct cl_device vdv_cl; @@ -267,7 +252,7 @@ static inline struct inode *vvp_object_inode(const struct cl_object *obj) static inline struct page *cl2vm_page(const struct cl_page_slice *slice) { - return cl2vvp_page(slice)->vpg_page; + return slice->cpl_page->cp_vmpage; } # define CLOBINVRNT(env, clob, expr) \ diff --git a/fs/lustre/llite/vvp_io.c b/fs/lustre/llite/vvp_io.c index 6fd2e2dd..ef7a3d92 100644 --- a/fs/lustre/llite/vvp_io.c +++ b/fs/lustre/llite/vvp_io.c @@ -1044,15 +1044,13 @@ static bool page_list_sanity_check(struct cl_object *obj, pgoff_t index = CL_PAGE_EOF; cl_page_list_for_each(page, plist) { - struct vvp_page *vpg = cl_object_page_slice(obj, page); - if (index == CL_PAGE_EOF) { - index = vvp_index(vpg); + index = cl_page_index(page); continue; } ++index; - if (index == vvp_index(vpg)) + if (index == cl_page_index(page)) continue; return false; @@ -1438,7 +1436,6 @@ static int vvp_io_fault_start(const struct lu_env *env, wait_on_page_writeback(vmpage); if (!PageDirty(vmpage)) { struct cl_page_list *plist = &vio->u.fault.ft_queue; - struct vvp_page *vpg = cl_object_page_slice(obj, page); int to = PAGE_SIZE; /* vvp_page_assume() calls wait_on_page_writeback(). */ @@ -1448,7 +1445,7 @@ static int vvp_io_fault_start(const struct lu_env *env, cl_page_list_add(plist, page, true); /* size fixup */ - if (last_index == vvp_index(vpg)) + if (last_index == cl_page_index(page)) to = ((size - 1) & ~PAGE_MASK) + 1; /* Do not set Dirty bit here so that in case IO is diff --git a/fs/lustre/llite/vvp_object.c b/fs/lustre/llite/vvp_object.c index 64ecdb9..0ef055f 100644 --- a/fs/lustre/llite/vvp_object.c +++ b/fs/lustre/llite/vvp_object.c @@ -228,7 +228,7 @@ static int __vvp_object_init(const struct lu_env *env, const struct cl_object_conf *conf) { vob->vob_inode = conf->coc_inode; - cl_object_page_init(&vob->vob_cl, sizeof(struct vvp_page)); + cl_object_page_init(&vob->vob_cl, sizeof(struct cl_page_slice)); return 0; } diff --git a/fs/lustre/llite/vvp_page.c b/fs/lustre/llite/vvp_page.c index 7c80908..f359596 100644 --- a/fs/lustre/llite/vvp_page.c +++ b/fs/lustre/llite/vvp_page.c @@ -125,13 +125,12 @@ static void vvp_page_completion_write(const struct lu_env *env, const struct cl_page_slice *slice, int ioret) { - struct vvp_page *vpg = cl2vvp_page(slice); - struct cl_page *pg = slice->cpl_page; - struct page *vmpage = vpg->vpg_page; + struct cl_page *cp = slice->cpl_page; + struct page *vmpage = cp->cp_vmpage; - CL_PAGE_HEADER(D_PAGE, env, pg, "completing WRITE with %d\n", ioret); + CL_PAGE_HEADER(D_PAGE, env, cp, "completing WRITE with %d\n", ioret); - if (pg->cp_sync_io) { + if (cp->cp_sync_io) { LASSERT(PageLocked(vmpage)); LASSERT(!PageWriteback(vmpage)); } else { @@ -140,7 +139,7 @@ static void vvp_page_completion_write(const struct lu_env *env, * Only mark the page error only when it's an async write * because applications won't wait for IO to finish. */ - vvp_vmpage_error(vvp_object_inode(pg->cp_obj), vmpage, ioret); + vvp_vmpage_error(vvp_object_inode(cp->cp_obj), vmpage, ioret); end_page_writeback(vmpage); } @@ -164,18 +163,16 @@ static void vvp_page_completion_write(const struct lu_env *env, int vvp_page_init(const struct lu_env *env, struct cl_object *obj, struct cl_page *page, pgoff_t index) { - struct vvp_page *vpg = cl_object_page_slice(obj, page); + struct cl_page_slice *cpl = cl_object_page_slice(obj, page); struct page *vmpage = page->cp_vmpage; CLOBINVRNT(env, obj, vvp_object_invariant(obj)); - vpg->vpg_page = vmpage; - if (page->cp_type == CPT_TRANSIENT) { /* DIO pages are referenced by userspace, we don't need to take * a reference on them. (contrast with get_page() call above) */ - cl_page_slice_add(page, &vpg->vpg_cl, obj, + cl_page_slice_add(page, cpl, obj, &vvp_transient_page_ops); } else { get_page(vmpage); @@ -183,8 +180,7 @@ int vvp_page_init(const struct lu_env *env, struct cl_object *obj, refcount_inc(&page->cp_ref); SetPagePrivate(vmpage); vmpage->private = (unsigned long)page; - cl_page_slice_add(page, &vpg->vpg_cl, obj, - &vvp_page_ops); + cl_page_slice_add(page, cpl, obj, &vvp_page_ops); } return 0; } From patchwork Sun Sep 18 05:22:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979322 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5AF8FC6FA86 for ; Sun, 18 Sep 2022 05:22:57 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbmd0VnQz1y78; Sat, 17 Sep 2022 22:22:57 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVbm54cV1z1yD2 for ; Sat, 17 Sep 2022 22:22:29 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 015918F0F; Sun, 18 Sep 2022 01:22:17 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id F34AAE8BA5; Sun, 18 Sep 2022 01:22:16 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:22:05 -0400 Message-Id: <1663478534-19917-16-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 15/24] lustre: clio: remove unused convenience functions X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: "John L. Hammond" Remove the unused convenience functions cl_page_top(), cl_page_at(), cl_page_at_trusted(), and cl2vm_page(). WC-bug-id: https://jira.whamcloud.com/browse/LU-10994 Lustre-commit: 27e19a5420ae3ec22 ("LU-10994 clio: remove unused convenience functions") Signed-off-by: John L. Hammond Reviewed-on: https://review.whamcloud.com/47401 Reviewed-by: Patrick Farrell Reviewed-by: Bobi Jam Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/cl_object.h | 3 --- fs/lustre/llite/vvp_internal.h | 5 ----- fs/lustre/obdclass/cl_page.c | 28 ---------------------------- 3 files changed, 36 deletions(-) diff --git a/fs/lustre/include/cl_object.h b/fs/lustre/include/cl_object.h index c816b1d..c5adbda 100644 --- a/fs/lustre/include/cl_object.h +++ b/fs/lustre/include/cl_object.h @@ -2217,9 +2217,6 @@ void cl_page_header_print(const struct lu_env *env, void *cookie, lu_printer_t printer, const struct cl_page *pg); struct cl_page *cl_vmpage_page(struct page *vmpage, struct cl_object *obj); -const struct cl_page_slice *cl_page_at(const struct cl_page *page, - const struct lu_device_type *dtype); - /** * \name ownership * diff --git a/fs/lustre/llite/vvp_internal.h b/fs/lustre/llite/vvp_internal.h index 17feae6..c73923b 100644 --- a/fs/lustre/llite/vvp_internal.h +++ b/fs/lustre/llite/vvp_internal.h @@ -250,11 +250,6 @@ static inline struct inode *vvp_object_inode(const struct cl_object *obj) int vvp_object_invariant(const struct cl_object *obj); struct vvp_object *cl_inode2vvp(struct inode *inode); -static inline struct page *cl2vm_page(const struct cl_page_slice *slice) -{ - return slice->cpl_page->cp_vmpage; -} - # define CLOBINVRNT(env, clob, expr) \ ((void)sizeof(env), (void)sizeof(clob), (void)sizeof(!!(expr))) diff --git a/fs/lustre/obdclass/cl_page.c b/fs/lustre/obdclass/cl_page.c index 2e3374c..7829b90d 100644 --- a/fs/lustre/obdclass/cl_page.c +++ b/fs/lustre/obdclass/cl_page.c @@ -95,27 +95,6 @@ static void cl_page_get_trust(struct cl_page *page) slice = cl_page_slice_get(cl_page, i); i >= 0; \ slice = cl_page_slice_get(cl_page, --i)) -/** - * Returns a slice within a cl_page, corresponding to the given layer in the - * device stack. - * - * \see cl_lock_at() - */ -static const struct cl_page_slice * -cl_page_at_trusted(const struct cl_page *cl_page, - const struct lu_device_type *dtype) -{ - const struct cl_page_slice *slice; - int i; - - cl_page_slice_for_each(cl_page, slice, i) { - if (slice->cpl_obj->co_lu.lo_dev->ld_type == dtype) - return slice; - } - - return NULL; -} - static void __cl_page_free(struct cl_page *cl_page, unsigned short bufsize) { int index = cl_page->cp_kmem_index; @@ -477,13 +456,6 @@ struct cl_page *cl_vmpage_page(struct page *vmpage, struct cl_object *obj) } EXPORT_SYMBOL(cl_vmpage_page); -const struct cl_page_slice *cl_page_at(const struct cl_page *page, - const struct lu_device_type *dtype) -{ - return cl_page_at_trusted(page, dtype); -} -EXPORT_SYMBOL(cl_page_at); - static void cl_page_owner_clear(struct cl_page *page) { if (page->cp_owner) { From patchwork Sun Sep 18 05:22:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979325 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A1F19C32771 for ; Sun, 18 Sep 2022 05:23:03 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbml2qVbz1yFk; Sat, 17 Sep 2022 22:23:03 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVbm62nSDz1yD9 for ; Sat, 17 Sep 2022 22:22:30 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 038A88F10; Sun, 18 Sep 2022 01:22:17 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 0247AE8BA9; Sun, 18 Sep 2022 01:22:17 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:22:06 -0400 Message-Id: <1663478534-19917-17-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 16/24] lustre: clio: remove cpl_obj X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: "John L. Hammond" Remove cpl_obj from struct cl_page_slice. This member is only used in the osc layer and struct osc_page already contains a pointer to the osc_object. WC-bug-id: https://jira.whamcloud.com/browse/LU-10994 Lustre-commit: 10da530a2411f28988ab5 ("LU-10994 clio: remove cpl_obj") Signed-off-by: John L. Hammond Reviewed-on: https://review.whamcloud.com/47402 Reviewed-by: Patrick Farrell Reviewed-by: Bobi Jam Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/cl_object.h | 5 ----- fs/lustre/include/lustre_osc.h | 5 +++++ fs/lustre/obdclass/cl_page.c | 1 - fs/lustre/osc/osc_cache.c | 7 +++---- fs/lustre/osc/osc_page.c | 8 ++++---- 5 files changed, 12 insertions(+), 14 deletions(-) diff --git a/fs/lustre/include/cl_object.h b/fs/lustre/include/cl_object.h index c5adbda..41ce0b0 100644 --- a/fs/lustre/include/cl_object.h +++ b/fs/lustre/include/cl_object.h @@ -795,11 +795,6 @@ struct cl_page { */ struct cl_page_slice { struct cl_page *cpl_page; - /** - * Object slice corresponding to this page slice. Immutable after - * creation. - */ - struct cl_object *cpl_obj; const struct cl_page_operations *cpl_ops; }; diff --git a/fs/lustre/include/lustre_osc.h b/fs/lustre/include/lustre_osc.h index 7551390..16d6a8e 100644 --- a/fs/lustre/include/lustre_osc.h +++ b/fs/lustre/include/lustre_osc.h @@ -858,6 +858,11 @@ static inline pgoff_t osc_index(struct osc_page *opg) return opg->ops_oap.oap_obj_off >> PAGE_SHIFT; } +static inline struct osc_object *osc_page_object(struct osc_page *ops) +{ + return ops->ops_oap.oap_obj; +} + static inline struct cl_page *oap2cl_page(struct osc_async_page *oap) { return oap2osc(oap)->ops_cl.cpl_page; diff --git a/fs/lustre/obdclass/cl_page.c b/fs/lustre/obdclass/cl_page.c index 7829b90d..7011235 100644 --- a/fs/lustre/obdclass/cl_page.c +++ b/fs/lustre/obdclass/cl_page.c @@ -1078,7 +1078,6 @@ void cl_page_slice_add(struct cl_page *cl_page, struct cl_page_slice *slice, LASSERT(cl_page->cp_layer_count < CP_MAX_LAYER); LASSERT(offset < (1 << sizeof(cl_page->cp_layer_offset[0]) * 8)); cl_page->cp_layer_offset[cl_page->cp_layer_count++] = offset; - slice->cpl_obj = obj; slice->cpl_ops = ops; slice->cpl_page = cl_page; } diff --git a/fs/lustre/osc/osc_cache.c b/fs/lustre/osc/osc_cache.c index 072cfac..e7b8372 100644 --- a/fs/lustre/osc/osc_cache.c +++ b/fs/lustre/osc/osc_cache.c @@ -1328,14 +1328,13 @@ static int osc_refresh_count(const struct lu_env *env, { struct osc_page *opg = oap2osc_page(oap); pgoff_t index = osc_index(oap2osc(oap)); - struct cl_object *obj; + struct cl_object *obj = osc2cl(osc_page_object(opg)); struct cl_attr *attr = &osc_env_info(env)->oti_attr; int result; loff_t kms; /* readpage queues with _COUNT_STABLE, shouldn't get here. */ LASSERT(!(cmd & OBD_BRW_READ)); - obj = opg->ops_cl.cpl_obj; cl_object_attr_lock(obj); result = cl_object_attr_get(env, obj, attr); @@ -1378,7 +1377,7 @@ static int osc_completion(const struct lu_env *env, struct osc_async_page *oap, /* statistic */ if (rc == 0 && srvlock) { - struct lu_device *ld = opg->ops_cl.cpl_obj->co_lu.lo_dev; + struct lu_device *ld = osc_page_object(opg)->oo_cl.co_lu.lo_dev; struct osc_stats *stats = &lu2osc_dev(ld)->od_stats; size_t bytes = oap->oap_count; @@ -2588,7 +2587,7 @@ int osc_flush_async_page(const struct lu_env *env, struct cl_io *io, struct osc_page *ops) { struct osc_extent *ext = NULL; - struct osc_object *obj = cl2osc(ops->ops_cl.cpl_obj); + struct osc_object *obj = osc_page_object(ops); struct cl_page *cp = ops->ops_cl.cpl_page; pgoff_t index = osc_index(ops); struct osc_async_page *oap = &ops->ops_oap; diff --git a/fs/lustre/osc/osc_page.c b/fs/lustre/osc/osc_page.c index fd87698..7470dee 100644 --- a/fs/lustre/osc/osc_page.c +++ b/fs/lustre/osc/osc_page.c @@ -86,7 +86,7 @@ static void osc_page_transfer_put(const struct lu_env *env, static void osc_page_transfer_add(const struct lu_env *env, struct osc_page *opg, enum cl_req_type crt) { - struct osc_object *obj = cl2osc(opg->ops_cl.cpl_obj); + struct osc_object *obj = osc_page_object(opg); osc_lru_use(osc_cli(obj), opg); } @@ -129,7 +129,7 @@ static int osc_page_print(const struct lu_env *env, { struct osc_page *opg = cl2osc_page(slice); struct osc_async_page *oap = &opg->ops_oap; - struct osc_object *obj = cl2osc(slice->cpl_obj); + struct osc_object *obj = osc_page_object(opg); struct client_obd *cli = &osc_export(obj)->exp_obd->u.cli; return (*printer)(env, cookie, LUSTRE_OSC_NAME @@ -171,7 +171,7 @@ static void osc_page_delete(const struct lu_env *env, const struct cl_page_slice *slice) { struct osc_page *opg = cl2osc_page(slice); - struct osc_object *obj = cl2osc(opg->ops_cl.cpl_obj); + struct osc_object *obj = osc_page_object(opg); int rc; CDEBUG(D_TRACE, "%p\n", opg); @@ -235,7 +235,7 @@ static void osc_page_touch(const struct lu_env *env, const struct cl_page_slice *slice, size_t to) { struct osc_page *opg = cl2osc_page(slice); - struct cl_object *obj = opg->ops_cl.cpl_obj; + struct cl_object *obj = osc2cl(osc_page_object(opg)); osc_page_touch_at(env, obj, osc_index(opg), to); } From patchwork Sun Sep 18 05:22:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979329 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 134CFC32771 for ; Sun, 18 Sep 2022 05:23:12 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbmv5N40z1yFD; Sat, 17 Sep 2022 22:23:11 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVbm718x6z1yDP for ; Sat, 17 Sep 2022 22:22:31 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 06EE78F11; Sun, 18 Sep 2022 01:22:17 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 0554C1C6DE; Sun, 18 Sep 2022 01:22:17 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:22:07 -0400 Message-Id: <1663478534-19917-18-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 17/24] lustre: osc: remove oap_cli X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: "John L. Hammond" Remove the redundant oap_cli member from struct osc_async_page. ...:(cl_page.c:216:__cl_page_alloc()) slab-alloced 'cl_page': 256 at 000000009ab84b37. WC-bug-id: https://jira.whamcloud.com/browse/LU-10994 Lustre-commit: 9b4fb6e65414fa75a ("LU-10994 osc: remove oap_cli") Signed-off-by: John L. Hammond Reviewed-on: https://review.whamcloud.com/47403 Reviewed-by: Patrick Farrell Reviewed-by: Bobi Jam Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_osc.h | 1 - fs/lustre/osc/osc_cache.c | 6 ++---- fs/lustre/osc/osc_page.c | 2 +- 3 files changed, 3 insertions(+), 6 deletions(-) diff --git a/fs/lustre/include/lustre_osc.h b/fs/lustre/include/lustre_osc.h index 16d6a8e..d630169 100644 --- a/fs/lustre/include/lustre_osc.h +++ b/fs/lustre/include/lustre_osc.h @@ -88,7 +88,6 @@ struct osc_async_page { struct brw_page oap_brw_page; struct ptlrpc_request *oap_request; - struct client_obd *oap_cli; struct osc_object *oap_obj; spinlock_t oap_lock; diff --git a/fs/lustre/osc/osc_cache.c b/fs/lustre/osc/osc_cache.c index e7b8372..29d13f5 100644 --- a/fs/lustre/osc/osc_cache.c +++ b/fs/lustre/osc/osc_cache.c @@ -2314,7 +2314,6 @@ int __osc_io_unplug(const struct lu_env *env, struct client_obd *cli, int osc_prep_async_page(struct osc_object *osc, struct osc_page *ops, struct cl_page *page, loff_t offset) { - struct obd_export *exp = osc_export(osc); struct osc_async_page *oap = &ops->ops_oap; struct page *vmpage = page->cp_vmpage; @@ -2322,7 +2321,6 @@ int osc_prep_async_page(struct osc_object *osc, struct osc_page *ops, return -EIO; oap->oap_magic = OAP_MAGIC; - oap->oap_cli = &exp->exp_obd->u.cli; oap->oap_obj = osc; oap->oap_page = vmpage; @@ -2353,9 +2351,9 @@ int osc_queue_async_io(const struct lu_env *env, struct cl_io *io, struct osc_io *oio = osc_env_io(env); struct osc_extent *ext = NULL; struct osc_async_page *oap = &ops->ops_oap; - struct client_obd *cli = oap->oap_cli; struct osc_object *osc = oap->oap_obj; - struct pagevec *pvec = &osc_env_info(env)->oti_pagevec; + struct client_obd *cli = osc_cli(osc); + struct pagevec *pvec = &osc_env_info(env)->oti_pagevec; pgoff_t index; unsigned int grants = 0, tmp; int brw_flags = OBD_BRW_ASYNC; diff --git a/fs/lustre/osc/osc_page.c b/fs/lustre/osc/osc_page.c index 7470dee..5300ac9 100644 --- a/fs/lustre/osc/osc_page.c +++ b/fs/lustre/osc/osc_page.c @@ -142,7 +142,7 @@ static int osc_page_print(const struct lu_env *env, /* 2 */ oap->oap_obj_off, oap->oap_page_off, oap->oap_count, oap->oap_async_flags, oap->oap_brw_flags, - oap->oap_request, oap->oap_cli, obj, + oap->oap_request, cli, obj, /* 3 */ opg->ops_transfer_pinned, osc_submit_duration(opg), opg->ops_srvlock, From patchwork Sun Sep 18 05:22:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979324 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 99FB0C6FA86 for ; Sun, 18 Sep 2022 05:23:02 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbmk2grjz1y7W; Sat, 17 Sep 2022 22:23:02 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVbm762VQz1yDb for ; Sat, 17 Sep 2022 22:22:31 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 0A7438F12; Sun, 18 Sep 2022 01:22:17 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 085C81C6DF; Sun, 18 Sep 2022 01:22:17 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:22:08 -0400 Message-Id: <1663478534-19917-19-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 18/24] lustre: osc: Remove submit time X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Patrick Farrell The osc page submit time is an unused bit of debugging information, but it's allocated for every page. Let's just remove it to save memory. WC-bug-id: https://jira.whamcloud.com/browse/LU-15619 Lustre-commit: 28a7bbdb81cfd8359 ("LU-15619 osc: Remove submit time") Signed-off-by: Patrick Farrell Reviewed-on: https://review.whamcloud.com/46712 Reviewed-by: Andreas Dilger Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_osc.h | 6 +----- fs/lustre/osc/osc_cache.c | 9 +-------- fs/lustre/osc/osc_io.c | 3 +-- fs/lustre/osc/osc_page.c | 15 +++------------ 4 files changed, 6 insertions(+), 27 deletions(-) diff --git a/fs/lustre/include/lustre_osc.h b/fs/lustre/include/lustre_osc.h index d630169..89f02c5 100644 --- a/fs/lustre/include/lustre_osc.h +++ b/fs/lustre/include/lustre_osc.h @@ -542,10 +542,6 @@ struct osc_page { * lru page list. See osc_lru_{del|use}() in osc_page.c for usage. */ struct list_head ops_lru; - /* - * Submit time - the time when the page is starting RPC. For debugging. - */ - ktime_t ops_submit_time; }; struct osc_brw_async_args { @@ -582,7 +578,7 @@ void osc_index2policy(union ldlm_policy_data *policy, pgoff_t start, pgoff_t end); void osc_lru_add_batch(struct client_obd *cli, struct list_head *list); void osc_page_submit(const struct lu_env *env, struct osc_page *opg, - enum cl_req_type crt, int brw_flags, ktime_t submit_time); + enum cl_req_type crt, int brw_flags); int lru_queue_work(const struct lu_env *env, void *data); long osc_lru_shrink(const struct lu_env *env, struct client_obd *cli, long target, bool force); diff --git a/fs/lustre/osc/osc_cache.c b/fs/lustre/osc/osc_cache.c index 29d13f5..b6f0cdb 100644 --- a/fs/lustre/osc/osc_cache.c +++ b/fs/lustre/osc/osc_cache.c @@ -1310,17 +1310,11 @@ static inline int osc_is_ready(struct osc_object *osc) static int osc_make_ready(const struct lu_env *env, struct osc_async_page *oap, int cmd) { - struct osc_page *opg = oap2osc_page(oap); struct cl_page *page = oap2cl_page(oap); - int result; LASSERT(cmd == OBD_BRW_WRITE); /* no cached reads */ - result = cl_page_make_ready(env, page, CRT_WRITE); - if (result == 0) - opg->ops_submit_time = ktime_get(); - - return result; + return cl_page_make_ready(env, page, CRT_WRITE); } static int osc_refresh_count(const struct lu_env *env, @@ -1372,7 +1366,6 @@ static int osc_completion(const struct lu_env *env, struct osc_async_page *oap, /* Clear opg->ops_transfer_pinned before VM lock is released. */ opg->ops_transfer_pinned = 0; - opg->ops_submit_time = ktime_set(0, 0); srvlock = oap->oap_brw_flags & OBD_BRW_SRVLOCK; /* statistic */ diff --git a/fs/lustre/osc/osc_io.c b/fs/lustre/osc/osc_io.c index 1361d7f..655c7c6 100644 --- a/fs/lustre/osc/osc_io.c +++ b/fs/lustre/osc/osc_io.c @@ -132,7 +132,6 @@ int osc_io_submit(const struct lu_env *env, const struct cl_io_slice *ios, unsigned int max_pages; unsigned int ppc_bits; /* pages per chunk bits */ unsigned int ppc; - ktime_t submit_time = ktime_get(); bool sync_queue = false; LASSERT(qin->pl_nr > 0); @@ -200,7 +199,7 @@ int osc_io_submit(const struct lu_env *env, const struct cl_io_slice *ios, spin_unlock(&oap->oap_lock); } - osc_page_submit(env, opg, crt, brw_flags, submit_time); + osc_page_submit(env, opg, crt, brw_flags); list_add_tail(&oap->oap_pending_item, &list); if (page->cp_sync_io) diff --git a/fs/lustre/osc/osc_page.c b/fs/lustre/osc/osc_page.c index 5300ac9..12ba108 100644 --- a/fs/lustre/osc/osc_page.c +++ b/fs/lustre/osc/osc_page.c @@ -115,14 +115,6 @@ void osc_index2policy(union ldlm_policy_data *policy, policy->l_extent.end = cl_offset(obj, end + 1) - 1; } -static inline s64 osc_submit_duration(struct osc_page *opg) -{ - if (ktime_to_ns(opg->ops_submit_time) == 0) - return 0; - - return ktime_ms_delta(ktime_get(), opg->ops_submit_time); -} - static int osc_page_print(const struct lu_env *env, const struct cl_page_slice *slice, void *cookie, lu_printer_t printer) @@ -133,7 +125,7 @@ static int osc_page_print(const struct lu_env *env, struct client_obd *cli = &osc_export(obj)->exp_obd->u.cli; return (*printer)(env, cookie, LUSTRE_OSC_NAME - "-page@%p %lu: 1< %#x %d %c %c > 2< %llu %u %u %#x %#x | %p %p %p > 3< %d %lld %d > 4< %d %d %d %lu %c | %c %c %c %c > 5< %c %c %c %c | %d %c | %d %c %c>\n", + "-page@%p %lu: 1< %#x %d %c %c > 2< %llu %u %u %#x %#x | %p %p %p > 3< %d %d > 4< %d %d %d %lu %c | %c %c %c %c > 5< %c %c %c %c | %d %c | %d %c %c>\n", opg, osc_index(opg), /* 1 */ oap->oap_magic, oap->oap_cmd, @@ -145,7 +137,7 @@ static int osc_page_print(const struct lu_env *env, oap->oap_request, cli, obj, /* 3 */ opg->ops_transfer_pinned, - osc_submit_duration(opg), opg->ops_srvlock, + opg->ops_srvlock, /* 4 */ cli->cl_r_in_flight, cli->cl_w_in_flight, cli->cl_max_rpcs_in_flight, @@ -296,7 +288,7 @@ int osc_page_init(const struct lu_env *env, struct cl_object *obj, * transfer (i.e., transferred synchronously). */ void osc_page_submit(const struct lu_env *env, struct osc_page *opg, - enum cl_req_type crt, int brw_flags, ktime_t submit_time) + enum cl_req_type crt, int brw_flags) { struct osc_io *oio = osc_env_io(env); struct osc_async_page *oap = &opg->ops_oap; @@ -317,7 +309,6 @@ void osc_page_submit(const struct lu_env *env, struct osc_page *opg, oap->oap_cmd |= OBD_BRW_SYS_RESOURCE; } - opg->ops_submit_time = submit_time; osc_page_transfer_get(opg, "transfer\0imm"); osc_page_transfer_add(env, opg, crt); } From patchwork Sun Sep 18 05:22:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979323 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3CEA2C6FA8B for ; Sun, 18 Sep 2022 05:23:00 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbmg6sKFz1yD6; Sat, 17 Sep 2022 22:22:59 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVbm84CG3z1yC1 for ; Sat, 17 Sep 2022 22:22:32 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 0D4FE8F13; Sun, 18 Sep 2022 01:22:17 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 0B6A91C6E1; Sun, 18 Sep 2022 01:22:17 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:22:09 -0400 Message-Id: <1663478534-19917-20-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 19/24] lnet: selftest: revert "LU-16011 lnet: use preallocate bulk for server" X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Andreas Dilger This reverts commit 7c8f661bae1348f8fb9a1cd2ae5eae824f275977 due to OOM on aarch64 clients. WC-bug-id: https://jira.whamcloud.com/browse/LU-16140 Lustre-commit: 1b09fb2296993c52e ("LU-16140 lnet: revert "LU-16011 lnet: use preallocate bulk for server") Signed-off-by: Andreas Dilger Reviewed-on: https://review.whamcloud.com/48457 Signed-off-by: James Simmons --- net/lnet/selftest/brw_test.c | 67 +++++++++++++------------------------------ net/lnet/selftest/framework.c | 18 +++++++----- net/lnet/selftest/rpc.c | 51 +++++++++++--------------------- net/lnet/selftest/selftest.h | 15 ++++------ 4 files changed, 52 insertions(+), 99 deletions(-) diff --git a/net/lnet/selftest/brw_test.c b/net/lnet/selftest/brw_test.c index a00b731..87ad765 100644 --- a/net/lnet/selftest/brw_test.c +++ b/net/lnet/selftest/brw_test.c @@ -124,12 +124,11 @@ list_for_each_entry(tsu, &tsi->tsi_units, tsu_list) { bulk = srpc_alloc_bulk(lnet_cpt_of_nid(tsu->tsu_dest.nid, NULL), - npg); + off, npg, len, opc == LST_BRW_READ); if (!bulk) { brw_client_fini(tsi); return -ENOMEM; } - srpc_init_bulk(bulk, off, npg, len, opc == LST_BRW_READ); tsu->tsu_private = bulk; } @@ -390,6 +389,8 @@ static int brw_inject_one_error(void) CDEBUG(D_NET, "Transferred %d pages bulk data %s %s\n", blk->bk_niov, blk->bk_sink ? "from" : "to", libcfs_id2str(rpc->srpc_peer)); + + sfw_free_pages(rpc); } static int @@ -437,6 +438,7 @@ static int brw_inject_one_error(void) struct srpc_brw_reply *reply = &replymsg->msg_body.brw_reply; struct srpc_brw_reqst *reqst = &reqstmsg->msg_body.brw_reqst; int npg; + int rc; LASSERT(sv->sv_id == SRPC_SERVICE_BRW); @@ -487,8 +489,11 @@ static int brw_inject_one_error(void) return 0; } - srpc_init_bulk(rpc->srpc_bulk, 0, npg, reqst->brw_len, - reqst->brw_rw == LST_BRW_WRITE); + rc = sfw_alloc_pages(rpc, rpc->srpc_scd->scd_cpt, npg, + reqst->brw_len, + reqst->brw_rw == LST_BRW_WRITE); + if (rc) + return rc; if (reqst->brw_rw == LST_BRW_READ) brw_fill_bulk(rpc->srpc_bulk, reqst->brw_flags, BRW_MAGIC); @@ -498,55 +503,23 @@ static int brw_inject_one_error(void) return 0; } -static int -brw_srpc_init(struct srpc_server_rpc *rpc, int cpt) -{ - /* just alloc a maximal size - actual values will be adjusted later */ - rpc->srpc_bulk = srpc_alloc_bulk(cpt, LNET_MAX_IOV); - if (!rpc->srpc_bulk) - return -ENOMEM; - - srpc_init_bulk(rpc->srpc_bulk, 0, LNET_MAX_IOV, 0, 0); - - return 0; -} +struct sfw_test_client_ops brw_test_client; -static void -brw_srpc_fini(struct srpc_server_rpc *rpc) +void brw_init_test_client(void) { - /* server RPC have just MAX_IOV size */ - srpc_init_bulk(rpc->srpc_bulk, 0, LNET_MAX_IOV, 0, 0); - - srpc_free_bulk(rpc->srpc_bulk); - rpc->srpc_bulk = NULL; -} - -struct sfw_test_client_ops brw_test_client = { - .tso_init = brw_client_init, - .tso_fini = brw_client_fini, - .tso_prep_rpc = brw_client_prep_rpc, - .tso_done_rpc = brw_client_done_rpc, + brw_test_client.tso_init = brw_client_init; + brw_test_client.tso_fini = brw_client_fini; + brw_test_client.tso_prep_rpc = brw_client_prep_rpc; + brw_test_client.tso_done_rpc = brw_client_done_rpc; }; -struct srpc_service brw_test_service = { - .sv_id = SRPC_SERVICE_BRW, - .sv_name = "brw_test", - .sv_handler = brw_server_handle, - .sv_bulk_ready = brw_bulk_ready, - - .sv_srpc_init = brw_srpc_init, - .sv_srpc_fini = brw_srpc_fini, -}; +struct srpc_service brw_test_service; void brw_init_test_service(void) { - unsigned long cache_size = totalram_pages() >> 1; - - /* brw prealloc cache should don't eat more than half memory */ - cache_size /= LNET_MAX_IOV; - + brw_test_service.sv_id = SRPC_SERVICE_BRW; + brw_test_service.sv_name = "brw_test"; + brw_test_service.sv_handler = brw_server_handle; + brw_test_service.sv_bulk_ready = brw_bulk_ready; brw_test_service.sv_wi_total = brw_srv_workitems; - - if (brw_test_service.sv_wi_total > cache_size) - brw_test_service.sv_wi_total = cache_size; } diff --git a/net/lnet/selftest/framework.c b/net/lnet/selftest/framework.c index 121bdf0..e84904e 100644 --- a/net/lnet/selftest/framework.c +++ b/net/lnet/selftest/framework.c @@ -290,10 +290,8 @@ swi_state2str(rpc->srpc_wi.swi_state), status); - if (rpc->srpc_bulk) { - srpc_free_bulk(rpc->srpc_bulk); - rpc->srpc_bulk = NULL; - } + if (rpc->srpc_bulk) + sfw_free_pages(rpc); } static void @@ -1090,6 +1088,13 @@ return -ENOENT; } +void +sfw_free_pages(struct srpc_server_rpc *rpc) +{ + srpc_free_bulk(rpc->srpc_bulk); + rpc->srpc_bulk = NULL; +} + int sfw_alloc_pages(struct srpc_server_rpc *rpc, int cpt, int npages, int len, int sink) @@ -1097,12 +1102,10 @@ LASSERT(!rpc->srpc_bulk); LASSERT(npages > 0 && npages <= LNET_MAX_IOV); - rpc->srpc_bulk = srpc_alloc_bulk(cpt, npages); + rpc->srpc_bulk = srpc_alloc_bulk(cpt, 0, npages, len, sink); if (!rpc->srpc_bulk) return -ENOMEM; - srpc_init_bulk(rpc->srpc_bulk, 0, npages, len, sink); - return 0; } @@ -1626,6 +1629,7 @@ struct srpc_client_rpc * INIT_LIST_HEAD(&sfw_data.fw_zombie_rpcs); INIT_LIST_HEAD(&sfw_data.fw_zombie_sessions); + brw_init_test_client(); brw_init_test_service(); rc = sfw_register_test(&brw_test_service, &brw_test_client); LASSERT(!rc); diff --git a/net/lnet/selftest/rpc.c b/net/lnet/selftest/rpc.c index b9d8211..c376019 100644 --- a/net/lnet/selftest/rpc.c +++ b/net/lnet/selftest/rpc.c @@ -109,12 +109,14 @@ void srpc_get_counters(struct srpc_counters *cnt) } static int -srpc_init_bulk_page(struct srpc_bulk *bk, int i, int off, int nob) +srpc_add_bulk_page(struct srpc_bulk *bk, struct page *pg, int i, int off, + int nob) { LASSERT(off < PAGE_SIZE); LASSERT(nob > 0 && nob <= PAGE_SIZE); bk->bk_iovs[i].bv_offset = off; + bk->bk_iovs[i].bv_page = pg; bk->bk_iovs[i].bv_len = nob; return nob; } @@ -138,7 +140,9 @@ void srpc_get_counters(struct srpc_counters *cnt) kfree(bk); } -struct srpc_bulk *srpc_alloc_bulk(int cpt, unsigned int bulk_npg) +struct srpc_bulk * +srpc_alloc_bulk(int cpt, unsigned int bulk_off, unsigned int bulk_npg, + unsigned int bulk_len, int sink) { struct srpc_bulk *bk; int i; @@ -153,10 +157,13 @@ struct srpc_bulk *srpc_alloc_bulk(int cpt, unsigned int bulk_npg) } memset(bk, 0, offsetof(struct srpc_bulk, bk_iovs[bulk_npg])); + bk->bk_sink = sink; + bk->bk_len = bulk_len; bk->bk_niov = bulk_npg; for (i = 0; i < bulk_npg; i++) { struct page *pg; + int nob; pg = alloc_pages_node(cfs_cpt_spread_node(lnet_cpt_table(), cpt), @@ -166,37 +173,15 @@ struct srpc_bulk *srpc_alloc_bulk(int cpt, unsigned int bulk_npg) srpc_free_bulk(bk); return NULL; } - bk->bk_iovs[i].bv_page = pg; - } - - return bk; -} - -void -srpc_init_bulk(struct srpc_bulk *bk, unsigned int bulk_off, - unsigned int bulk_npg, unsigned int bulk_len, int sink) -{ - int i; - - LASSERT(bk); - LASSERT(bulk_npg > 0 && bulk_npg <= LNET_MAX_IOV); - - bk->bk_sink = sink; - bk->bk_len = bulk_len; - bk->bk_niov = bulk_npg; - - for (i = 0; i < bulk_npg && bulk_len > 0; i++) { - int nob; - - LASSERT(bk->bk_iovs[i].bv_page); nob = min_t(unsigned int, bulk_off + bulk_len, PAGE_SIZE) - bulk_off; - - srpc_init_bulk_page(bk, i, bulk_off, nob); + srpc_add_bulk_page(bk, pg, i, bulk_off, nob); bulk_len -= nob; bulk_off = 0; } + + return bk; } static inline u64 @@ -210,6 +195,7 @@ struct srpc_bulk *srpc_alloc_bulk(int cpt, unsigned int bulk_npg) struct srpc_service_cd *scd, struct srpc_buffer *buffer) { + memset(rpc, 0, sizeof(*rpc)); swi_init_workitem(&rpc->srpc_wi, srpc_handle_rpc, srpc_serv_is_framework(scd->scd_svc) ? lst_serial_wq : lst_test_wq[scd->scd_cpt]); @@ -221,9 +207,6 @@ struct srpc_bulk *srpc_alloc_bulk(int cpt, unsigned int bulk_npg) rpc->srpc_peer = buffer->buf_peer; rpc->srpc_self = buffer->buf_self; LNetInvalidateMDHandle(&rpc->srpc_replymdh); - - rpc->srpc_aborted = 0; - rpc->srpc_status = 0; } static void @@ -261,8 +244,6 @@ struct srpc_bulk *srpc_alloc_bulk(int cpt, unsigned int bulk_npg) struct srpc_server_rpc, srpc_list)) != NULL) { list_del(&rpc->srpc_list); - if (svc->sv_srpc_fini) - svc->sv_srpc_fini(rpc); kfree(rpc); } } @@ -333,8 +314,7 @@ struct srpc_bulk *srpc_alloc_bulk(int cpt, unsigned int bulk_npg) for (j = 0; j < nrpcs; j++) { rpc = kzalloc_cpt(sizeof(*rpc), GFP_NOFS, i); - if (!rpc || - (svc->sv_srpc_init && svc->sv_srpc_init(rpc, i))) { + if (!rpc) { srpc_service_fini(svc); return -ENOMEM; } @@ -966,7 +946,8 @@ struct srpc_bulk *srpc_alloc_bulk(int cpt, unsigned int bulk_npg) atomic_inc(&RPC_STAT32(SRPC_RPC_DROP)); if (rpc->srpc_done) - (*rpc->srpc_done)(rpc); + (*rpc->srpc_done) (rpc); + LASSERT(!rpc->srpc_bulk); spin_lock(&scd->scd_lock); diff --git a/net/lnet/selftest/selftest.h b/net/lnet/selftest/selftest.h index 8ae258d..223a432 100644 --- a/net/lnet/selftest/selftest.h +++ b/net/lnet/selftest/selftest.h @@ -316,12 +316,6 @@ struct srpc_service { */ int (*sv_handler)(struct srpc_server_rpc *); int (*sv_bulk_ready)(struct srpc_server_rpc *, int); - - /** Service side srpc constructor/destructor. - * used for the bulk preallocation as usual. - */ - int (*sv_srpc_init)(struct srpc_server_rpc *rpc, int cpt); - void (*sv_srpc_fini)(struct srpc_server_rpc *rpc); }; struct sfw_session { @@ -430,6 +424,7 @@ int sfw_create_test_rpc(struct sfw_test_unit *tsu, void sfw_post_rpc(struct srpc_client_rpc *rpc); void sfw_client_rpc_done(struct srpc_client_rpc *rpc); void sfw_unpack_message(struct srpc_msg *msg); +void sfw_free_pages(struct srpc_server_rpc *rpc); void sfw_add_bulk_page(struct srpc_bulk *bk, struct page *pg, int i); int sfw_alloc_pages(struct srpc_server_rpc *rpc, int cpt, int npages, int len, int sink); @@ -444,10 +439,9 @@ struct srpc_client_rpc * void srpc_post_rpc(struct srpc_client_rpc *rpc); void srpc_abort_rpc(struct srpc_client_rpc *rpc, int why); void srpc_free_bulk(struct srpc_bulk *bk); -struct srpc_bulk *srpc_alloc_bulk(int cpt, unsigned int bulk_npg); -void srpc_init_bulk(struct srpc_bulk *bk, unsigned int off, - unsigned int bulk_npg, unsigned int bulk_len, int sink); - +struct srpc_bulk *srpc_alloc_bulk(int cpt, unsigned int off, + unsigned int bulk_npg, unsigned int bulk_len, + int sink); void srpc_send_rpc(struct swi_workitem *wi); int srpc_send_reply(struct srpc_server_rpc *rpc); int srpc_add_service(struct srpc_service *sv); @@ -611,6 +605,7 @@ void srpc_init_bulk(struct srpc_bulk *bk, unsigned int off, } extern struct sfw_test_client_ops brw_test_client; +void brw_init_test_client(void); extern struct srpc_service brw_test_service; void brw_init_test_service(void); From patchwork Sun Sep 18 05:22:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979326 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1732BC32771 for ; Sun, 18 Sep 2022 05:23:06 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbmn5gG4z215W; Sat, 17 Sep 2022 22:23:05 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVbm92QZ8z1yC1 for ; Sat, 17 Sep 2022 22:22:33 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 1016B8F14; Sun, 18 Sep 2022 01:22:17 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 0E9401CA2D; Sun, 18 Sep 2022 01:22:17 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:22:10 -0400 Message-Id: <1663478534-19917-21-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 20/24] lustre: flr: allow layout version update from client/MDS X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Bobi Jam Client write request always carries its layout version so that OFD can reject the request if the carried layout version is a stale one. This patch makes OFD allow layout version change request from client as well as MDS. And during resync write, all OST objects will get layout version updated. WC-bug-id: https://jira.whamcloud.com/browse/LU-14642 Lustre-commit: fa6574150b6f745a66 ("LU-14642 flr: allow layout version update from client/MDS") Signed-off-by: Bobi Jam Reviewed-on: https://review.whamcloud.com/45443 Reviewed-by: Andreas Dilger Reviewed-by: Alex Zhuravlev Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/obd_support.h | 2 +- fs/lustre/llite/file.c | 30 +++++++++++------------------- include/uapi/linux/lustre/lustre_user.h | 15 ++++++++++----- 3 files changed, 22 insertions(+), 25 deletions(-) diff --git a/fs/lustre/include/obd_support.h b/fs/lustre/include/obd_support.h index 0909351..c98c8a4 100644 --- a/fs/lustre/include/obd_support.h +++ b/fs/lustre/include/obd_support.h @@ -517,7 +517,7 @@ /* FLR */ #define OBD_FAIL_FLR_LV_DELAY 0x1A01 -#define OBD_FAIL_FLR_LV_INC 0x1A02 +#define OBD_FAIL_FLR_LV_INC 0x1A02 /* unused since 2.15 */ #define OBD_FAIL_FLR_RANDOM_PICK_MIRROR 0x1A03 /* LNet is allocated failure locations 0xe000 to 0xffff */ diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 5394cce..e75f482 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -3553,6 +3553,7 @@ static long ll_file_unlock_lease(struct file *file, struct ll_ioc_lease *ioc, bool lease_broken = false; fmode_t fmode = 0; enum mds_op_bias bias = 0; + int fdv; struct file *layout_file = NULL; void *data = NULL; size_t data_size = 0; @@ -3592,21 +3593,19 @@ static long ll_file_unlock_lease(struct file *file, struct ll_ioc_lease *ioc, bias = MDS_CLOSE_RESYNC_DONE; break; - case LL_LEASE_LAYOUT_MERGE: { - int fd; - + case LL_LEASE_LAYOUT_MERGE: if (ioc->lil_count != 1) { rc = -EINVAL; goto out_lease_close; } arg += sizeof(*ioc); - if (copy_from_user(&fd, (void __user *)arg, sizeof(u32))) { + if (copy_from_user(&fdv, (void __user *)arg, sizeof(u32))) { rc = -EFAULT; goto out_lease_close; } - layout_file = fget(fd); + layout_file = fget(fdv); if (!layout_file) { rc = -EBADF; goto out_lease_close; @@ -3621,9 +3620,7 @@ static long ll_file_unlock_lease(struct file *file, struct ll_ioc_lease *ioc, data = file_inode(layout_file); bias = MDS_CLOSE_LAYOUT_MERGE; break; - } case LL_LEASE_LAYOUT_SPLIT: { - int fdv; int mirror_id; if (ioc->lil_count != 2) { @@ -3700,29 +3697,24 @@ static long ll_file_unlock_lease(struct file *file, struct ll_ioc_lease *ioc, if (lease_broken) fmode = 0; - out: - switch (ioc->lil_flags) { - case LL_LEASE_RESYNC_DONE: + if (ioc->lil_flags == LL_LEASE_RESYNC_DONE && data) kfree(data); - break; - case LL_LEASE_LAYOUT_MERGE: - case LL_LEASE_LAYOUT_SPLIT: - if (layout_file) - fput(layout_file); - ll_layout_refresh(inode, &fd->fd_layout_version); - break; - case LL_LEASE_PCC_ATTACH: + if (layout_file) + fput(layout_file); + + if (ioc->lil_flags == LL_LEASE_PCC_ATTACH) { if (!rc) rc = rc2; rc = pcc_readwrite_attach_fini(file, inode, param.pa_layout_gen, lease_broken, rc, attached); - break; } + ll_layout_refresh(inode, &fd->fd_layout_version); + if (!rc) rc = ll_lease_type_from_fmode(fmode); return rc; diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index 8cfee7f..6577202 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -578,11 +578,6 @@ enum lov_comp_md_entry_flags { /* The mirror flags sent by client */ #define LCME_MIRROR_FLAGS (LCME_FL_NOSYNC) -/* the highest bit in obdo::o_layout_version is used to mark if the file is - * being resynced. - */ -#define LU_LAYOUT_RESYNC LCME_FL_NEG - /* lcme_id can be specified as certain flags, and the first * bit of lcme_id is used to indicate that the ID is representing * certain LCME_FL_* but not a real ID. Which implies we can have @@ -595,6 +590,16 @@ enum lcme_id { LCME_ID_NOT_ID = LCME_FL_NEG }; +/* layout version equals to lcme_id, except some bits have special meanings */ +enum layout_version_flags { + /* layout version reaches the high water mark to be increased to + * circularly reuse the smallest value + */ + LU_LAYOUT_HIGEN = 0x40000000, + /* the highest bit is used to mark if the file is being resynced */ + LU_LAYOUT_RESYNC = 0x80000000, +}; + struct lov_comp_md_entry_v1 { __u32 lcme_id; /* unique id of component */ __u32 lcme_flags; /* LCME_FL_XXX */ From patchwork Sun Sep 18 05:22:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979320 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8AD50C32771 for ; Sun, 18 Sep 2022 05:22:53 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbmY1zycz1y7W; Sat, 17 Sep 2022 22:22:53 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVbmB1q6Fz1yCq for ; Sat, 17 Sep 2022 22:22:34 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 142CC8F15; Sun, 18 Sep 2022 01:22:17 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 11BDCE8B93; Sun, 18 Sep 2022 01:22:17 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:22:11 -0400 Message-Id: <1663478534-19917-22-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 21/24] lustre: ptlrpc: adds configurable ping interval X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Boyko , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Alexander Boyko The patch adds ability to change ping interval and eviction mutliplier. A default values stay as before. Example lctl set_param ping_interval=10 lctl set_param evict_multiplier=5 HPE-bug-id: LUS-11054 WC-bug-id: https://jira.whamcloud.com/browse/LU-16002 Lustre-commit: 8e66f061c01e53cda ("LU-16002 ptlrpc: adds configurable ping interval") Signed-off-by: Alexander Boyko Reviewed-on: https://review.whamcloud.com/47982 Reviewed-by: Sergey Cheremencev Reviewed-by: Alexander Zarochentsev Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/obd_support.h | 6 ++++-- fs/lustre/obdclass/class_obd.c | 5 +++++ fs/lustre/obdclass/obd_config.c | 1 + fs/lustre/obdclass/obd_sysfs.c | 32 ++++++++++++++++++++++++++++++-- 4 files changed, 40 insertions(+), 4 deletions(-) diff --git a/fs/lustre/include/obd_support.h b/fs/lustre/include/obd_support.h index c98c8a4..b58c1df 100644 --- a/fs/lustre/include/obd_support.h +++ b/fs/lustre/include/obd_support.h @@ -47,6 +47,8 @@ * networking / disk / timings affected by load (use Adaptive Timeouts) */ extern unsigned int obd_timeout; /* seconds */ +extern unsigned int ping_interval; /* seconds */ +extern unsigned int ping_evict_timeout_multiplier; extern unsigned int obd_timeout_set; extern unsigned int at_min; extern unsigned int at_max; @@ -66,7 +68,7 @@ /* Should be very conservative; must catch the first reconnect after reboot */ #define OBD_RECOVERY_TIME_SOFT (obd_timeout * 3) /* Change recovery-small 26b time if you change this */ -#define PING_INTERVAL max(obd_timeout / 4, 1U) +#define PING_INTERVAL ping_interval /* a bit more than maximal journal commit time in seconds */ #define PING_INTERVAL_SHORT min(PING_INTERVAL, 7U) /* Client may skip 1 ping; we must wait at least 2.5. But for multiple @@ -75,7 +77,7 @@ * and there's no urgent need to evict a client just because it's idle, we * should be very conservative here. */ -#define PING_EVICT_TIMEOUT (PING_INTERVAL * 6) +#define PING_EVICT_TIMEOUT (PING_INTERVAL * ping_evict_timeout_multiplier) #define DISK_TIMEOUT 50 /* Beyond this we warn about disk speed */ #define CONNECTION_SWITCH_MIN 5U /* Connection switching rate limiter */ /* Max connect interval for nonresponsive servers; ~50s to avoid building up diff --git a/fs/lustre/obdclass/class_obd.c b/fs/lustre/obdclass/class_obd.c index b30d941..f455ed7 100644 --- a/fs/lustre/obdclass/class_obd.c +++ b/fs/lustre/obdclass/class_obd.c @@ -63,6 +63,11 @@ EXPORT_SYMBOL(obd_dirty_pages); unsigned int obd_timeout = OBD_TIMEOUT_DEFAULT; /* seconds */ EXPORT_SYMBOL(obd_timeout); +unsigned int ping_interval = (OBD_TIMEOUT_DEFAULT > 4) ? + (OBD_TIMEOUT_DEFAULT / 4) : 1; +EXPORT_SYMBOL(ping_interval); +unsigned int ping_evict_timeout_multiplier = 6; +EXPORT_SYMBOL(ping_evict_timeout_multiplier); unsigned int obd_timeout_set; EXPORT_SYMBOL(obd_timeout_set); /* Adaptive timeout defs here instead of ptlrpc module for /sys/fs/ access */ diff --git a/fs/lustre/obdclass/obd_config.c b/fs/lustre/obdclass/obd_config.c index 4db7399..7d001ff 100644 --- a/fs/lustre/obdclass/obd_config.c +++ b/fs/lustre/obdclass/obd_config.c @@ -869,6 +869,7 @@ int class_process_config(struct lustre_cfg *lcfg) CDEBUG(D_IOCTL, "changing lustre timeout from %d to %d\n", obd_timeout, lcfg->lcfg_num); obd_timeout = max(lcfg->lcfg_num, 1U); + ping_interval = max(obd_timeout / 4, 1U); obd_timeout_set = 1; err = 0; goto out; diff --git a/fs/lustre/obdclass/obd_sysfs.c b/fs/lustre/obdclass/obd_sysfs.c index 93d2abc..fc8debb 100644 --- a/fs/lustre/obdclass/obd_sysfs.c +++ b/fs/lustre/obdclass/obd_sysfs.c @@ -109,7 +109,6 @@ static ssize_t static_uintvalue_store(struct kobject *kobj, { __ATTR(name, 0644, static_uintvalue_show, \ static_uintvalue_store), value } -LUSTRE_STATIC_UINT_ATTR(timeout, &obd_timeout); LUSTRE_STATIC_UINT_ATTR(debug_peer_on_timeout, &obd_debug_peer_on_timeout); LUSTRE_STATIC_UINT_ATTR(dump_on_timeout, &obd_dump_on_timeout); LUSTRE_STATIC_UINT_ATTR(dump_on_eviction, &obd_dump_on_eviction); @@ -119,6 +118,8 @@ static ssize_t static_uintvalue_store(struct kobject *kobj, LUSTRE_STATIC_UINT_ATTR(at_early_margin, &at_early_margin); LUSTRE_STATIC_UINT_ATTR(at_history, &at_history); LUSTRE_STATIC_UINT_ATTR(lbug_on_eviction, &obd_lbug_on_eviction); +LUSTRE_STATIC_UINT_ATTR(ping_interval, &ping_interval); +LUSTRE_STATIC_UINT_ATTR(evict_multiplier, &ping_evict_timeout_multiplier); static ssize_t max_dirty_mb_show(struct kobject *kobj, struct attribute *attr, char *buf) @@ -311,6 +312,30 @@ static ssize_t jobid_this_session_store(struct kobject *kobj, return ret ?: count; } +static ssize_t timeout_show(struct kobject *kobj, + struct attribute *attr, + char *buf) +{ + return sprintf(buf, "%u\n", obd_timeout); +} + +static ssize_t timeout_store(struct kobject *kobj, + struct attribute *attr, + const char *buffer, + size_t count) +{ + unsigned int val; + int rc; + + rc = kstrtouint(buffer, 10, &val); + if (rc) + return rc; + obd_timeout = val ?: 1U; + ping_interval = max(obd_timeout / 4, 1U); + + return count; +} + /* Root for /sys/kernel/debug/lustre */ struct dentry *debugfs_lustre_root; EXPORT_SYMBOL_GPL(debugfs_lustre_root); @@ -321,6 +346,7 @@ static ssize_t jobid_this_session_store(struct kobject *kobj, LUSTRE_RW_ATTR(jobid_var); LUSTRE_RW_ATTR(jobid_name); LUSTRE_RW_ATTR(jobid_this_session); +LUSTRE_RW_ATTR(timeout); static struct attribute *lustre_attrs[] = { &lustre_attr_version.attr, @@ -329,7 +355,7 @@ static ssize_t jobid_this_session_store(struct kobject *kobj, &lustre_attr_jobid_name.attr, &lustre_attr_jobid_var.attr, &lustre_attr_jobid_this_session.attr, - &lustre_sattr_timeout.u.attr, + &lustre_attr_timeout.attr, &lustre_attr_max_dirty_mb.attr, &lustre_sattr_debug_peer_on_timeout.u.attr, &lustre_sattr_dump_on_timeout.u.attr, @@ -340,6 +366,8 @@ static ssize_t jobid_this_session_store(struct kobject *kobj, &lustre_sattr_at_early_margin.u.attr, &lustre_sattr_at_history.u.attr, &lustre_sattr_lbug_on_eviction.u.attr, + &lustre_sattr_ping_interval.u.attr, + &lustre_sattr_evict_multiplier.u.attr, NULL, }; From patchwork Sun Sep 18 05:22:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979330 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 19209C6FA86 for ; Sun, 18 Sep 2022 05:23:15 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbmy5x93z1yD9; Sat, 17 Sep 2022 22:23:14 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVbmB74htz1yDM for ; Sat, 17 Sep 2022 22:22:34 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 17A178F16; Sun, 18 Sep 2022 01:22:17 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 14B74E8B94; Sun, 18 Sep 2022 01:22:17 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:22:12 -0400 Message-Id: <1663478534-19917-23-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 22/24] lnet: allow direct messages regardless of peer NI status X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Serguei Smirnov , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Serguei Smirnov If check_routers_before_use is enabled, the router needs to be pinged before it is used, which is not possible because its NIs are assumed to be down at start-up. Don't prevent discovery of the router in this case. This change allows non-routed traffic to peer NIs with "down" status. WC-bug-id: https://jira.whamcloud.com/browse/LU-16106 Lustre-commit: 3345a8a54e89c342a4ce ("LU-16106 lnet: allow direct messages regardless of peer NI status") Signed-off-by: Serguei Smirnov Reviewed-on: https://review.whamcloud.com/48355 Reviewed-by: Frank Sehr Reviewed-by: Chris Horn Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/lnet/lib-move.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/net/lnet/lnet/lib-move.c b/net/lnet/lnet/lib-move.c index 3c9602e..a8a5ddb 100644 --- a/net/lnet/lnet/lib-move.c +++ b/net/lnet/lnet/lib-move.c @@ -597,6 +597,10 @@ void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats, if (lnet_msg_is_response(msg)) return false; + /* always send non-routed messages */ + if (!msg->msg_routing) + return false; + /* assume peer_ni is alive as long as we're within the configured * peer timeout */ From patchwork Sun Sep 18 05:22:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979327 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3EBBCC6FA8B for ; Sun, 18 Sep 2022 05:23:07 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbmp6V5Cz1yFg; Sat, 17 Sep 2022 22:23:06 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVbmC4sPkz1yD9 for ; Sat, 17 Sep 2022 22:22:35 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 198838F17; Sun, 18 Sep 2022 01:22:17 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 17D99E8BA5; Sun, 18 Sep 2022 01:22:17 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:22:13 -0400 Message-Id: <1663478534-19917-24-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 23/24] lnet: Honor peer timeout of zero X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Chris Horn , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Chris Horn Zero is a valid value for the peer_timeout parameter (it is supposed to disable the LNet Peer Health feature used on routers), but DLC treats zero as uninitialized and assigns the default peer timeout instead. HPE-bug-id: LUS-11233 WC-bug-id: https://jira.whamcloud.com/browse/LU-16145 Lustre-commit: eee27c88adc39da36 ("LU-16145 lnet: Honor peer timeout of zero") Signed-off-by: Chris Horn Reviewed-on: https://review.whamcloud.com/48489 Reviewed-by: Frank Sehr Reviewed-by: Serguei Smirnov Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/lnet/api-ni.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/net/lnet/lnet/api-ni.c b/net/lnet/lnet/api-ni.c index 9bf2860..89c7b99 100644 --- a/net/lnet/lnet/api-ni.c +++ b/net/lnet/lnet/api-ni.c @@ -3466,7 +3466,7 @@ static int lnet_add_net_common(struct lnet_net *net, lnet_set_tune_defaults(struct lnet_ioctl_config_lnd_tunables *tun) { if (tun) { - if (!tun->lt_cmn.lct_peer_timeout) + if (tun->lt_cmn.lct_peer_timeout < 0) tun->lt_cmn.lct_peer_timeout = DEFAULT_PEER_TIMEOUT; if (!tun->lt_cmn.lct_peer_tx_credits) tun->lt_cmn.lct_peer_tx_credits = DEFAULT_PEER_CREDITS; From patchwork Sun Sep 18 05:22:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12979328 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 72D6BC32771 for ; Sun, 18 Sep 2022 05:23:09 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4MVbms0xVjz1y7Z; Sat, 17 Sep 2022 22:23:09 -0700 (PDT) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4MVbmD2bNQz1y7v for ; Sat, 17 Sep 2022 22:22:36 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 1C50D8F18; Sun, 18 Sep 2022 01:22:17 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 1B67AE8BA9; Sun, 18 Sep 2022 01:22:17 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 18 Sep 2022 01:22:14 -0400 Message-Id: <1663478534-19917-25-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> References: <1663478534-19917-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 24/24] lustre: update version to 2.15.52 X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Oleg Drokin New tag 2.15.52 Signed-off-by: Oleg Drokin Signed-off-by: James Simmons --- include/uapi/linux/lustre/lustre_ver.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/uapi/linux/lustre/lustre_ver.h b/include/uapi/linux/lustre/lustre_ver.h index 7e15b81..d9a6847 100644 --- a/include/uapi/linux/lustre/lustre_ver.h +++ b/include/uapi/linux/lustre/lustre_ver.h @@ -3,9 +3,9 @@ #define LUSTRE_MAJOR 2 #define LUSTRE_MINOR 15 -#define LUSTRE_PATCH 51 +#define LUSTRE_PATCH 52 #define LUSTRE_FIX 0 -#define LUSTRE_VERSION_STRING "2.15.51" +#define LUSTRE_VERSION_STRING "2.15.52" #define OBD_OCD_VERSION(major, minor, patch, fix) \ (((major) << 24) + ((minor) << 16) + ((patch) << 8) + (fix))