From patchwork Wed Nov 3 11:24:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Luis Henriques X-Patchwork-Id: 12600599 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21F59C433F5 for ; Wed, 3 Nov 2021 11:24:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F30DA611C6 for ; Wed, 3 Nov 2021 11:24:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231903AbhKCL0q (ORCPT ); Wed, 3 Nov 2021 07:26:46 -0400 Received: from smtp-out2.suse.de ([195.135.220.29]:41298 "EHLO smtp-out2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230250AbhKCL0p (ORCPT ); Wed, 3 Nov 2021 07:26:45 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 529F61FD4C; Wed, 3 Nov 2021 11:24:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1635938648; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=f5AjT56p9WNY85grFsex7X6u+OrrmGyvo0mAAlM+lE4=; b=pwENhj3cOsVsChFIbP4GCN9HWRmkF9Pd39wOD1DPFXGrkmkbrNzaETMWvPFL402+Dj25dr odckUUyCPx2r5zzG2yQizXEQqYqVUj3tfQwI/UYRnt9hty7Sxwj9s2aIw18txsXdmOoGcu QoviQYtoA1CbIP/EaIVL+TvHJxAkCLw= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1635938648; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=f5AjT56p9WNY85grFsex7X6u+OrrmGyvo0mAAlM+lE4=; b=CtMs3PxP8uWr7D8O27H9i/iLg3IR2b5aOMyv52LqDJ4YeCsuxOqyLlX2I7sby6UegzCtpS 2+eJPwNcUP+N58Cg== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id CE4AF13DC1; Wed, 3 Nov 2021 11:24:07 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id MLOmL1dxgmHyMQAAMHmgww (envelope-from ); Wed, 03 Nov 2021 11:24:07 +0000 Received: from localhost (brahms [local]) by brahms (OpenSMTPD) with ESMTPA id 09cb5d08; Wed, 3 Nov 2021 11:24:06 +0000 (UTC) From: =?utf-8?q?Lu=C3=ADs_Henriques?= To: Jeff Layton , Ilya Dryomov , Xiubo Li Cc: Patrick Donnelly , ceph-devel@vger.kernel.org, linux-kernel@vger.kernel.org, =?utf-8?q?Lu=C3=ADs_Henriques?= Subject: [PATCH 1/2] libceph: have ceph_osdc_copy_from() return the osd request Date: Wed, 3 Nov 2021 11:24:04 +0000 Message-Id: <20211103112405.8733-2-lhenriques@suse.de> In-Reply-To: <20211103112405.8733-1-lhenriques@suse.de> References: <20211103112405.8733-1-lhenriques@suse.de> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org This patch modifies the behaviour of ceph_osdc_copy_from() function so that it will create the osd request and send it but won't block waiting for the result. It is now the responsibility of the callers (currently only ceph_do_objects_copy()) to do the wait and release the request. Signed-off-by: Luís Henriques --- fs/ceph/file.c | 9 ++++++++- include/linux/ceph/osd_client.h | 21 +++++++++++---------- net/ceph/osd_client.c | 27 ++++++++++++++------------- 3 files changed, 33 insertions(+), 24 deletions(-) diff --git a/fs/ceph/file.c b/fs/ceph/file.c index 6005b430f6f7..a39703b8ef99 100644 --- a/fs/ceph/file.c +++ b/fs/ceph/file.c @@ -2218,6 +2218,7 @@ static ssize_t ceph_do_objects_copy(struct ceph_inode_info *src_ci, u64 *src_off { struct ceph_object_locator src_oloc, dst_oloc; struct ceph_object_id src_oid, dst_oid; + struct ceph_osd_request *req; size_t bytes = 0; u64 src_objnum, src_objoff, dst_objnum, dst_objoff; u32 src_objlen, dst_objlen; @@ -2243,7 +2244,7 @@ static ssize_t ceph_do_objects_copy(struct ceph_inode_info *src_ci, u64 *src_off ceph_oid_printf(&dst_oid, "%llx.%08llx", dst_ci->i_vino.ino, dst_objnum); /* Do an object remote copy */ - ret = ceph_osdc_copy_from(&fsc->client->osdc, + req = ceph_osdc_copy_from(&fsc->client->osdc, src_ci->i_vino.snap, 0, &src_oid, &src_oloc, CEPH_OSD_OP_FLAG_FADVISE_SEQUENTIAL | @@ -2254,6 +2255,12 @@ static ssize_t ceph_do_objects_copy(struct ceph_inode_info *src_ci, u64 *src_off dst_ci->i_truncate_seq, dst_ci->i_truncate_size, CEPH_OSD_COPY_FROM_FLAG_TRUNCATE_SEQ); + if (IS_ERR(req)) + ret = PTR_ERR(req); + else { + ret = ceph_osdc_wait_request(&fsc->client->osdc, req); + ceph_osdc_put_request(req); + } if (ret) { if (ret == -EOPNOTSUPP) { fsc->have_copy_from2 = false; diff --git a/include/linux/ceph/osd_client.h b/include/linux/ceph/osd_client.h index 83fa08a06507..74d590cd29c9 100644 --- a/include/linux/ceph/osd_client.h +++ b/include/linux/ceph/osd_client.h @@ -515,16 +515,17 @@ int ceph_osdc_call(struct ceph_osd_client *osdc, struct page *req_page, size_t req_len, struct page **resp_pages, size_t *resp_len); -int ceph_osdc_copy_from(struct ceph_osd_client *osdc, - u64 src_snapid, u64 src_version, - struct ceph_object_id *src_oid, - struct ceph_object_locator *src_oloc, - u32 src_fadvise_flags, - struct ceph_object_id *dst_oid, - struct ceph_object_locator *dst_oloc, - u32 dst_fadvise_flags, - u32 truncate_seq, u64 truncate_size, - u8 copy_from_flags); +struct ceph_osd_request * +ceph_osdc_copy_from(struct ceph_osd_client *osdc, + u64 src_snapid, u64 src_version, + struct ceph_object_id *src_oid, + struct ceph_object_locator *src_oloc, + u32 src_fadvise_flags, + struct ceph_object_id *dst_oid, + struct ceph_object_locator *dst_oloc, + u32 dst_fadvise_flags, + u32 truncate_seq, u64 truncate_size, + u8 copy_from_flags); /* watch/notify */ struct ceph_osd_linger_request * diff --git a/net/ceph/osd_client.c b/net/ceph/osd_client.c index ff8624a7c964..78384b431748 100644 --- a/net/ceph/osd_client.c +++ b/net/ceph/osd_client.c @@ -5347,23 +5347,24 @@ static int osd_req_op_copy_from_init(struct ceph_osd_request *req, return 0; } -int ceph_osdc_copy_from(struct ceph_osd_client *osdc, - u64 src_snapid, u64 src_version, - struct ceph_object_id *src_oid, - struct ceph_object_locator *src_oloc, - u32 src_fadvise_flags, - struct ceph_object_id *dst_oid, - struct ceph_object_locator *dst_oloc, - u32 dst_fadvise_flags, - u32 truncate_seq, u64 truncate_size, - u8 copy_from_flags) +struct ceph_osd_request * +ceph_osdc_copy_from(struct ceph_osd_client *osdc, + u64 src_snapid, u64 src_version, + struct ceph_object_id *src_oid, + struct ceph_object_locator *src_oloc, + u32 src_fadvise_flags, + struct ceph_object_id *dst_oid, + struct ceph_object_locator *dst_oloc, + u32 dst_fadvise_flags, + u32 truncate_seq, u64 truncate_size, + u8 copy_from_flags) { struct ceph_osd_request *req; int ret; req = ceph_osdc_alloc_request(osdc, NULL, 1, false, GFP_KERNEL); if (!req) - return -ENOMEM; + return ERR_PTR(-ENOMEM); req->r_flags = CEPH_OSD_FLAG_WRITE; @@ -5382,11 +5383,11 @@ int ceph_osdc_copy_from(struct ceph_osd_client *osdc, goto out; ceph_osdc_start_request(osdc, req, false); - ret = ceph_osdc_wait_request(osdc, req); + return req; out: ceph_osdc_put_request(req); - return ret; + return ERR_PTR(ret); } EXPORT_SYMBOL(ceph_osdc_copy_from); From patchwork Wed Nov 3 11:24:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Luis Henriques X-Patchwork-Id: 12600601 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35FF7C433FE for ; Wed, 3 Nov 2021 11:24:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 10B4A6113B for ; Wed, 3 Nov 2021 11:24:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231354AbhKCL0v (ORCPT ); Wed, 3 Nov 2021 07:26:51 -0400 Received: from smtp-out2.suse.de ([195.135.220.29]:41316 "EHLO smtp-out2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230291AbhKCL0q (ORCPT ); Wed, 3 Nov 2021 07:26:46 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id D48071FD4F; Wed, 3 Nov 2021 11:24:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1635938648; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=E4A9LwbWSrS822C4Vswjzc2ote0q6H+7MR5LublF8dA=; b=YP4cLlGpiKstukvxrqhyZErxBsYBAlHf3xpMbT/LR1TJ4QdEy/n7Tt6/mxFncVj5hRGctb TYpaRDXAWK+4FwYb945qaGuecV77UN2WHvrTenDNfgSgvurGfJYIqEFlnIQx1k+D4haE/T FHefFLY2PuKODg8Pkf8HhSioxOR7VK0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1635938648; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=E4A9LwbWSrS822C4Vswjzc2ote0q6H+7MR5LublF8dA=; b=9cw2Odc4aGglhxaAff4lQFC5iERUnH615q0jCqrmB5sLvzQ5vIBCizv8CYpfpdReyi7RSe ZDg9SrbCIuWP0uAg== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 5931914060; Wed, 3 Nov 2021 11:24:08 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id qFD5ElhxgmHyMQAAMHmgww (envelope-from ); Wed, 03 Nov 2021 11:24:08 +0000 Received: from localhost (brahms [local]) by brahms (OpenSMTPD) with ESMTPA id 3b547176; Wed, 3 Nov 2021 11:24:07 +0000 (UTC) From: =?utf-8?q?Lu=C3=ADs_Henriques?= To: Jeff Layton , Ilya Dryomov , Xiubo Li Cc: Patrick Donnelly , ceph-devel@vger.kernel.org, linux-kernel@vger.kernel.org, =?utf-8?q?Lu=C3=ADs_Henriques?= Subject: [PATCH 2/2] ceph: add a new metric to keep track of remote object copies Date: Wed, 3 Nov 2021 11:24:05 +0000 Message-Id: <20211103112405.8733-3-lhenriques@suse.de> In-Reply-To: <20211103112405.8733-1-lhenriques@suse.de> References: <20211103112405.8733-1-lhenriques@suse.de> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org This patch adds latency and size metrics for remote object copies operations ("copyfrom"). For now, these metrics will be available on the client only, they won't be sent to the MDS. Cc: Patrick Donnelly Signed-off-by: Luís Henriques --- fs/ceph/debugfs.c | 3 ++- fs/ceph/file.c | 4 ++++ fs/ceph/metric.h | 8 ++++++++ 3 files changed, 14 insertions(+), 1 deletion(-) diff --git a/fs/ceph/debugfs.c b/fs/ceph/debugfs.c index e04ae1098431..3cf7c9c1085b 100644 --- a/fs/ceph/debugfs.c +++ b/fs/ceph/debugfs.c @@ -167,7 +167,8 @@ static int metrics_file_show(struct seq_file *s, void *p) static const char * const metric_str[] = { "read", "write", - "metadata" + "metadata", + "copyfrom" }; static int metrics_latency_show(struct seq_file *s, void *p) { diff --git a/fs/ceph/file.c b/fs/ceph/file.c index a39703b8ef99..c61d71cef55d 100644 --- a/fs/ceph/file.c +++ b/fs/ceph/file.c @@ -2259,6 +2259,10 @@ static ssize_t ceph_do_objects_copy(struct ceph_inode_info *src_ci, u64 *src_off ret = PTR_ERR(req); else { ret = ceph_osdc_wait_request(&fsc->client->osdc, req); + ceph_update_copyfrom_metrics(&fsc->mdsc->metric, + req->r_start_latency, + req->r_end_latency, + object_size, ret); ceph_osdc_put_request(req); } if (ret) { diff --git a/fs/ceph/metric.h b/fs/ceph/metric.h index e67fc997760b..bb45608181e7 100644 --- a/fs/ceph/metric.h +++ b/fs/ceph/metric.h @@ -129,6 +129,7 @@ enum metric_type { METRIC_READ, METRIC_WRITE, METRIC_METADATA, + METRIC_COPYFROM, METRIC_MAX }; @@ -214,4 +215,11 @@ static inline void ceph_update_metadata_metrics(struct ceph_client_metric *m, ceph_update_metrics(&m->metric[METRIC_METADATA], r_start, r_end, 0, rc); } +static inline void ceph_update_copyfrom_metrics(struct ceph_client_metric *m, + ktime_t r_start, ktime_t r_end, + unsigned int size, int rc) +{ + ceph_update_metrics(&m->metric[METRIC_COPYFROM], + r_start, r_end, size, rc); +} #endif /* _FS_CEPH_MDS_METRIC_H */