From patchwork Wed Mar 20 14:56:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Dryomov X-Patchwork-Id: 10862067 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6523D17E9 for ; Wed, 20 Mar 2019 14:55:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4A8D828AE0 for ; Wed, 20 Mar 2019 14:55:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3E569299DA; Wed, 20 Mar 2019 14:55:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CCED728AE0 for ; Wed, 20 Mar 2019 14:55:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728512AbfCTOzn (ORCPT ); Wed, 20 Mar 2019 10:55:43 -0400 Received: from mail-wm1-f66.google.com ([209.85.128.66]:53579 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727929AbfCTOzm (ORCPT ); Wed, 20 Mar 2019 10:55:42 -0400 Received: by mail-wm1-f66.google.com with SMTP id q16so1219774wmj.3 for ; Wed, 20 Mar 2019 07:55:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=agfoJ3rFLkW+wbK1tlJBH5nxSajBYYCd/rapSteNUjE=; b=HRsKWGy60RUftN6ASvKlOUG46MgI7OgcfQZnNpk4PLyJdf/eshK28v3Xwh5s3R3d3w HBEQjE5rEqPtFWfhqs8hsC91d4UM3FJUgH/JLluCxY1I63vJFlgGN6l+PAbHSxXD4MNh i/D4N8xEJr/xEt6m475Y38qdgDglAVsrcI85ySI5mCBiwMdx8Cg7R24auVdMfV7hdNxm wXIYFV9EZc6nY2VLoloqW9xWoO/lUv6cZT0j6oPRByUkv2ilhduVT62XfQYiHhxGg9Ze BvL7aLE9ie6SIc0l/85Wkpg7CVw/E+3dZ9B/xCGr5aeps9V0oORviQfWaA+C+rB4SJp2 skaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=agfoJ3rFLkW+wbK1tlJBH5nxSajBYYCd/rapSteNUjE=; b=Tb2UJLkWM7BSgKPBUsZnW+GjxiV9HtaKz/D7q01k+JQRaSppBUIKm1xyLMYwzMaOte 7GfVNcGbkGUUjNIrZzOPzgisGiUhPtVaCbkeeJ5orYL9u72c2GrJ5ABM314nlo6MNiGc z5VNhQfxbeYAiRR72/CtvqL79CH/rmNhl/fBsKgTUVVi3fbbBJqPL0ZBuRpbnq02MqQY 466m0cQ9fwm7nTyIHLEPNXGXI8A744fABSFO73zSq8npBJuwdGSuk4wyEsnCEA0JXU/a knO/UnOgm8DKOR8T+kU4cyANfgbiP5ipZmgBUet6hM7iJXQmeS2fNCj+Huf2P1sDdV+w MC6g== X-Gm-Message-State: APjAAAU95lXQ2huDu2U5osvX1WugPS2yQqm+NroAtJtMtCZXXWYLIgZP noSh3WOsvFGajbzFLN3t0lSjsxsc X-Google-Smtp-Source: APXvYqw/+gHwVutTD4i1eA6tVx0ze17vkgiqU+r/eERF/oG7vzvs4zG1hBHj90Z6LhZ6CW/OxJj7WQ== X-Received: by 2002:a1c:d189:: with SMTP id i131mr8700294wmg.151.1553093741044; Wed, 20 Mar 2019 07:55:41 -0700 (PDT) Received: from kwango.redhat.com (ovpn-brq.redhat.com. [213.175.37.11]) by smtp.gmail.com with ESMTPSA id z6sm3311931wml.40.2019.03.20.07.55.39 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 20 Mar 2019 07:55:40 -0700 (PDT) From: Ilya Dryomov To: ceph-devel@vger.kernel.org Subject: [PATCH 1/2] libceph: wait for latest osdmap in ceph_monc_blacklist_add() Date: Wed, 20 Mar 2019 15:56:41 +0100 Message-Id: <20190320145642.31214-1-idryomov@gmail.com> X-Mailer: git-send-email 2.19.2 MIME-Version: 1.0 Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Because map updates are distributed lazily, an OSD may not know about the new blacklist for quite some time after "osd blacklist add" command is completed. This makes it possible for a blacklisted but still alive client to overwrite a post-blacklist update, resulting in data corruption. Waiting for latest osdmap in ceph_monc_blacklist_add() and thus using the post-blacklist epoch for all post-blacklist requests ensures that all such requests "wait" for the blacklist to come into force on their respective OSDs. Cc: stable@vger.kernel.org Fixes: 6305a3b41515 ("libceph: support for blacklisting clients") Signed-off-by: Ilya Dryomov --- include/linux/ceph/libceph.h | 2 ++ net/ceph/ceph_common.c | 18 +++++++++++++++++- net/ceph/mon_client.c | 9 +++++++++ 3 files changed, 28 insertions(+), 1 deletion(-) diff --git a/include/linux/ceph/libceph.h b/include/linux/ceph/libceph.h index a420c07904bc..337d5049ff93 100644 --- a/include/linux/ceph/libceph.h +++ b/include/linux/ceph/libceph.h @@ -294,6 +294,8 @@ extern void ceph_destroy_client(struct ceph_client *client); extern int __ceph_open_session(struct ceph_client *client, unsigned long started); extern int ceph_open_session(struct ceph_client *client); +int ceph_wait_for_latest_osdmap(struct ceph_client *client, + unsigned long timeout); /* pagevec.c */ extern void ceph_release_page_vector(struct page **pages, int num_pages); diff --git a/net/ceph/ceph_common.c b/net/ceph/ceph_common.c index 9cab80207ced..79eac465ec65 100644 --- a/net/ceph/ceph_common.c +++ b/net/ceph/ceph_common.c @@ -738,7 +738,6 @@ int __ceph_open_session(struct ceph_client *client, unsigned long started) } EXPORT_SYMBOL(__ceph_open_session); - int ceph_open_session(struct ceph_client *client) { int ret; @@ -754,6 +753,23 @@ int ceph_open_session(struct ceph_client *client) } EXPORT_SYMBOL(ceph_open_session); +int ceph_wait_for_latest_osdmap(struct ceph_client *client, + unsigned long timeout) +{ + u64 newest_epoch; + int ret; + + ret = ceph_monc_get_version(&client->monc, "osdmap", &newest_epoch); + if (ret) + return ret; + + if (client->osdc.osdmap->epoch >= newest_epoch) + return 0; + + ceph_osdc_maybe_request_map(&client->osdc); + return ceph_monc_wait_osdmap(&client->monc, newest_epoch, timeout); +} +EXPORT_SYMBOL(ceph_wait_for_latest_osdmap); static int __init init_ceph_lib(void) { diff --git a/net/ceph/mon_client.c b/net/ceph/mon_client.c index 18deb3d889c4..a53e4fbb6319 100644 --- a/net/ceph/mon_client.c +++ b/net/ceph/mon_client.c @@ -922,6 +922,15 @@ int ceph_monc_blacklist_add(struct ceph_mon_client *monc, mutex_unlock(&monc->mutex); ret = wait_generic_request(req); + if (!ret) + /* + * Make sure we have the osdmap that includes the blacklist + * entry. This is needed to ensure that the OSDs pick up the + * new blacklist before processing any future requests from + * this client. + */ + ret = ceph_wait_for_latest_osdmap(monc->client, 0); + out: put_generic_request(req); return ret; From patchwork Wed Mar 20 14:56:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Dryomov X-Patchwork-Id: 10862071 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9838D14DE for ; Wed, 20 Mar 2019 14:55:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7F787299DA for ; Wed, 20 Mar 2019 14:55:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 73C5729DAD; Wed, 20 Mar 2019 14:55:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 19AB7299DA for ; Wed, 20 Mar 2019 14:55:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728536AbfCTOzp (ORCPT ); Wed, 20 Mar 2019 10:55:45 -0400 Received: from mail-wr1-f66.google.com ([209.85.221.66]:42316 "EHLO mail-wr1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726123AbfCTOzo (ORCPT ); Wed, 20 Mar 2019 10:55:44 -0400 Received: by mail-wr1-f66.google.com with SMTP id n9so3097555wrr.9 for ; Wed, 20 Mar 2019 07:55:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Igj+6zeSZOSCAqdyZWtDGfjbbd3ANqyoqd59BTe0UvI=; b=Do8tAmMhUtKz3jXC25qGgLeA8KmNZ6sVS1IDEesKldOfANjEQSuNs46uvroIY4RJmS ReFit0zvS5l89Sdj4O3FP1MrADtAi2rkKSf9bsr7cWUST1O1reSgfQDZDVnNb+Tvjj/z 2onAyQPINP73rmykGoEnYRpptCFWQeZgzxIqmDKnvr4tKeQgT7sDPnFWNSbqqUjvOdsb W1TpCbX+hNinDv3y6n6x55afaaKz9Yq/uuiFeEtkVxMEy8ZNnYYZc5EBzorW+hNvA89g HDL81BfX+WhBwsyU1188cf0eniP/METGxFZmdF38zkO518d0/Kiad4z3gV4w+lZ+bNjM smqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Igj+6zeSZOSCAqdyZWtDGfjbbd3ANqyoqd59BTe0UvI=; b=E59yLqXoedps56FiFURMBsxAmjHgPVwQ8PklO6/aj+h7RYR/jtzAmT8DPxcU3kv3sc LSfd0KP00faa7yIb/Ph8+SeBwuG/ofHvMF4VvMHzPPnUraAd6RFdIT9yyf8Cb4ceO+6G i3xX4j14SegEFTwjCEAmnPGLfQ8JlGJGZbB5qV+Qg2pHuF5l0udovQzZjyo2ZD6DC9d/ epn76EDf0QUh7HzWNqzbKcroxDlJV1kuWMZbZaGcj5SKpXK46K7kbvnNqdPHx3lB8fr1 s38qCZnzv6aZnzqbB+QWhcuRGzEQ1qlzngKYn2CHWgyygrEVvQVkpe19Mwz/Esi/IZMN 92yw== X-Gm-Message-State: APjAAAUPgDZJdHxUnMkifQLknPsTRi4S5wjx+Rr3zM1cPL9XAfW2CSFq TJS5E97T35x3c4eHfFFs40JQaf7V X-Google-Smtp-Source: APXvYqxfpVbywAXNbLbLI+KRrjaqAVx8CchOCgk2i9WbeOVAQvl2GpcxcnmcfQrZceQpQUnKfItpVQ== X-Received: by 2002:adf:ea04:: with SMTP id q4mr11325102wrm.97.1553093742367; Wed, 20 Mar 2019 07:55:42 -0700 (PDT) Received: from kwango.redhat.com (ovpn-brq.redhat.com. [213.175.37.11]) by smtp.gmail.com with ESMTPSA id z6sm3311931wml.40.2019.03.20.07.55.41 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 20 Mar 2019 07:55:41 -0700 (PDT) From: Ilya Dryomov To: ceph-devel@vger.kernel.org Subject: [PATCH 2/2] rbd: drop wait_for_latest_osdmap() Date: Wed, 20 Mar 2019 15:56:42 +0100 Message-Id: <20190320145642.31214-2-idryomov@gmail.com> X-Mailer: git-send-email 2.19.2 In-Reply-To: <20190320145642.31214-1-idryomov@gmail.com> References: <20190320145642.31214-1-idryomov@gmail.com> MIME-Version: 1.0 Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Ilya Dryomov Reviewed-by: Jason Dillaman --- drivers/block/rbd.c | 20 ++------------------ 1 file changed, 2 insertions(+), 18 deletions(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index e037f1ab2fde..bae0668db84b 100644 --- a/drivers/block/rbd.c +++ b/drivers/block/rbd.c @@ -925,23 +925,6 @@ static void rbd_put_client(struct rbd_client *rbdc) kref_put(&rbdc->kref, rbd_client_release); } -static int wait_for_latest_osdmap(struct ceph_client *client) -{ - u64 newest_epoch; - int ret; - - ret = ceph_monc_get_version(&client->monc, "osdmap", &newest_epoch); - if (ret) - return ret; - - if (client->osdc.osdmap->epoch >= newest_epoch) - return 0; - - ceph_osdc_maybe_request_map(&client->osdc); - return ceph_monc_wait_osdmap(&client->monc, newest_epoch, - client->options->mount_timeout); -} - /* * Get a ceph client with specific addr and configuration, if one does * not exist create it. Either way, ceph_opts is consumed by this @@ -961,7 +944,8 @@ static struct rbd_client *rbd_get_client(struct ceph_options *ceph_opts) * Using an existing client. Make sure ->pg_pools is up to * date before we look up the pool id in do_rbd_add(). */ - ret = wait_for_latest_osdmap(rbdc->client); + ret = ceph_wait_for_latest_osdmap(rbdc->client, + rbdc->client->options->mount_timeout); if (ret) { rbd_warn(NULL, "failed to get latest osdmap: %d", ret); rbd_put_client(rbdc);