From patchwork Mon Apr 21 13:26:16 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Dryomov X-Patchwork-Id: 4024941 Return-Path: X-Original-To: patchwork-ceph-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 0E86C9F1F4 for ; Mon, 21 Apr 2014 13:26:32 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 31FD5201F5 for ; Mon, 21 Apr 2014 13:26:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 55C4920375 for ; Mon, 21 Apr 2014 13:26:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752737AbaDUN00 (ORCPT ); Mon, 21 Apr 2014 09:26:26 -0400 Received: from mail-ee0-f49.google.com ([74.125.83.49]:61050 "EHLO mail-ee0-f49.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752281AbaDUN0W (ORCPT ); Mon, 21 Apr 2014 09:26:22 -0400 Received: by mail-ee0-f49.google.com with SMTP id c41so3596864eek.22 for ; Mon, 21 Apr 2014 06:26:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=0sMKE37/eklofQNvuJhXEh3ZCq6/5iAIlnpKUeArCAA=; b=MiDxRby5h31+TUCTz7NP8Yw2UDYzxce+3saxqAbGr31XcZnyysztt+p9DFYBJ06v22 rbWo8ZXE8Wv3+oe/BaI8CgwL5qBkm3G3A8TNf1gwtnd4vE9JZGYxn5Vtyb2fdaOt18U2 WnH+7Ech8QtFr/Kf1P9eQfnhkSeP1L/QJzDl7MGjm8jKpZqT4g0XxWsnvUDzrLXiJqsV VBjWSCJroYr7JkQylN5EFVxL8oE0XCUrU4LwkexsNAB8rCmJmf76cxUvXbV6lIlr5hAR MkKc5KfYEHwDrtHLup+cq4MYJ+NvfqR/zcqgn31HV/w3NysLNOFV5KUY1mzEbVso1kT6 NF8w== X-Gm-Message-State: ALoCoQlWE4czqsttd2LN0bSF7S7PSWB5GB+lTZtPMV4uTjDOXm2L6vWd6t+cfY8jcoRxG6tSHALz X-Received: by 10.14.221.2 with SMTP id q2mr3604123eep.68.1398086781591; Mon, 21 Apr 2014 06:26:21 -0700 (PDT) Received: from localhost ([109.110.67.18]) by mx.google.com with ESMTPSA id u1sm103628764eex.31.2014.04.21.06.26.20 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Mon, 21 Apr 2014 06:26:21 -0700 (PDT) From: Ilya Dryomov To: ceph-devel@vger.kernel.org Subject: [PATCH 2/2] rbd: retry mapping with a new osdmap if pool doesn't exist Date: Mon, 21 Apr 2014 17:26:16 +0400 Message-Id: <1398086776-7228-2-git-send-email-ilya.dryomov@inktank.com> X-Mailer: git-send-email 1.7.10.4 In-Reply-To: <1398086776-7228-1-git-send-email-ilya.dryomov@inktank.com> References: <1398086776-7228-1-git-send-email-ilya.dryomov@inktank.com> Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Spam-Status: No, score=-7.5 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Given an existing idle mapping (img1), mapping an image (img2) in a newly created pool (pool2) fails: $ ceph osd pool create pool1 8 8 $ rbd create --size 1000 pool1/img1 $ sudo rbd map pool1/img1 $ ceph osd pool create pool2 8 8 $ rbd create --size 1000 pool2/img2 $ sudo rbd map pool2/img2 rbd: sysfs write failed rbd: map failed: (2) No such file or directory This is because client instances are shared by default and we don't request an osdmap update when bumping a ref on an existing client. Doing this in a generic way is hard because monitors don't send any meaningful response if the client osdmap epoch is equal to the server epoch. (We get a subscribe ack, but it's stateless and relying on it is probably not a good idea.) So fix this in an ad-hoc way, without hurting the common case. The side effect of this fix is that $ sudo rbd map / can take up to mount_timeout to error out, but it's interruptible (and can be worked around in userspace by checking the arguments more carefully in the future). Signed-off-by: Ilya Dryomov --- drivers/block/rbd.c | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index 552a2edcaa74..9d71c726a691 100644 --- a/drivers/block/rbd.c +++ b/drivers/block/rbd.c @@ -4682,6 +4682,23 @@ out_err: return ret; } +static int rbd_add_get_pool_id(struct rbd_client *rbdc, const char *pool_name) +{ + int tries = 0; + int ret; + +again: + ret = ceph_pg_poolid_by_name(rbdc->client->osdc.osdmap, pool_name); + if (ret == -ENOENT && tries++ < 1) { + ceph_monc_request_next_osdmap(&rbdc->client->monc); + (void) ceph_monc_wait_next_osdmap(&rbdc->client->monc, + rbdc->client->options->mount_timeout * HZ); + goto again; + } + + return ret; +} + /* * An rbd format 2 image has a unique identifier, distinct from the * name given to it by the user. Internally, that identifier is @@ -5053,7 +5070,6 @@ static ssize_t do_rbd_add(struct bus_type *bus, struct rbd_options *rbd_opts = NULL; struct rbd_spec *spec = NULL; struct rbd_client *rbdc; - struct ceph_osd_client *osdc; bool read_only; int rc = -ENOMEM; @@ -5075,8 +5091,7 @@ static ssize_t do_rbd_add(struct bus_type *bus, } /* pick the pool */ - osdc = &rbdc->client->osdc; - rc = ceph_pg_poolid_by_name(osdc->osdmap, spec->pool_name); + rc = rbd_add_get_pool_id(rbdc, spec->pool_name); if (rc < 0) goto err_out_client; spec->pool_id = (u64)rc;