From patchwork Sat Jan 26 20:41:21 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Elder X-Patchwork-Id: 2050321 Return-Path: X-Original-To: patchwork-ceph-devel@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork1.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork1.kernel.org (Postfix) with ESMTP id 7A8113FCDE for ; Sat, 26 Jan 2013 20:41:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754324Ab3AZUlX (ORCPT ); Sat, 26 Jan 2013 15:41:23 -0500 Received: from mail-ia0-f171.google.com ([209.85.210.171]:54816 "EHLO mail-ia0-f171.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754188Ab3AZUlX (ORCPT ); Sat, 26 Jan 2013 15:41:23 -0500 Received: by mail-ia0-f171.google.com with SMTP id z13so2427263iaz.16 for ; Sat, 26 Jan 2013 12:41:22 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:subject :references:in-reply-to:content-type:content-transfer-encoding :x-gm-message-state; bh=i2HhV9KENNElLXwBtcgZrJlTPUQXKHu7COj6fv7sblg=; b=M9YdYIJ+cV8JNCr/uDFp8nUiWzy5fuXkzSRtbmOB9iQ9ta96SjNPG1j7uH/t6FTQy2 4DwdSDCbicVS97w08cUaM4CQAEVmTnZB4aq36eYxASSw1ln7z8jVQ2ekAlxTnasP+fil okJWk7tEFxd5uJfCf84qASgKujddSCasDkpnmWa6RnnSBOt/I0e8gxbVH991M0Dw639W AnrNdtUxWhe9zO3y1mWrtasZxibhu0kRDwKMJXPVIopUkIUJdWBxwFmCAppDm4iGzsK5 yXU6ig8grW6u1I1Gov4/jI2iknUZIUk/sAVfV776cN0NbEzWn/tuijL+cOIK0xYqe2i1 tYmA== X-Received: by 10.50.187.134 with SMTP id fs6mr1693064igc.79.1359232882864; Sat, 26 Jan 2013 12:41:22 -0800 (PST) Received: from [172.22.22.4] (c-71-195-31-37.hsd1.mn.comcast.net. [71.195.31.37]) by mx.google.com with ESMTPS id u4sm2367838igw.6.2013.01.26.12.41.21 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 26 Jan 2013 12:41:21 -0800 (PST) Message-ID: <51043F71.3050208@inktank.com> Date: Sat, 26 Jan 2013 14:41:21 -0600 From: Alex Elder User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130106 Thunderbird/17.0.2 MIME-Version: 1.0 To: "ceph-devel@vger.kernel.org" Subject: [PATCH 4/4] rbd: don't drop watch requests on completion References: <51043EF2.4070305@inktank.com> In-Reply-To: <51043EF2.4070305@inktank.com> X-Gm-Message-State: ALoCoQlFqgkT3Q86zhmSnvVNSSzupdIrfF1kJiBOLOpnScDXGqiPTDnMiAvvRpbYxxTX1v/qFQVL Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org The new request code arranges to get a callback for every osd request we submit (this was not the case previously). We register a lingering object watch request for the header object for each mapped rbd image. If a connection problem occurs, the osd client will re-submit lingering requests. And each time such a request is re-submitted, its callback function will get called again. We therefore need to ensure the object request associated with the lingering osd request stays valid, and the way to do that is to have an extra reference to the lingering osd request. So when a request to initiate a watch has completed, do not drop a reference as one normally would. Instead, hold off dropping that reference until the request to tear down that watch request is done. Also, only set the rbd device's watch_request pointer after the watch request has been completed successfully, and clear the pointer once it's been torn down. Signed-off-by: Alex Elder --- drivers/block/rbd.c | 31 ++++++++++++++++++++++--------- 1 file changed, 22 insertions(+), 9 deletions(-) ret = -ENOMEM; @@ -1735,32 +1736,44 @@ static int rbd_dev_header_watch_sync(struct rbd_device *rbd_dev, int start) if (!obj_request->osd_req) goto out_cancel; - if (start) { + if (start) ceph_osdc_set_request_linger(osdc, obj_request->osd_req); - rbd_dev->watch_request = obj_request; - } else { + else ceph_osdc_unregister_linger_request(osdc, rbd_dev->watch_request->osd_req); - rbd_dev->watch_request = NULL; - } ret = rbd_obj_request_submit(osdc, obj_request); if (ret) goto out_cancel; ret = rbd_obj_request_wait(obj_request); if (ret) goto out_cancel; - ret = obj_request->result; if (ret) goto out_cancel; - if (start) - goto done; /* Done if setting up the watch request */ + /* + * Since a watch request is set to linger the osd client + * will hang onto it in case it needs to be re-sent in the + * event of connection loss. If we're initiating the watch + * we therefore do *not* want to drop our reference to the + * object request now; we'll effectively transfer ownership + * of it to the osd client instead. Instead, we'll drop + * that reference when the watch request gets torn down. + */ + if (start) { + rbd_dev->watch_request = obj_request; + + return 0; + } + + /* We have successfully torn down the watch request */ + + rbd_obj_request_put(rbd_dev->watch_request); + rbd_dev->watch_request = NULL; out_cancel: /* Cancel the event if we're tearing down, or on error */ ceph_osdc_cancel_event(rbd_dev->watch_event); rbd_dev->watch_event = NULL; -done: if (obj_request) rbd_obj_request_put(obj_request); diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index 340773f..177ba0c 100644 --- a/drivers/block/rbd.c +++ b/drivers/block/rbd.c @@ -1716,6 +1716,7 @@ static int rbd_dev_header_watch_sync(struct rbd_device *rbd_dev, int start) &rbd_dev->watch_event); if (ret < 0) return ret; + rbd_assert(rbd_dev->watch_event != NULL); }