From patchwork Tue Oct 9 21:33:21 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Elder X-Patchwork-Id: 1571691 Return-Path: X-Original-To: patchwork-ceph-devel@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork2.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork2.kernel.org (Postfix) with ESMTP id 58A7DDFFAD for ; Tue, 9 Oct 2012 21:33:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756920Ab2JIVd1 (ORCPT ); Tue, 9 Oct 2012 17:33:27 -0400 Received: from mail-pa0-f46.google.com ([209.85.220.46]:47143 "EHLO mail-pa0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754075Ab2JIVd1 (ORCPT ); Tue, 9 Oct 2012 17:33:27 -0400 Received: by mail-pa0-f46.google.com with SMTP id hz1so5642696pad.19 for ; Tue, 09 Oct 2012 14:33:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding :x-gm-message-state; bh=FNCDyfaqzl0a9R3BXvPISOCLEfxmThHhyjLXk99ZN9k=; b=nLQplPSkGm0fkxhl2EjxY1O6vDDqTgm3wcU09ED5czIzM1v9ba8UsgCkStrnEJsnk5 ODnzkJP2agXGqm8vOUovVXyImZroOFjjBKfH6x6mpPwU2nqnC5XByRJFf6Zqt2aeqhpI Al908fiTX3+k3KPvVZGdPhJ3H6o2TY0O6fZnDp0gstsjNL6yImQCXH+gznPK56n7V8T6 SOPmeREHpiz3Xe7k9FJYATbf+mZeRL0zLNRrhMksSx2bdES4pbA88YsIDEdud1QEKBcv qG0KquKH8xA4GSs/DaWkgYRK4U6t2mtvRLgkwr9epg2qPw702rdS85UewF9Re5piJfGK il2Q== Received: by 10.68.222.105 with SMTP id ql9mr66928258pbc.97.1349818406783; Tue, 09 Oct 2012 14:33:26 -0700 (PDT) Received: from ?IPv6:2607:f298:a:607:4823:f93f:93c7:61da? ([2607:f298:a:607:4823:f93f:93c7:61da]) by mx.google.com with ESMTPS id n7sm10168504pav.26.2012.10.09.14.33.21 (version=SSLv3 cipher=OTHER); Tue, 09 Oct 2012 14:33:25 -0700 (PDT) Message-ID: <50749821.9090907@inktank.com> Date: Tue, 09 Oct 2012 14:33:21 -0700 From: Alex Elder User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20120912 Thunderbird/15.0.1 MIME-Version: 1.0 To: ceph-devel@vger.kernel.org Subject: [PATCH 3/3] libceph: define common queue_con_delay() References: <507497AC.5030609@inktank.com> In-Reply-To: <507497AC.5030609@inktank.com> X-Gm-Message-State: ALoCoQmccHEkOxREPOs5TWpOf09nnAZRGP1NFD+66bF7nflsjJ6+7KP3KlxC7IMNoE+IXZAHthwh Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org This patch defines a single function, queue_con_delay() to call queue_delayed_work() for a connection. It basically generalizes what was previously queue_con() by adding the delay argument. queue_con() is now a simple helper that passes 0 for its delay. queue_con_delay() returns 0 if it queued work or an errno if it did not for some reason. If con_work() finds the BACKOFF flag set for a connection, it now calls queue_con_delay() to handle arranging to start again after a delay. Note about connection reference counts: con_work() only ever gets called as a work item function. At the time that work is scheduled, a reference to the connection is acquired, and the corresponding con_work() call is then responsible for dropping that reference before it returns. Previously, the backoff handling inside con_work() silently handed off its reference to delayed work it scheduled. Now that queue_con_delay() is used, a new reference is acquired for the newly-scheduled work, and the original reference is dropped by the con->ops->put() call at the end of the function. Signed-off-by: Alex Elder Reviewed-by: Sage Weil --- net/ceph/messenger.c | 38 +++++++++++++++++++++++--------------- 1 file changed, 23 insertions(+), 15 deletions(-) /* @@ -2294,14 +2305,11 @@ restart: if (test_and_clear_bit(CON_FLAG_BACKOFF, &con->flags)) { dout("con_work %p backing off\n", con); - if (queue_delayed_work(ceph_msgr_wq, &con->work, - round_jiffies_relative(con->delay))) { - dout("con_work %p backoff %lu\n", con, con->delay); - mutex_unlock(&con->mutex); - return; - } else { + ret = queue_con_delay(con, round_jiffies_relative(con->delay)); + if (ret) { dout("con_work %p FAILED to back off %lu\n", con, con->delay); + BUG_ON(ret == -ENOENT); set_bit(CON_FLAG_BACKOFF, &con->flags); } goto done; diff --git a/net/ceph/messenger.c b/net/ceph/messenger.c index 9170c20..77cc8b1 100644 --- a/net/ceph/messenger.c +++ b/net/ceph/messenger.c @@ -2244,22 +2244,33 @@ bad_tag: /* - * Atomically queue work on a connection. Bump @con reference to - * avoid races with connection teardown. + * Atomically queue work on a connection after the specified delay. + * Bump @con reference to avoid races with connection teardown. + * Returns 0 if work was queued, or an error code otherwise. */ -static void queue_con(struct ceph_connection *con) +static int queue_con_delay(struct ceph_connection *con, unsigned long delay) { if (!con->ops->get(con)) { - dout("queue_con %p ref count 0\n", con); - return; + dout("%s %p ref count 0\n", __func__, con); + + return -ENOENT; } - if (!queue_delayed_work(ceph_msgr_wq, &con->work, 0)) { - dout("queue_con %p - already queued\n", con); + if (!queue_delayed_work(ceph_msgr_wq, &con->work, delay)) { + dout("%s %p - already queued\n", __func__, con); con->ops->put(con); - } else { - dout("queue_con %p\n", con); + + return -EBUSY; } + + dout("%s %p %lu\n", __func__, con, delay); + + return 0; +} + +static void queue_con(struct ceph_connection *con) +{ + (void) queue_con_delay(con, 0); }