From patchwork Thu Mar 10 23:32:16 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jim Schutt X-Patchwork-Id: 626391 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by demeter1.kernel.org (8.14.4/8.14.3) with ESMTP id p2ANWP2C007003 for ; Thu, 10 Mar 2011 23:32:27 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753996Ab1CJXcY (ORCPT ); Thu, 10 Mar 2011 18:32:24 -0500 Received: from sentry-three.sandia.gov ([132.175.109.17]:33565 "EHLO sentry-three.sandia.gov" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753788Ab1CJXcY (ORCPT ); Thu, 10 Mar 2011 18:32:24 -0500 X-WSS-ID: 0LHV81U-0C-7WC-02 X-M-MSG: Received: from sentry.sandia.gov (mm03snlnto.sandia.gov [132.175.109.20]) by sentry-three.sandia.gov (Postfix) with ESMTP id 165D654E1D2; Thu, 10 Mar 2011 16:32:18 -0700 (MST) Received: from [134.253.103.2] by sentry.sandia.gov with ESMTP (SMTP Relay 01 (Email Firewall v6.3.2)); Thu, 10 Mar 2011 16:32:16 -0700 X-Server-Uuid: 6BFC7783-7E22-49B4-B610-66D6BE496C0E Received: from [134.253.4.20] (134.253.95.171) by mail.sandia.gov ( 134.253.103.2) with Microsoft SMTP Server (TLS) id 14.1.270.1; Thu, 10 Mar 2011 16:32:16 -0700 Subject: Re: cosd multi-second stalls cause "wrongly marked me down" From: "Jim Schutt" To: "Sage Weil" cc: "Gregory Farnum" , "ceph-devel@vger.kernel.org" In-Reply-To: References: <1297891508.25491.120.camel@sale659.sandia.gov> <75157CFDA63D45458FC47FB7BA6CB974@gmail.com> <1297893011.25491.124.camel@sale659.sandia.gov> <1297957574.25491.152.camel@sale659.sandia.gov> <1297985503.25491.175.camel@sale659.sandia.gov> <1299686572.4750.329.camel@sale659.sandia.gov> <2E7A0FAAE7EA416896A1990C7A96D59A@gmail.com> <1299695809.4750.346.camel@sale659.sandia.gov> <144B52A5BA504DCB992CD253287A397A@gmail.com> <1299798589.4750.372.camel@sale659.sandia.gov> Date: Thu, 10 Mar 2011 16:32:16 -0700 Message-ID: <1299799936.4750.378.camel@sale659.sandia.gov> MIME-Version: 1.0 X-Mailer: Evolution 2.12.3 (2.12.3-19.el5) X-Originating-IP: [134.253.95.171] X-TMWD-Spam-Summary: TS=20110310233217; ID=1; SEV=2.3.1; DFV=B2011031023; IFV=NA; AIF=B2011031023; RPD=5.03.0010; ENG=NA; RPDID=7374723D303030312E30413031303230392E34443739354638312E303036412C73733D312C6667733D30; CAT=NONE; CON=NONE; SIG=AAABAJsKIgAAAAAAAAAAAAAAAAAAAH0= X-MMS-Spam-Filter-ID: B2011031023_5.03.0010 X-WSS-ID: 6167800A2PC1405962-01-01 Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Greylist: IP, sender and recipient auto-whitelisted, not delayed by milter-greylist-4.2.6 (demeter1.kernel.org [140.211.167.41]); Thu, 10 Mar 2011 23:32:30 +0000 (UTC) diff --git a/src/msg/Message.h b/src/msg/Message.h index 3758b1b..ac32b94 100644 --- a/src/msg/Message.h +++ b/src/msg/Message.h @@ -154,8 +154,14 @@ struct RefCountedObject { } void put() { //generic_dout(0) << "RefCountedObject::put " << this << " " << nref.read() << " -> " << (nref.read() - 1) << dendl; - if (nref.dec() == 0) + if (nref.dec() == 0) { + utime_t s = g_clock.now(); delete this; + utime_t e = g_clock.now(); + if (e - s > 0.5) { + generic_dout(1) << "RefCountedObject::put delete " << this << " took " << e - s << " secs!" << dendl; + } + } } }; diff --git a/src/msg/SimpleMessenger.cc b/src/msg/SimpleMessenger.cc index 7df3d44..a86ced8 100644 --- a/src/msg/SimpleMessenger.cc +++ b/src/msg/SimpleMessenger.cc @@ -2243,6 +2243,23 @@ int SimpleMessenger::Pipe::write_message(Message *m) goto out; } +/* Clean up sent list */ +void SimpleMessenger::Pipe::handle_ack(uint64_t seq) +{ + dout(15) << "reader got ack seq " << seq << dendl; + // trim sent list + while (!sent.empty() && + sent.front()->get_seq() <= seq) { + Message *m = sent.front(); + sent.pop_front(); + dout(10) << "reader got ack seq " + << seq << " >= " << m->get_seq() << " on " << m << " " << *m << dendl; + m->put(); + dout(20) << "handle_ack finished put on " << m << dendl; + } +} + + /******************************************** * SimpleMessenger diff --git a/src/msg/SimpleMessenger.h b/src/msg/SimpleMessenger.h index d6ee0df..4031836 100644 --- a/src/msg/SimpleMessenger.h +++ b/src/msg/SimpleMessenger.h @@ -174,20 +174,7 @@ private: void fail(); void was_session_reset(); - - /* Clean up sent list */ - void handle_ack(uint64_t seq) { - dout(15) << "reader got ack seq " << seq << dendl; - // trim sent list - while (!sent.empty() && - sent.front()->get_seq() <= seq) { - Message *m = sent.front(); - sent.pop_front(); - dout(10) << "reader got ack seq " - << seq << " >= " << m->get_seq() << " on " << m << " " << *m << dendl; - m->put(); - } - } + void handle_ack(uint64_t seq); // threads class Reader : public Thread {