From patchwork Mon Aug 19 17:10:11 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Hefty, Sean" X-Patchwork-Id: 2846635 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 045779F239 for ; Mon, 19 Aug 2013 17:10:20 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id DC4D3201CE for ; Mon, 19 Aug 2013 17:10:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0140D201B3 for ; Mon, 19 Aug 2013 17:10:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750843Ab3HSRKP (ORCPT ); Mon, 19 Aug 2013 13:10:15 -0400 Received: from mga14.intel.com ([143.182.124.37]:49200 "EHLO mga14.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750826Ab3HSRKO convert rfc822-to-8bit (ORCPT ); Mon, 19 Aug 2013 13:10:14 -0400 Received: from azsmga001.ch.intel.com ([10.2.17.19]) by azsmga102.ch.intel.com with ESMTP; 19 Aug 2013 10:10:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.89,914,1367996400"; d="scan'208";a="348465140" Received: from orsmsx108.amr.corp.intel.com ([10.22.240.6]) by azsmga001.ch.intel.com with ESMTP; 19 Aug 2013 10:10:12 -0700 Received: from orsmsx157.amr.corp.intel.com (10.22.240.23) by ORSMSX108.amr.corp.intel.com (10.22.240.6) with Microsoft SMTP Server (TLS) id 14.3.123.3; Mon, 19 Aug 2013 10:10:11 -0700 Received: from orsmsx109.amr.corp.intel.com ([169.254.2.134]) by ORSMSX157.amr.corp.intel.com ([169.254.9.234]) with mapi id 14.03.0123.003; Mon, 19 Aug 2013 10:10:11 -0700 From: "Hefty, Sean" To: Andreas Bluemle CC: "Atchley, Scott" , Matthew Anderson , "ceph-devel@vger.kernel.org" , "linux-rdma@vger.kernel.org (linux-rdma@vger.kernel.org)" Subject: RE: [ceph-users] Help needed porting Ceph to RSockets Thread-Topic: [ceph-users] Help needed porting Ceph to RSockets Thread-Index: Ac6YMmAasET08DnLQxSx5FaUT5h2pAAOrRkgACMebAABAUwhIA== Date: Mon, 19 Aug 2013 17:10:11 +0000 Message-ID: <1828884A29C6694DAF28B7E6B8A8237388CA6C27@ORSMSX109.amr.corp.intel.com> References: <20130812075513.43c338e1@andylap> <20130812180644.447ca089@andylap> <20130813075312.7cac0d46@andylap> <20130813160612.037ea9f2@andylap> <1978D1F9-C675-4A37-AA57-C7E1158B2F72@ornl.gov> <1828884A29C6694DAF28B7E6B8A8237388CA5A0F@ORSMSX109.amr.corp.intel.com> <20130814092124.3f8c3b03@doppio> In-Reply-To: <20130814092124.3f8c3b03@doppio> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.22.254.140] MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-9.7 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Can you see if the patch below fixes the hang? Signed-off-by: Sean Hefty --- src/rsocket.c | 11 ++++++++++- 1 files changed, 10 insertions(+), 1 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/src/rsocket.c b/src/rsocket.c index d544dd0..e45b26d 100644 --- a/src/rsocket.c +++ b/src/rsocket.c @@ -2948,10 +2948,12 @@ static int rs_poll_events(struct pollfd *rfds, struct pollfd *fds, nfds_t nfds) rs = idm_lookup(&idm, fds[i].fd); if (rs) { + fastlock_acquire(&rs->cq_wait_lock); if (rs->type == SOCK_STREAM) rs_get_cq_event(rs); else ds_get_cq_event(rs); + fastlock_release(&rs->cq_wait_lock); fds[i].revents = rs_poll_rs(rs, fds[i].events, 1, rs_poll_all); } else { fds[i].revents = rfds[i].revents; @@ -3098,7 +3100,8 @@ int rselect(int nfds, fd_set *readfds, fd_set *writefds, /* * For graceful disconnect, notify the remote side that we're - * disconnecting and wait until all outstanding sends complete. + * disconnecting and wait until all outstanding sends complete, provided + * that the remote side has not sent a disconnect message. */ int rshutdown(int socket, int how) { @@ -3138,6 +3141,12 @@ int rshutdown(int socket, int how) if (rs->state & rs_connected) rs_process_cq(rs, 0, rs_conn_all_sends_done); + if (rs->state & rs_disconnected) { + /* Generate event by flushing receives to unblock rpoll */ + ibv_req_notify_cq(rs->cm_id->recv_cq, 0); + rdma_disconnect(rs->cm_id); + } + if ((rs->fd_flags & O_NONBLOCK) && (rs->state & rs_connected)) rs_set_nonblocking(rs, rs->fd_flags);