From patchwork Sat May 11 17:37:00 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Elder X-Patchwork-Id: 2554231 Return-Path: X-Original-To: patchwork-ceph-devel@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork1.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork1.kernel.org (Postfix) with ESMTP id 42C0C3FC5A for ; Sat, 11 May 2013 17:37:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753926Ab3EKRhD (ORCPT ); Sat, 11 May 2013 13:37:03 -0400 Received: from mail-ie0-f173.google.com ([209.85.223.173]:60229 "EHLO mail-ie0-f173.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753777Ab3EKRhD (ORCPT ); Sat, 11 May 2013 13:37:03 -0400 Received: by mail-ie0-f173.google.com with SMTP id k5so10047305iea.18 for ; Sat, 11 May 2013 10:37:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding:x-gm-message-state; bh=pHMWFNTO5w+Dzo2q//k9+3vdC9u13fXqhd1UzrYymjs=; b=ZREgw32oNgssMl1KU3T7Zhd7hHdl2qUnxv2btVvkDBvCx4TIpwpsMYGb7bNUBLPCPn EblqOoeL5Tmh/G6CoZZ2dDdR7CI7gv0Mg2cqkn3EskLnhd6W/29wSG94ejjFJfUR4wC2 FclAX3y2PBjpTHs3m9TFOON+9i5VPDCjFfOQbKYPHy9X014NJ19uKoeiyKfetlFG/Zal E56TFUUqP5V02KkxESAlrnGg/vvZKgoUF/ZMGp6XKNvyHhAVDbZ3nZwLNe1JDd8mFsDN rsEQY/Hmbo+lqVpytBg486Qrjqal0dPIka+F2ygjd7d2G5DuKIhc7ZxEeohOUEzTBqlP 9gyQ== X-Received: by 10.50.7.69 with SMTP id h5mr5639961iga.69.1368293821780; Sat, 11 May 2013 10:37:01 -0700 (PDT) Received: from [172.22.22.4] (c-71-195-31-37.hsd1.mn.comcast.net. [71.195.31.37]) by mx.google.com with ESMTPSA id xf4sm5593890igb.8.2013.05.11.10.37.00 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 11 May 2013 10:37:01 -0700 (PDT) Message-ID: <518E81BC.5020702@inktank.com> Date: Sat, 11 May 2013 12:37:00 -0500 From: Alex Elder User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130329 Thunderbird/17.0.5 MIME-Version: 1.0 To: ceph-devel@vger.kernel.org Subject: [PATCH] libceph: init sent and completed when starting X-Gm-Message-State: ALoCoQkZZNgeZl+y1d/LyLV2S7vX1dpSHGEu86urCEcIFnduCu/Rw0vXcvJinA8z6sznLSrKkd+B Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org The rbd code has a need to be able to restart an osd request that has already been started and completed once before. This currently wouldn't work right because the osd client code assumes an osd request will be started exactly once Certain fields in a request are never cleared and this leads to trouble if you try to reuse it. Specifically, the r_sent, r_got_reply, and r_completed fields are never cleared. The r_sent field records the osd incarnation at the time the request was sent to that osd. If that's non-zero, the message won't get re-mapped to a target osd properly, and won't be put on the unsafe requests list the first time it's sent as it should. The r_got_reply field is used in handle_reply() to ensure the reply to a request is processed only once. And the r_completed field is used for lingering requests to avoid calling the callback function every time the osd client re-sends the request on behalf of its initiator. Each osd request passes through ceph_osdc_start_request() when responsibility for the request is handed over to the osd client for completion. We can safely zero these three fields there each time a request gets started. One last related change--clear the r_linger flag when a request is no longer registered as a linger request. This resolves: http://tracker.ceph.com/issues/5026 Signed-off-by: Alex Elder Reviewed-by: Josh Durgin --- net/ceph/osd_client.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) mutex_unlock(&osdc->request_mutex); @@ -2120,7 +2121,9 @@ int ceph_osdc_start_request(struct ceph_osd_client *osdc, down_read(&osdc->map_sem); mutex_lock(&osdc->request_mutex); __register_request(osdc, req); - WARN_ON(req->r_sent); + req->r_sent = 0; + req->r_got_reply = 0; + req->r_completed = 0; rc = __map_request(osdc, req, 0); if (rc < 0) { if (nofail) { diff --git a/net/ceph/osd_client.c b/net/ceph/osd_client.c index a3395fd..d5953b8 100644 --- a/net/ceph/osd_client.c +++ b/net/ceph/osd_client.c @@ -1204,6 +1204,7 @@ void ceph_osdc_unregister_linger_request(struct ceph_osd_client *osdc, mutex_lock(&osdc->request_mutex); if (req->r_linger) { __unregister_linger_request(osdc, req); + req->r_linger = 0; ceph_osdc_put_request(req); }