From patchwork Tue Mar 26 14:50:59 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Elder X-Patchwork-Id: 2337791 Return-Path: X-Original-To: patchwork-ceph-devel@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork1.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork1.kernel.org (Postfix) with ESMTP id 245D63FC54 for ; Tue, 26 Mar 2013 14:51:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934525Ab3CZOvD (ORCPT ); Tue, 26 Mar 2013 10:51:03 -0400 Received: from mail-ia0-f170.google.com ([209.85.210.170]:38864 "EHLO mail-ia0-f170.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933525Ab3CZOvB (ORCPT ); Tue, 26 Mar 2013 10:51:01 -0400 Received: by mail-ia0-f170.google.com with SMTP id h8so6529889iaa.1 for ; Tue, 26 Mar 2013 07:51:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:subject :references:in-reply-to:content-type:content-transfer-encoding :x-gm-message-state; bh=BKj73Z3yxMj1QB+qwgfWEvzfPDy7B/mJPLwYMhWWjDs=; b=SgRLqVSWJz7r8YpXRySml5H7TUwSGCbgUfTLOzo0nVpLriuIHfUPJ2k9lmp0u1iUiv s0vEBj6AjbL+XW6voRZrI3fcEVk99zt+ohPLUkoHQA/3YbK3fUekfCafduWqk95DHJcr n5CMIMeFZw/OYoogvclW6wB8vnfW/LHwYwXhVgDy+6nDlW7cbE5KNPt7XRbK91Xb9+12 mTfDF9nEUogJhtmsG/NWzytDwKotKX61xKoefwhW8xCMJPzalivvvq0O/nVRNe473rz1 56GXqKybAjxLr5RgBXdtuuJunbfaxboFbKousURRmSAoGg8roVMW8JpolaCWwQptbQ7z iSDg== X-Received: by 10.43.91.5 with SMTP id bk5mr9380085icc.20.1364309461408; Tue, 26 Mar 2013 07:51:01 -0700 (PDT) Received: from [172.22.22.4] (c-71-195-31-37.hsd1.mn.comcast.net. [71.195.31.37]) by mx.google.com with ESMTPS id qn10sm2400431igc.6.2013.03.26.07.51.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 26 Mar 2013 07:51:00 -0700 (PDT) Message-ID: <5151B5D3.5040802@inktank.com> Date: Tue, 26 Mar 2013 09:50:59 -0500 From: Alex Elder User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130308 Thunderbird/17.0.4 MIME-Version: 1.0 To: ceph-devel@vger.kernel.org Subject: [PATCH 6/6, v2] libceph: verify requests queued in order References: <5151071C.3000309@inktank.com> <515107C9.2000906@inktank.com> In-Reply-To: <515107C9.2000906@inktank.com> X-Gm-Message-State: ALoCoQm2l0eX4xguzWMczaMWhklUdQzDn28hLN0KugcKZuTB1CeDoXG0oYdM8n1WaNP5zTMSPbG/ Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org Add checking to verify all osd requests for an osd are added to the queue in order of increasing tid. Signed-off-by: Alex Elder --- v2: Only check the last of the requests queued to resend. net/ceph/osd_client.c | 44 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 43 insertions(+), 1 deletion(-) * Linger requests are re-registered before sending, which @@ -618,6 +658,7 @@ static void __kick_osd_requests(struct ceph_osd_client *osdc, */ BUG_ON(!list_empty(&req->r_req_lru_item)); __register_request(osdc, req); + osdc_unsent_check(osdc, req, false); list_add_tail(&req->r_req_lru_item, &osdc->req_unsent); list_add_tail(&req->r_osd_item, &req->r_osd->o_requests); __unregister_linger_request(osdc, req); @@ -1036,6 +1077,7 @@ static int __map_request(struct ceph_osd_client *osdc, if (req->r_osd) { __remove_osd_from_lru(req->r_osd); list_add_tail(&req->r_osd_item, &req->r_osd->o_requests); + osdc_unsent_check(osdc, req, false); list_move_tail(&req->r_req_lru_item, &osdc->req_unsent); } else { list_move_tail(&req->r_req_lru_item, &osdc->req_notarget); diff --git a/net/ceph/osd_client.c b/net/ceph/osd_client.c index 3b6657f..943400c 100644 --- a/net/ceph/osd_client.c +++ b/net/ceph/osd_client.c @@ -563,6 +563,41 @@ __lookup_request_ge(struct ceph_osd_client *osdc, return NULL; } +#define list_last_entry(ptr, type, member) \ + list_entry((ptr)->prev, type, member) + +/* + * Returns true if it's OK to move the given request to the + * the osd client's unsent list. Called before moving the + * request to the beginning of the list (prepend == true) or + * to the end (prepend == * false). + */ +static bool osdc_unsent_check(struct ceph_osd_client *osdc, + struct ceph_osd_request *req, + bool prepend) +{ + struct ceph_osd_request *list_req; + int bad; + + if (list_empty(&osdc->req_unsent)) + return true; + + if (prepend) { + list_req = list_first_entry(&osdc->req_unsent, + struct ceph_osd_request, + r_req_lru_item); + bad = WARN_ON(req->r_tid > list_req->r_tid); + } else { + list_req = list_last_entry(&osdc->req_unsent, + struct ceph_osd_request, + r_req_lru_item); + bad = WARN_ON(req->r_tid < list_req->r_tid); + } + bad = bad || WARN_ON(req->r_tid == list_req->r_tid); + + return !bad; +} + /* * Resubmit requests pending on the given osd. */ @@ -603,7 +638,12 @@ static void __kick_osd_requests(struct ceph_osd_client *osdc, if (!req->r_linger) req->r_flags |= CEPH_OSD_FLAG_RETRY; } - list_splice(&resend, &osdc->req_unsent); + if (!list_empty(&resend)) { + req = list_last_entry(&resend, struct ceph_osd_request, + r_req_lru_item); + osdc_unsent_check(osdc, req, true); + list_splice(&resend, &osdc->req_unsent); + } /*