From patchwork Mon Dec 23 17:09:44 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Dryomov X-Patchwork-Id: 3397511 Return-Path: X-Original-To: patchwork-ceph-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 3CF579F37A for ; Mon, 23 Dec 2013 17:11:20 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 1622020672 for ; Mon, 23 Dec 2013 17:11:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id DF73B20688 for ; Mon, 23 Dec 2013 17:11:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757673Ab3LWRLG (ORCPT ); Mon, 23 Dec 2013 12:11:06 -0500 Received: from mail-ea0-f178.google.com ([209.85.215.178]:51576 "EHLO mail-ea0-f178.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757648Ab3LWRLF (ORCPT ); Mon, 23 Dec 2013 12:11:05 -0500 Received: by mail-ea0-f178.google.com with SMTP id d10so2511256eaj.9 for ; Mon, 23 Dec 2013 09:11:04 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=rBGf1AT4MZmtRFRKBe3qWveHlHSZDIQOuFlpc7tNHEM=; b=RUDHu+TtHyZfZbIaj/Oh8eq36PbodhrX5qB7OfjqI+udHyMV42Rqted+eQc2aZcFEz I1X42cReLzWqqQXFA4jcy4uimsDwZXjI/xPzVIRlkj/AmG/57iaWxZ/36k/jTU8Jm5Gy /aXGqglq0wP+GlxRfZTidasR1Hwjqkprev8aWx5avHmaFllgYt3zfLzqSuRKpSEkTcKz Fduh4x0OLRAQ1fMgNTsMsJpk9KA/Hp8h8KW8VrTqTGyIqrD+Ssy4ov6wx5n+QE+Xnsev iHqz0ZCJCo4M0/ygHmlLL7BIYFHVS4NrUPbWif5qm7+kpk0DLZKcbIV5iWx59hgBkZTL NapQ== X-Gm-Message-State: ALoCoQkPLyVmDY1km6OjKPyep7INl4PaDGvnMnGFacheYgqxDwVzvzhjg9jjRVHUu7it3oxQVJDf X-Received: by 10.14.53.14 with SMTP id f14mr527282eec.105.1387818664344; Mon, 23 Dec 2013 09:11:04 -0800 (PST) Received: from localhost ([109.110.66.29]) by mx.google.com with ESMTPSA id o1sm47321261eea.10.2013.12.23.09.11.03 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Mon, 23 Dec 2013 09:11:03 -0800 (PST) From: Ilya Dryomov To: ceph-devel@vger.kernel.org Cc: Sage Weil , ilya.dryomov@inktank.com Subject: [PATCH 13/19] crush: apply chooseleaf_tries to firstn mode too Date: Mon, 23 Dec 2013 19:09:44 +0200 Message-Id: <1387818590-30483-14-git-send-email-ilya.dryomov@inktank.com> X-Mailer: git-send-email 1.7.10.4 In-Reply-To: <1387818590-30483-1-git-send-email-ilya.dryomov@inktank.com> References: <1387818590-30483-1-git-send-email-ilya.dryomov@inktank.com> Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Spam-Status: No, score=-7.4 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Parameterize the attempts for the _firstn choose method, and apply the rule-specified tries count to firstn mode as well. Note that we have slightly different behavior here than with indep: If the firstn value is not specified for firstn, we pass through the normal attempt count. This maintains compatibility with legacy behavior. Note that this is usually *not* actually N^2 work, though, because of the descend_once tunable. However, descend_once is unfortunately *not* the same thing as 1 chooseleaf try because it is only checked on a reject but not on a collision. Sigh. In contrast, for indep, if tries is not specified we default to 1 recursive attempt, because that is simply more sane, and we have the option to do so. The descend_once tunable has no effect for indep. Reflects ceph.git commit 64aeded50d80942d66a5ec7b604ff2fcbf5d7b63. Signed-off-by: Ilya Dryomov --- include/linux/crush/crush.h | 5 ++++- net/ceph/crush/mapper.c | 14 ++++++++++---- 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/include/linux/crush/crush.h b/include/linux/crush/crush.h index 2e50bab91655..07b8fd4f81fc 100644 --- a/include/linux/crush/crush.h +++ b/include/linux/crush/crush.h @@ -165,7 +165,10 @@ struct crush_map { __u32 choose_local_fallback_tries; /* choose attempts before giving up */ __u32 choose_total_tries; - /* attempt chooseleaf inner descent once; on failure retry outer descent */ + /* attempt chooseleaf inner descent once for firstn mode; on + * reject retry outer descent. Note that this does *not* + * apply to a collision: in that case we will retry as we used + * to. */ __u32 chooseleaf_descend_once; }; diff --git a/net/ceph/crush/mapper.c b/net/ceph/crush/mapper.c index e3ade074541c..c34320518c8b 100644 --- a/net/ceph/crush/mapper.c +++ b/net/ceph/crush/mapper.c @@ -299,6 +299,8 @@ static int crush_choose_firstn(const struct crush_map *map, const __u32 *weight, int weight_max, int x, int numrep, int type, int *out, int outpos, + unsigned int attempts, + unsigned int recurse_attempts, int recurse_to_leaf, int descend_once, int *out2) { @@ -385,6 +387,7 @@ static int crush_choose_firstn(const struct crush_map *map, weight, weight_max, x, outpos+1, 0, out2, outpos, + recurse_attempts, 0, 0, map->chooseleaf_descend_once, NULL) <= outpos) @@ -421,7 +424,7 @@ reject: flocal <= in->size + map->choose_local_fallback_tries) /* exhaustive bucket search */ retry_bucket = 1; - else if (ftotal <= map->choose_total_tries) + else if (ftotal <= attempts) /* then retry descent */ retry_descent = 1; else @@ -634,7 +637,8 @@ int crush_do_rule(const struct crush_map *map, __u32 step; int i, j; int numrep; - int choose_leaf_tries = 1; + int choose_tries = map->choose_total_tries; + int choose_leaf_tries = 0; const int descend_once = 0; if ((__u32)ruleno >= map->max_rules) { @@ -701,6 +705,8 @@ int crush_do_rule(const struct crush_map *map, x, numrep, curstep->arg2, o+osize, j, + choose_tries, + choose_leaf_tries ? choose_leaf_tries : choose_tries, recurse_to_leaf, descend_once, c+osize); } else { @@ -711,8 +717,8 @@ int crush_do_rule(const struct crush_map *map, x, numrep, numrep, curstep->arg2, o+osize, j, - map->choose_total_tries, - choose_leaf_tries, + choose_tries, + choose_leaf_tries ? choose_leaf_tries : 1, recurse_to_leaf, c+osize, 0);