From patchwork Tue Oct 9 21:29:46 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Elder X-Patchwork-Id: 1571661 Return-Path: X-Original-To: patchwork-ceph-devel@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork2.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork2.kernel.org (Postfix) with ESMTP id 1C722DFFAD for ; Tue, 9 Oct 2012 21:29:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756886Ab2JIV3u (ORCPT ); Tue, 9 Oct 2012 17:29:50 -0400 Received: from mail-pb0-f46.google.com ([209.85.160.46]:64213 "EHLO mail-pb0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753278Ab2JIV3u (ORCPT ); Tue, 9 Oct 2012 17:29:50 -0400 Received: by mail-pb0-f46.google.com with SMTP id rr4so5688715pbb.19 for ; Tue, 09 Oct 2012 14:29:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding :x-gm-message-state; bh=vlYjVE+kh7y+zL1bNAvoLRrJkmkEDzWD1ht8jIos/QU=; b=FLyvQG2+cC14WmacQnB0M2jnPu4EWQDkxpz6g25Zb+f8Xc9tyyg4ZHELU8YYR/mvQ4 yucKisCtvumqdoXaUkbXAC71WPjM2ha34JB9EJOXP520of6ztgnDs46oYWFWUmSpzQIo ZH2JBe/M17ttX21iasX/mgAyj4gvPah+C1XnPfdG0te+Gto6BbYGuAtuEohfyMTaGpUq rui0rS848qChlo1b3Q7zy0K183xxVwiouGJtm6ehagPe47brzUIoMhdMbD8Ih29KOQQT pyNR5k3s94HlHPJkGav9jNOSzOIMuQRDoUMtr/PQ+t8gt1eAjHitWzd0NF5hzJKEVhOv GxFg== Received: by 10.68.131.40 with SMTP id oj8mr34240345pbb.40.1349818189659; Tue, 09 Oct 2012 14:29:49 -0700 (PDT) Received: from ?IPv6:2607:f298:a:607:4823:f93f:93c7:61da? ([2607:f298:a:607:4823:f93f:93c7:61da]) by mx.google.com with ESMTPS id bm8sm13135954pab.3.2012.10.09.14.29.46 (version=SSLv3 cipher=OTHER); Tue, 09 Oct 2012 14:29:48 -0700 (PDT) Message-ID: <5074974A.3010000@inktank.com> Date: Tue, 09 Oct 2012 14:29:46 -0700 From: Alex Elder User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20120912 Thunderbird/15.0.1 MIME-Version: 1.0 To: ceph-devel@vger.kernel.org Subject: [PATCH 2/2] libceph: mark ceph_msgr_wq unbounded References: <50749701.8010208@inktank.com> In-Reply-To: <50749701.8010208@inktank.com> X-Gm-Message-State: ALoCoQnbzs7KFCLL2Z9C6LwpMitvc/uRJJOOm2ItwH3Z5uNYTlDGzBzxtcuY3vAap2Sp+S9SF1gr Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org Andreas Bluemle observed an undesirable behavior in the ceph messenger when using networking hardware that binds received data handling to a single CPU. The messenger uses a workqueue to process message activity, and in this case all work was being queued on the same per-cpu workqueue. Consequently that CPU becomes a bottleneck, forced to process all incoming messages. Marking the workqueue UNBOUND allows any available CPU to process a work item on the queue. The concurrency of the workqueue will be limited by the number of active connections. Reported-by: Andreas Bluemle Signed-off-by: Alex Elder --- net/ceph/messenger.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/net/ceph/messenger.c b/net/ceph/messenger.c index 28ed904..f9f65fe 100644 --- a/net/ceph/messenger.c +++ b/net/ceph/messenger.c @@ -190,7 +190,7 @@ int ceph_msgr_init(void) zero_page = ZERO_PAGE(0); page_cache_get(zero_page); - ceph_msgr_wq = alloc_workqueue("ceph-msgr", 0, 0); + ceph_msgr_wq = alloc_workqueue("ceph-msgr", WQ_UNBOUND, 0); if (ceph_msgr_wq) return 0;