From patchwork Wed May 1 21:38:06 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Elder X-Patchwork-Id: 2508971 Return-Path: X-Original-To: patchwork-ceph-devel@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork1.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork1.kernel.org (Postfix) with ESMTP id F32493FD85 for ; Wed, 1 May 2013 21:38:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932943Ab3EAViL (ORCPT ); Wed, 1 May 2013 17:38:11 -0400 Received: from mail-ie0-f177.google.com ([209.85.223.177]:61616 "EHLO mail-ie0-f177.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1760761Ab3EAViJ (ORCPT ); Wed, 1 May 2013 17:38:09 -0400 Received: by mail-ie0-f177.google.com with SMTP id 9so2442638iec.22 for ; Wed, 01 May 2013 14:38:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:subject :references:in-reply-to:content-type:content-transfer-encoding :x-gm-message-state; bh=NHJqKkzrVAQKPWUNNxweSyWINIdV+wQUIXkAD+iWu4M=; b=UXxsQwOY3lbK62ORhyLwCiqexHeLvL/aegc7qCYSFl6Hp8+pyRBYbbY3meoJ6WY5jW gLOmGd5nbTW0xekVMx7f2B9amY+hmvaI7/ohmzzBZ7R4j8+j8loi83CwZEWhogLanQbP 3M+pUaZ27fuCVg12ZKj403/RlUcpDd+PFygXtXk513ZoOWcH+B1s5nC6SJCi9vp+yjDV nqF+Luberx3giaifvQ9GE2x3O+I2jvD7NLLjFQIJdZDgWYSGiWTDJdJk4nrCNX80Zwox MtuRxjyEoKJKIcASkcT/fB/fa0jICrpg2k8+q0ZyDK5OnFLDcqHh+OTJV4TfmejhiiZ9 uJ3g== X-Received: by 10.50.73.133 with SMTP id l5mr2591921igv.108.1367444288745; Wed, 01 May 2013 14:38:08 -0700 (PDT) Received: from [172.22.22.4] (c-71-195-31-37.hsd1.mn.comcast.net. [71.195.31.37]) by mx.google.com with ESMTPSA id dy5sm5801450igc.1.2013.05.01.14.38.07 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 01 May 2013 14:38:07 -0700 (PDT) Message-ID: <51818B3E.8020007@inktank.com> Date: Wed, 01 May 2013 16:38:06 -0500 From: Alex Elder User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130329 Thunderbird/17.0.5 MIME-Version: 1.0 To: ceph-devel@vger.kernel.org Subject: [PATCH 1/3] libceph: allocate ceph messages with a slab allocator References: <51818B12.4010307@inktank.com> In-Reply-To: <51818B12.4010307@inktank.com> X-Gm-Message-State: ALoCoQkTptAqWBvO7EmO4WuiVcTUEG/vkt5+oeuQcn5h9djTIkGd/8fm/A8czhw1ASdwnjX9gyj1 Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org Create a slab cache to manage ceph_msg structure allocation. This is part of: http://tracker.ceph.com/issues/3926 Signed-off-by: Alex Elder --- net/ceph/messenger.c | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) static char tag_ack = CEPH_MSGR_TAG_ACK; @@ -226,6 +230,22 @@ static void encode_my_addr(struct ceph_messenger *msgr) */ static struct workqueue_struct *ceph_msgr_wq; +static int ceph_msgr_slab_init(void) +{ + BUG_ON(ceph_msg_cache); + ceph_msg_cache = kmem_cache_create("ceph_msg", + sizeof (struct ceph_msg), + __alignof__(struct ceph_msg), 0, NULL); + return ceph_msg_cache ? 0 : -ENOMEM; +} + +static void ceph_msgr_slab_exit(void) +{ + BUG_ON(!ceph_msg_cache); + kmem_cache_destroy(ceph_msg_cache); + ceph_msg_cache = NULL; +} + static void _ceph_msgr_exit(void) { if (ceph_msgr_wq) { @@ -233,6 +253,8 @@ static void _ceph_msgr_exit(void) ceph_msgr_wq = NULL; } + ceph_msgr_slab_exit(); + BUG_ON(zero_page == NULL); kunmap(zero_page); page_cache_release(zero_page); @@ -245,6 +267,9 @@ int ceph_msgr_init(void) zero_page = ZERO_PAGE(0); page_cache_get(zero_page); + if (ceph_msgr_slab_init()) + return -ENOMEM; + ceph_msgr_wq = alloc_workqueue("ceph-msgr", WQ_NON_REENTRANT, 0); if (ceph_msgr_wq) return 0; @@ -3068,7 +3093,7 @@ struct ceph_msg *ceph_msg_new(int type, int front_len, gfp_t flags, { struct ceph_msg *m; - m = kzalloc(sizeof(*m), flags); + m = kmem_cache_zalloc(ceph_msg_cache, flags); if (m == NULL) goto out; @@ -3215,7 +3240,7 @@ void ceph_msg_kfree(struct ceph_msg *m) vfree(m->front.iov_base); else kfree(m->front.iov_base); - kfree(m); + kmem_cache_free(ceph_msg_cache, m); } /* diff --git a/net/ceph/messenger.c b/net/ceph/messenger.c index 91dd451..bc1ba4c 100644 --- a/net/ceph/messenger.c +++ b/net/ceph/messenger.c @@ -152,6 +152,10 @@ static bool con_flag_test_and_set(struct ceph_connection *con, return test_and_set_bit(con_flag, &con->flags); } +/* Slab caches for frequently-allocated structures */ + +static struct kmem_cache *ceph_msg_cache; + /* static tag bytes (protocol control messages) */ static char tag_msg = CEPH_MSGR_TAG_MSG;