From patchwork Wed Aug 15 06:34:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 10566305 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7EB6913B4 for ; Wed, 15 Aug 2018 06:35:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6DB2029E56 for ; Wed, 15 Aug 2018 06:35:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6087929E6E; Wed, 15 Aug 2018 06:35:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C10EB29E56 for ; Wed, 15 Aug 2018 06:35:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B09AC6B0003; Wed, 15 Aug 2018 02:35:02 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AB8F96B0005; Wed, 15 Aug 2018 02:35:02 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9A85C6B0007; Wed, 15 Aug 2018 02:35:02 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by kanga.kvack.org (Postfix) with ESMTP id 3AA566B0003 for ; Wed, 15 Aug 2018 02:35:02 -0400 (EDT) Received: by mail-ed1-f71.google.com with SMTP id l16-v6so209108edq.18 for ; Tue, 14 Aug 2018 23:35:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id; bh=5m7m6nvFurxwpxSw/fGUFBF7pslYseBsq43Y9+UiQHs=; b=nN3poCDRCrJpPc5Yd6duUhIjGfvFy37wNGRsh/EhdoYB3H5vyIEhZ7fDbahSgc7d0L HFBDtKjR1gYcW0SBPaPYTJsfyeymh3bbJ8DxvJgVpcd5RQvyrxd6eu495+2J/jgVVZ8C 02ko5oynACjPzbkPMuT9hR78jYznnWpuXzlUMUO0nKutoZbK4Bk6kEIcwiAdrL3u7uHz hCuN5V/XyZlzmaZaS8E9W3y0BGmIlRnqx/MjNLPZa3VzP8hHmqsGZ6yiPCGtHlrzuSSn KnGdJKtdcVYxhkRAZu2BnOqrlfzCN+R/8mvaUHib0OcUzmTF0WsAdB/HKjyhYa2rrZnV Q+xw== X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 148.163.158.5 is neither permitted nor denied by best guess record for domain of rppt@linux.vnet.ibm.com) smtp.mailfrom=rppt@linux.vnet.ibm.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ibm.com X-Gm-Message-State: AOUpUlF0RW+IqCRuECL9xHENojFwUtsdMh0Oku0QFvAADzGiv3nsb7iA dRnf2JH5g6ucLAggbApbLUCmeb5HgUrv896iiY4qTLJeHNpuZh1/hITnLh1Q5do7difRCcXSYFc KyZew26OK2fG3A6I59krrbjQI973AKt7Oxqa9mXC5UVM+80zalnqxTBTeRcOOlio= X-Received: by 2002:a50:aa43:: with SMTP id p3-v6mr31094000edc.233.1534314901543; Tue, 14 Aug 2018 23:35:01 -0700 (PDT) X-Google-Smtp-Source: AA+uWPz8AM6RGpWaJEIMoc7J6k7fGElgv/6qfnSaEj09fX7PTzkY+TQ2Zk4hEuB+iRUc8M0ISuOk X-Received: by 2002:a50:aa43:: with SMTP id p3-v6mr31093918edc.233.1534314900113; Tue, 14 Aug 2018 23:35:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534314900; cv=none; d=google.com; s=arc-20160816; b=dThiEDIJPXEu0+BBPQcHOTKa9L6xtVU/nEV4haCJVsNDIl/E6J9BCqUO7E2IbRWL+3 7brjEq/y1CuWhX5TuEH7H4AOztH0L+XZSZtmFpChuA+PSQfv6IR7Nm+G0GKSf2x/WuHN Ux0jP6VrI7aqD0DFYgszGXVd9k32gaH9EXgtUjCIE6M8XQak6ZqVO9qirtRUZbrprFJP tI2oojsYi3CuPFlh7ecucr6GeScwJRU+ztLlPsFf9wRJpqcxp445acCwwIK1KezlXpNf /igrnDjqNSj3ChdOQdgvN5gUBE/3uv4NsG67u/z0O0rFisgTXK/7g8UgME4+R28joYJT M+Rg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:date:subject:cc:to:from:arc-authentication-results; bh=5m7m6nvFurxwpxSw/fGUFBF7pslYseBsq43Y9+UiQHs=; b=dNfQZyuA5YwKnQ1O83M3+Ofh9Bdbn4KkimphHp7zk9/QqEjMogENeMd+oOq99rTu+m qKQZWK73A8WGgtzyA+aGTLCpoHjSkn/zgVfx9S9zSiCb1mZEieMwKuGVJf/XQMwmkba4 BaqInhjSc3YVfd6lb3u63c4t31z2ITZsBo54QYkxoVQRIzM51ijfPO2P1uSmCorcDdcy whD5bf6Bt98tzrgDqZRs3vCXyQNpW1J4a2wBckh9OKPT7lA9YN8TjATrU7i/tjDjshn0 cergshf5TXZe+mhNh175XwFhYs4/0njBkl1RFSpX+gsw16yX4ZoGxrHCylBSXlT4My0J yGNQ== ARC-Authentication-Results: i=1; mx.google.com; spf=neutral (google.com: 148.163.158.5 is neither permitted nor denied by best guess record for domain of rppt@linux.vnet.ibm.com) smtp.mailfrom=rppt@linux.vnet.ibm.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com. [148.163.158.5]) by mx.google.com with ESMTPS id b50-v6si5523927eda.285.2018.08.14.23.34.59 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 14 Aug 2018 23:34:59 -0700 (PDT) Received-SPF: neutral (google.com: 148.163.158.5 is neither permitted nor denied by best guess record for domain of rppt@linux.vnet.ibm.com) client-ip=148.163.158.5; Authentication-Results: mx.google.com; spf=neutral (google.com: 148.163.158.5 is neither permitted nor denied by best guess record for domain of rppt@linux.vnet.ibm.com) smtp.mailfrom=rppt@linux.vnet.ibm.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w7F6Y5E8039640 for ; Wed, 15 Aug 2018 02:34:58 -0400 Received: from e06smtp03.uk.ibm.com (e06smtp03.uk.ibm.com [195.75.94.99]) by mx0b-001b2d01.pphosted.com with ESMTP id 2kve13aa9m-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 15 Aug 2018 02:34:58 -0400 Received: from localhost by e06smtp03.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 15 Aug 2018 07:34:56 +0100 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp03.uk.ibm.com (192.168.101.133) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 15 Aug 2018 07:34:52 +0100 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w7F6Yp9V45285386 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 15 Aug 2018 06:34:51 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DA67E4C040; Wed, 15 Aug 2018 09:34:56 +0100 (BST) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 381E74C04E; Wed, 15 Aug 2018 09:34:55 +0100 (BST) Received: from rapoport-lnx (unknown [9.148.8.197]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTPS; Wed, 15 Aug 2018 09:34:55 +0100 (BST) Received: by rapoport-lnx (sSMTP sendmail emulation); Wed, 15 Aug 2018 09:34:49 +0300 From: Mike Rapoport To: linux-mm@kvack.org Cc: Jonathan Corbet , Matthew Wilcox , Michal Hocko , Vlastimil Babka , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Hi@d06av22.portsmouth.uk.ibm.com, Mike Rapoport Subject: Date: Wed, 15 Aug 2018 09:34:47 +0300 X-Mailer: git-send-email 2.7.4 X-TM-AS-GCONF: 00 x-cbid: 18081506-0012-0000-0000-00000299FEB5 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18081506-0013-0000-0000-000020CD23B6 Message-Id: <1534314887-9202-1-git-send-email-rppt@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2018-08-15_03:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1808150071 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP As Vlastimil mentioned at [1], it would be nice to have some guide about memory allocation. I've drafted an initial version that tries to summarize "best practices" for allocation functions and GFP usage. [1] https://www.spinics.net/lists/netfilter-devel/msg55542.html From 8027c0d4b750b8dbd687234feda63305d0d5a057 Mon Sep 17 00:00:00 2001 From: Mike Rapoport Date: Wed, 15 Aug 2018 09:10:06 +0300 Subject: [RFC PATCH] docs/core-api: add memory allocation guide Signed-off-by: Mike Rapoport Acked-by: Michal Hocko --- Documentation/core-api/gfp_mask-from-fs-io.rst | 2 + Documentation/core-api/index.rst | 1 + Documentation/core-api/memory-allocation.rst | 117 +++++++++++++++++++++++++ Documentation/core-api/mm-api.rst | 2 + 4 files changed, 122 insertions(+) create mode 100644 Documentation/core-api/memory-allocation.rst diff --git a/Documentation/core-api/gfp_mask-from-fs-io.rst b/Documentation/core-api/gfp_mask-from-fs-io.rst index e0df8f4..e7c32a8 100644 --- a/Documentation/core-api/gfp_mask-from-fs-io.rst +++ b/Documentation/core-api/gfp_mask-from-fs-io.rst @@ -1,3 +1,5 @@ +.. _gfp_mask_from_fs_io: + ================================= GFP masks used from FS/IO context ================================= diff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst index cdc2020..8afc0da 100644 --- a/Documentation/core-api/index.rst +++ b/Documentation/core-api/index.rst @@ -27,6 +27,7 @@ Core utilities errseq printk-formats circular-buffers + memory-allocation mm-api gfp_mask-from-fs-io timekeeping diff --git a/Documentation/core-api/memory-allocation.rst b/Documentation/core-api/memory-allocation.rst new file mode 100644 index 0000000..b1f2ad5 --- /dev/null +++ b/Documentation/core-api/memory-allocation.rst @@ -0,0 +1,117 @@ +======================= +Memory Allocation Guide +======================= + +Linux supplies variety of APIs for memory allocation. You can allocate +small chunks using `kmalloc` or `kmem_cache_alloc` families, large +virtually contiguous areas using `vmalloc` and it's derivatives, or +you can directly request pages from the page allocator with +`__get_free_pages`. It is also possible to use more specialized +allocators, for instance `cma_alloc` or `zs_malloc`. + +Most of the memory allocations APIs use GFP flags to express how that +memory should be allocated. The GFP acronym stands for "get free +pages", the underlying memory allocation function. + +Diversity of the allocation APIs combined with the numerous GFP flags +makes the question "How should I allocate memory?" not that easy to +answer, although very likely you should use + +:: + + kzalloc(, GFP_KERNEL); + +Of course there are cases when other allocation APIs and different GFP +flags must be used. + +Get Free Page flags +=================== + +The GFP flags control the allocators behavior. They tell what memory +zones can be used, how hard the allocator should try to find a free +memory, whether the memory can be accessed by the userspace etc. The +:ref:`Documentation/core-api/mm-api.rst ` provides +reference documentation for the GFP flags and their combinations and +here we briefly outline their recommended usage: + + * Most of the times ``GFP_KERNEL`` is what you need. Memory for the + kernel data structures, DMAable memory, inode cache, all these and + many other allocations types can use ``GFP_KERNEL``. Note, that + using ``GFP_KERNEL`` implies ``GFP_RECLAIM``, which means that + direct reclaim may be triggered under memory pressure; the calling + context must be allowed to sleep. + * If the allocation is performed from an atomic context, e.g + interrupt handler, use ``GFP_ATOMIC``. + * Untrusted allocations triggered from userspace should be a subject + of kmem accounting and must have ``__GFP_ACCOUNT`` bit set. There + is handy ``GFP_KERNEL_ACCOUNT`` shortcut for ``GFP_KERNEL`` + allocations that should be accounted. + * Userspace allocations should use either of the ``GFP_USER``, + ``GFP_HIGHUSER`` and ``GFP_HIGHUSER_MOVABLE`` flags. The longer + the flag name the less restrictive it is. + + The ``GFP_HIGHUSER_MOVABLE`` does not require that allocated + memory will be directly accessible by the kernel or the hardware + and implies that the data may move. + + The ``GFP_HIGHUSER`` means that the allocated memory is not + movable, but it is not required to be directly accessible by the + kernel or the hardware. An example may be a hardware allocation + that maps data directly into userspace but has no addressing + limitations. + + The ``GFP_USER`` means that the allocated memory is not movable + and it must be directly accessible by the kernel or the + hardware. It is typically used by hardware for buffers that are + mapped to userspace (e.g. graphics) that hardware still must DMA + to. + +You may notice that quite a few allocations in the existing code +specify ``GFP_NOIO`` and ``GFP_NOFS``. Historically, they were used to +prevent recursion deadlocks caused by direct memory reclaim calling +back into the FS or IO paths and blocking on already held +resources. Since 4.12 the preferred way to address this issue is to +use new scope APIs described in +:ref:`Documentation/core-api/gfp_mask-from-fs-io.rst `. + +Another legacy GFP flags are ``GFP_DMA`` and ``GFP_DMA32``. They are +used to ensure that the allocated memory is accessible by hardware +with limited addressing capabilities. So unless you are writing a +driver for a device with such restrictions, avoid using these flags. + +Selecting memory allocator +========================== + +The most straightforward way to allocate memory is to use a function +from the `kmalloc` family. And, to be on the safe size it's best to +use routines that set memory to zero, like `kzalloc`. If you need to +allocate memory for an array, there are `kmalloc_array` and `kcalloc` +helpers. + +The maximal size of a chunk that can be allocated with `kmalloc` is +limited. The actual limit depends on the hardware and the kernel +configuration, but it is a good practice to use `kmalloc` for objects +smaller than page size. + +For large allocations you can use `vmalloc` and `vzalloc`, or directly +request pages from the page allocator. The memory allocated by +`vmalloc` and related functions is not physically contiguous. + +If you are not sure whether the allocation size is too large for +`kmalloc` it is possible to use `kvmalloc` and its derivatives. It +will try to allocate memory with `kmalloc` and if the allocation fails +it will be retried with `vmalloc`. There are restrictions on which GFP +flags can be used with `kvmalloc`, please see :c:func:`kvmalloc_node` +reference documentation. Note, that `kvmalloc` may return memory that +is not physically contiguous. + +If you need to allocate many identical objects you can use slab cache +allocator. The cache should be set up with `kmem_cache_create` before +it can be used. Afterwards `kmem_cache_alloc` and its convenience +wrappers can allocate memory from that cache. + +When the allocated memory is no longer needed it must be freed. You +can use `kvfree` for the memory allocated with `kmalloc`, `vmalloc` +and `kvmalloc`. The slab caches should be freed with +`kmem_cache_free`. And don't forget to destroy the cache with +`kmem_cache_destroy`. diff --git a/Documentation/core-api/mm-api.rst b/Documentation/core-api/mm-api.rst index 46ae353..5ce1ec1 100644 --- a/Documentation/core-api/mm-api.rst +++ b/Documentation/core-api/mm-api.rst @@ -14,6 +14,8 @@ User Space Memory Access .. kernel-doc:: mm/util.c :functions: get_user_pages_fast +.. _mm-api-gfp-flags: + Memory Allocation Controls ==========================