diff mbox

[v2,11/11] docs/mm: add description of boot time memory management

Message ID 1530370506-21751-12-git-send-email-rppt@linux.vnet.ibm.com (mailing list archive)
State New, archived
Headers show

Commit Message

Mike Rapoport June 30, 2018, 2:55 p.m. UTC 
Both bootmem and memblock are have pretty good internal documentation
coverage. With addition of some overview we get a nice description of the
early memory management.

Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
---
 Documentation/core-api/boot-time-mm.rst | 92 +++++++++++++++++++++++++++++++++
 Documentation/core-api/index.rst        |  1 +
 2 files changed, 93 insertions(+)
 create mode 100644 Documentation/core-api/boot-time-mm.rst

Comments

Michal Hocko July 3, 2018, 12:23 p.m. UTC | #1 
On Sat 30-06-18 17:55:06, Mike Rapoport wrote:
> Both bootmem and memblock are have pretty good internal documentation
> coverage. With addition of some overview we get a nice description of the
> early memory management.
> 
> Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>

Yes this looks reasonable. I would just mention the available debugging
options and CONFIG_ARCH_DISCARD_MEMBLOCK.

Other than that looks goot to get a rough idea. Improvements can be done
on top of course.

Acked-by: Michal Hocko <mhocko@suse.com>

> ---
>  Documentation/core-api/boot-time-mm.rst | 92 +++++++++++++++++++++++++++++++++
>  Documentation/core-api/index.rst        |  1 +
>  2 files changed, 93 insertions(+)
>  create mode 100644 Documentation/core-api/boot-time-mm.rst
> 
> diff --git a/Documentation/core-api/boot-time-mm.rst b/Documentation/core-api/boot-time-mm.rst
> new file mode 100644
> index 0000000..03cb164
> --- /dev/null
> +++ b/Documentation/core-api/boot-time-mm.rst
> @@ -0,0 +1,92 @@
> +===========================
> +Boot time memory management
> +===========================
> +
> +Early system initialization cannot use "normal" memory management
> +simply because it is not set up yet. But there is still need to
> +allocate memory for various data structures, for instance for the
> +physical page allocator. To address this, a specialized allocator
> +called the :ref:`Boot Memory Allocator <bootmem>`, or bootmem, was
> +introduced. Several years later PowerPC developers added a "Logical
> +Memory Blocks" allocator, which was later adopted by other
> +architectures and renamed to :ref:`memblock <memblock>`. There is also
> +a compatibility layer called `nobootmem` that translates bootmem
> +allocation interfaces to memblock calls.
> +
> +The selection of the early allocator is done using
> +``CONFIG_NO_BOOTMEM`` and ``CONFIG_HAVE_MEMBLOCK`` kernel
> +configuration options. These options are enabled or disabled
> +statically by the architectures' Kconfig files.
> +
> +* Architectures that rely only on bootmem select
> +  ``CONFIG_NO_BOOTMEM=n && CONFIG_HAVE_MEMBLOCK=n``.
> +* The users of memblock with the nobootmem compatibility layer set
> +  ``CONFIG_NO_BOOTMEM=y && CONFIG_HAVE_MEMBLOCK=y``.
> +* And for those that use both memblock and bootmem the configuration
> +  includes ``CONFIG_NO_BOOTMEM=n && CONFIG_HAVE_MEMBLOCK=y``.
> +
> +Whichever allocator is used, it is the responsibility of the
> +architecture specific initialization to set it up in
> +:c:func:`setup_arch` and tear it down in :c:func:`mem_init` functions.
> +
> +Once the early memory management is available it offers a variety of
> +functions and macros for memory allocations. The allocation request
> +may be directed to the first (and probably the only) node or to a
> +particular node in a NUMA system. There are API variants that panic
> +when an allocation fails and those that don't. And more recent and
> +advanced memblock even allows controlling its own behaviour.
> +
> +.. _bootmem:
> +
> +Bootmem
> +=======
> +
> +(mostly stolen from Mel Gorman's "Understanding the Linux Virtual
> +Memory Manager" `book`_)
> +
> +.. _book: https://www.kernel.org/doc/gorman/
> +
> +.. kernel-doc:: mm/bootmem.c
> +   :doc: bootmem overview
> +
> +.. _memblock:
> +
> +Memblock
> +========
> +
> +.. kernel-doc:: mm/memblock.c
> +   :doc: memblock overview
> +
> +
> +Functions and structures
> +========================
> +
> +Common API
> +----------
> +
> +The functions that are described in this section are available
> +regardless of what early memory manager is enabled.
> +
> +.. kernel-doc:: mm/nobootmem.c
> +
> +Bootmem specific API
> +--------------------
> +
> +These interfaces available only with bootmem, i.e when ``CONFIG_NO_BOOTMEM=n``
> +
> +.. kernel-doc:: include/linux/bootmem.h
> +.. kernel-doc:: mm/bootmem.c
> +   :nodocs:
> +
> +Memblock specific API
> +---------------------
> +
> +Here is the description of memblock data structures, functions and
> +macros. Some of them are actually internal, but since they are
> +documented it would be silly to omit them. Besides, reading the
> +descriptions for the internal functions can help to understand what
> +really happens under the hood.
> +
> +.. kernel-doc:: include/linux/memblock.h
> +.. kernel-doc:: mm/memblock.c
> +   :nodocs:
> diff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst
> index f5a66b7..93d5a46 100644
> --- a/Documentation/core-api/index.rst
> +++ b/Documentation/core-api/index.rst
> @@ -28,6 +28,7 @@ Core utilities
>     printk-formats
>     circular-buffers
>     gfp_mask-from-fs-io
> +   boot-time-mm
>  
>  Interfaces for kernel debugging
>  ===============================
> -- 
> 2.7.4
Mike Rapoport July 12, 2018, 7:26 a.m. UTC | #2 
Hi,

On Tue, Jul 03, 2018 at 02:23:24PM +0200, Michal Hocko wrote:
> On Sat 30-06-18 17:55:06, Mike Rapoport wrote:
> > Both bootmem and memblock are have pretty good internal documentation
> > coverage. With addition of some overview we get a nice description of the
> > early memory management.
> > 
> > Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
> 
> Yes this looks reasonable. I would just mention the available debugging
> options and CONFIG_ARCH_DISCARD_MEMBLOCK.

I'd really prefer to add it as a separate patch rather then resending the
whole series.

> Other than that looks goot to get a rough idea. Improvements can be done
> on top of course.
> 
> Acked-by: Michal Hocko <mhocko@suse.com>

Thanks for the review. I think Jon was mostly concerned about the patch
"mm/memblock: add a name for memblock flags enumeration" [1]. Could you
please review it as well?

[1] https://lore.kernel.org/lkml/1530370506-21751-7-git-send-email-rppt@linux.vnet.ibm.com/

> > ---
> >  Documentation/core-api/boot-time-mm.rst | 92 +++++++++++++++++++++++++++++++++
> >  Documentation/core-api/index.rst        |  1 +
> >  2 files changed, 93 insertions(+)
> >  create mode 100644 Documentation/core-api/boot-time-mm.rst
> > 
> > diff --git a/Documentation/core-api/boot-time-mm.rst b/Documentation/core-api/boot-time-mm.rst
> > new file mode 100644
> > index 0000000..03cb164
> > --- /dev/null
> > +++ b/Documentation/core-api/boot-time-mm.rst
> > @@ -0,0 +1,92 @@
> > +===========================
> > +Boot time memory management
> > +===========================
> > +
> > +Early system initialization cannot use "normal" memory management
> > +simply because it is not set up yet. But there is still need to
> > +allocate memory for various data structures, for instance for the
> > +physical page allocator. To address this, a specialized allocator
> > +called the :ref:`Boot Memory Allocator <bootmem>`, or bootmem, was
> > +introduced. Several years later PowerPC developers added a "Logical
> > +Memory Blocks" allocator, which was later adopted by other
> > +architectures and renamed to :ref:`memblock <memblock>`. There is also
> > +a compatibility layer called `nobootmem` that translates bootmem
> > +allocation interfaces to memblock calls.
> > +
> > +The selection of the early allocator is done using
> > +``CONFIG_NO_BOOTMEM`` and ``CONFIG_HAVE_MEMBLOCK`` kernel
> > +configuration options. These options are enabled or disabled
> > +statically by the architectures' Kconfig files.
> > +
> > +* Architectures that rely only on bootmem select
> > +  ``CONFIG_NO_BOOTMEM=n && CONFIG_HAVE_MEMBLOCK=n``.
> > +* The users of memblock with the nobootmem compatibility layer set
> > +  ``CONFIG_NO_BOOTMEM=y && CONFIG_HAVE_MEMBLOCK=y``.
> > +* And for those that use both memblock and bootmem the configuration
> > +  includes ``CONFIG_NO_BOOTMEM=n && CONFIG_HAVE_MEMBLOCK=y``.
> > +
> > +Whichever allocator is used, it is the responsibility of the
> > +architecture specific initialization to set it up in
> > +:c:func:`setup_arch` and tear it down in :c:func:`mem_init` functions.
> > +
> > +Once the early memory management is available it offers a variety of
> > +functions and macros for memory allocations. The allocation request
> > +may be directed to the first (and probably the only) node or to a
> > +particular node in a NUMA system. There are API variants that panic
> > +when an allocation fails and those that don't. And more recent and
> > +advanced memblock even allows controlling its own behaviour.
> > +
> > +.. _bootmem:
> > +
> > +Bootmem
> > +=======
> > +
> > +(mostly stolen from Mel Gorman's "Understanding the Linux Virtual
> > +Memory Manager" `book`_)
> > +
> > +.. _book: https://www.kernel.org/doc/gorman/
> > +
> > +.. kernel-doc:: mm/bootmem.c
> > +   :doc: bootmem overview
> > +
> > +.. _memblock:
> > +
> > +Memblock
> > +========
> > +
> > +.. kernel-doc:: mm/memblock.c
> > +   :doc: memblock overview
> > +
> > +
> > +Functions and structures
> > +========================
> > +
> > +Common API
> > +----------
> > +
> > +The functions that are described in this section are available
> > +regardless of what early memory manager is enabled.
> > +
> > +.. kernel-doc:: mm/nobootmem.c
> > +
> > +Bootmem specific API
> > +--------------------
> > +
> > +These interfaces available only with bootmem, i.e when ``CONFIG_NO_BOOTMEM=n``
> > +
> > +.. kernel-doc:: include/linux/bootmem.h
> > +.. kernel-doc:: mm/bootmem.c
> > +   :nodocs:
> > +
> > +Memblock specific API
> > +---------------------
> > +
> > +Here is the description of memblock data structures, functions and
> > +macros. Some of them are actually internal, but since they are
> > +documented it would be silly to omit them. Besides, reading the
> > +descriptions for the internal functions can help to understand what
> > +really happens under the hood.
> > +
> > +.. kernel-doc:: include/linux/memblock.h
> > +.. kernel-doc:: mm/memblock.c
> > +   :nodocs:
> > diff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst
> > index f5a66b7..93d5a46 100644
> > --- a/Documentation/core-api/index.rst
> > +++ b/Documentation/core-api/index.rst
> > @@ -28,6 +28,7 @@ Core utilities
> >     printk-formats
> >     circular-buffers
> >     gfp_mask-from-fs-io
> > +   boot-time-mm
> >  
> >  Interfaces for kernel debugging
> >  ===============================
> > -- 
> > 2.7.4
> 
> -- 
> Michal Hocko
> SUSE Labs
>
diff mbox

Patch

diff --git a/Documentation/core-api/boot-time-mm.rst b/Documentation/core-api/boot-time-mm.rst
new file mode 100644
index 0000000..03cb164
--- /dev/null
+++ b/Documentation/core-api/boot-time-mm.rst
@@ -0,0 +1,92 @@ 
+===========================
+Boot time memory management
+===========================
+
+Early system initialization cannot use "normal" memory management
+simply because it is not set up yet. But there is still need to
+allocate memory for various data structures, for instance for the
+physical page allocator. To address this, a specialized allocator
+called the :ref:`Boot Memory Allocator <bootmem>`, or bootmem, was
+introduced. Several years later PowerPC developers added a "Logical
+Memory Blocks" allocator, which was later adopted by other
+architectures and renamed to :ref:`memblock <memblock>`. There is also
+a compatibility layer called `nobootmem` that translates bootmem
+allocation interfaces to memblock calls.
+
+The selection of the early allocator is done using
+``CONFIG_NO_BOOTMEM`` and ``CONFIG_HAVE_MEMBLOCK`` kernel
+configuration options. These options are enabled or disabled
+statically by the architectures' Kconfig files.
+
+* Architectures that rely only on bootmem select
+  ``CONFIG_NO_BOOTMEM=n && CONFIG_HAVE_MEMBLOCK=n``.
+* The users of memblock with the nobootmem compatibility layer set
+  ``CONFIG_NO_BOOTMEM=y && CONFIG_HAVE_MEMBLOCK=y``.
+* And for those that use both memblock and bootmem the configuration
+  includes ``CONFIG_NO_BOOTMEM=n && CONFIG_HAVE_MEMBLOCK=y``.
+
+Whichever allocator is used, it is the responsibility of the
+architecture specific initialization to set it up in
+:c:func:`setup_arch` and tear it down in :c:func:`mem_init` functions.
+
+Once the early memory management is available it offers a variety of
+functions and macros for memory allocations. The allocation request
+may be directed to the first (and probably the only) node or to a
+particular node in a NUMA system. There are API variants that panic
+when an allocation fails and those that don't. And more recent and
+advanced memblock even allows controlling its own behaviour.
+
+.. _bootmem:
+
+Bootmem
+=======
+
+(mostly stolen from Mel Gorman's "Understanding the Linux Virtual
+Memory Manager" `book`_)
+
+.. _book: https://www.kernel.org/doc/gorman/
+
+.. kernel-doc:: mm/bootmem.c
+   :doc: bootmem overview
+
+.. _memblock:
+
+Memblock
+========
+
+.. kernel-doc:: mm/memblock.c
+   :doc: memblock overview
+
+
+Functions and structures
+========================
+
+Common API
+----------
+
+The functions that are described in this section are available
+regardless of what early memory manager is enabled.
+
+.. kernel-doc:: mm/nobootmem.c
+
+Bootmem specific API
+--------------------
+
+These interfaces available only with bootmem, i.e when ``CONFIG_NO_BOOTMEM=n``
+
+.. kernel-doc:: include/linux/bootmem.h
+.. kernel-doc:: mm/bootmem.c
+   :nodocs:
+
+Memblock specific API
+---------------------
+
+Here is the description of memblock data structures, functions and
+macros. Some of them are actually internal, but since they are
+documented it would be silly to omit them. Besides, reading the
+descriptions for the internal functions can help to understand what
+really happens under the hood.
+
+.. kernel-doc:: include/linux/memblock.h
+.. kernel-doc:: mm/memblock.c
+   :nodocs:
diff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst
index f5a66b7..93d5a46 100644
--- a/Documentation/core-api/index.rst
+++ b/Documentation/core-api/index.rst
@@ -28,6 +28,7 @@  Core utilities
    printk-formats
    circular-buffers
    gfp_mask-from-fs-io
+   boot-time-mm
 
 Interfaces for kernel debugging
 ===============================