| Message ID | 1481139600-24455-2-git-send-email-scott.branden@broadcom.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Wed, Dec 07, 2016 at 11:40:00AM -0800, Scott Branden wrote: > Make MAX_PHYSMEM_BITS and SECTIONS_SIZE_BITS configurable by adding > config options. > Default to current settings currently defined in sparesmem.h. > For systems wishing to save memory the config options can be overridden. > Example, changing MAX_PHYSMEM_BITS from 48 to 36 at the same time as > changing SECTION_SIZE_BITS from 30 to 26 frees 13MB of memory. I'm not keen on such change, it's a big departure from the single Image aims. I would rather reduce SECTION_SIZE_BITS permanently where feasible, like in this patch: http://lkml.kernel.org/r/1465821119-3384-1-git-send-email-jszhang@marvell.com
Hi Catalin, On 16-12-08 02:00 AM, Catalin Marinas wrote: > On Wed, Dec 07, 2016 at 11:40:00AM -0800, Scott Branden wrote: >> Make MAX_PHYSMEM_BITS and SECTIONS_SIZE_BITS configurable by adding >> config options. >> Default to current settings currently defined in sparesmem.h. >> For systems wishing to save memory the config options can be overridden. >> Example, changing MAX_PHYSMEM_BITS from 48 to 36 at the same time as >> changing SECTION_SIZE_BITS from 30 to 26 frees 13MB of memory. > > I'm not keen on such change, it's a big departure from the single Image > aims. A single Image is not entirely possible when the system needs to be tuned for memory usage, boot time, and other performance related issues. These are key features in embedded systems vs. general purpose computers. I would rather reduce SECTION_SIZE_BITS permanently where > feasible, like in this patch: > > http://lkml.kernel.org/r/1465821119-3384-1-git-send-email-jszhang@marvell.com > This patch does not meet my requirements as I need SECTION_SIZE_BITS to be set to 28 to reduce memory and to allow memory hotplug to allocate a 256 MB section. My patch future proofs the tuning of the parameters by allowing any section size to be made. I could combine the patch you list such that SECTION_SIZE_BITS defaults to 30 when CONFIG_ARM64_64_PAGES is selected and 27 otherwise. Should it default to something else for 16K and 4K pages? In terms of MAX_PHYSMEM_BITS, if our SoCs only use 40 (or less) bits I would also like the configuration functionality. This allows us to make the SECTION_SIZE_BITS smaller. Regards, Scott
On Thu, Dec 08, 2016 at 08:30:36AM -0800, Scott Branden wrote: > On 16-12-08 02:00 AM, Catalin Marinas wrote: > >On Wed, Dec 07, 2016 at 11:40:00AM -0800, Scott Branden wrote: > >>Make MAX_PHYSMEM_BITS and SECTIONS_SIZE_BITS configurable by adding > >>config options. > >>Default to current settings currently defined in sparesmem.h. > >>For systems wishing to save memory the config options can be overridden. > >>Example, changing MAX_PHYSMEM_BITS from 48 to 36 at the same time as > >>changing SECTION_SIZE_BITS from 30 to 26 frees 13MB of memory. [...] > > I would rather reduce SECTION_SIZE_BITS permanently where > >feasible, like in this patch: > > > >http://lkml.kernel.org/r/1465821119-3384-1-git-send-email-jszhang@marvell.com > > This patch does not meet my requirements as I need SECTION_SIZE_BITS to be > set to 28 to reduce memory So with this patch, we reduce it to 27, it should be fine-grained enough for 128MB sections. Alternatively, there were other suggestions here: http://lkml.iu.edu/hypermail/linux/kernel/1604.1/03036.html > and to allow memory hotplug to allocate a 256 MB section. Can memory hotplug not work with 2*128MB sections in this case? > My patch future proofs the tuning of the parameters by allowing > any section size to be made. While MAX_PHYSMEM_BITS makes sense to users in general, SECTION_SIZE_BITS is not always clear to the average user what it means and its min/max boundaries. That's another reason (apart from single/few Image case) why I prefer to not expose it as configuration option. > I could combine the patch you list such that > SECTION_SIZE_BITS defaults to 30 when CONFIG_ARM64_64_PAGES is selected and > 27 otherwise. Should it default to something else for 16K and 4K pages? I haven't done any calculations for 16K yet but we could probably come up with some formula based on PAGE_SHIFT to cover all cases. > In terms of MAX_PHYSMEM_BITS, if our SoCs only use 40 (or less) bits I would > also like the configuration functionality. This allows us to make the > SECTION_SIZE_BITS smaller. So how small do you want SECTION_SIZE_BITS to be? As I said above, 128MB sections should be sufficient in most cases and without the need to reduce MAX_PHYSMEM_BITS.
On 16-12-08 10:57 AM, Catalin Marinas wrote: > On Thu, Dec 08, 2016 at 08:30:36AM -0800, Scott Branden wrote: >> On 16-12-08 02:00 AM, Catalin Marinas wrote: >>> On Wed, Dec 07, 2016 at 11:40:00AM -0800, Scott Branden wrote: >>>> Make MAX_PHYSMEM_BITS and SECTIONS_SIZE_BITS configurable by adding >>>> config options. >>>> Default to current settings currently defined in sparesmem.h. >>>> For systems wishing to save memory the config options can be overridden. >>>> Example, changing MAX_PHYSMEM_BITS from 48 to 36 at the same time as >>>> changing SECTION_SIZE_BITS from 30 to 26 frees 13MB of memory. > [...] >>> I would rather reduce SECTION_SIZE_BITS permanently where >>> feasible, like in this patch: >>> >>> http://lkml.kernel.org/r/1465821119-3384-1-git-send-email-jszhang@marvell.com >> >> This patch does not meet my requirements as I need SECTION_SIZE_BITS to be >> set to 28 to reduce memory > > So with this patch, we reduce it to 27, it should be fine-grained enough > for 128MB sections. Alternatively, there were other suggestions here: > > http://lkml.iu.edu/hypermail/linux/kernel/1604.1/03036.html > >> and to allow memory hotplug to allocate a 256 MB section. > > Can memory hotplug not work with 2*128MB sections in this case? Yes, I then need to hotplug the memory at 2 locations for 1 memory addition but that will work in my current use case. I'm one step away from hotplug working on ARM64. Once that works I hope to break the dependencies between hotplug memory size created based on SECTION_SIZE_BITS in the future. Since I currently have your attention: I do think there is fundamental bug in the ARM64 mm implementation. If you look at /sys/devices/system/memory it only shows the last memoryX section created after init. It should be showing up multiple sections. As a quick test change SECTION_SIZE_BITS and you look at /sys/devices/system/memory to see what changes. Look at a standard x64 machine you will see all the memoryX entries present. > >> My patch future proofs the tuning of the parameters by allowing >> any section size to be made. > > While MAX_PHYSMEM_BITS makes sense to users in general, > SECTION_SIZE_BITS is not always clear to the average user what it means > and its min/max boundaries. That's another reason (apart from single/few > Image case) why I prefer to not expose it as configuration option. I agree SECTION_SIZE_BITS is confusing. If you could provide more documentation on what it means and how it is used that would help others for sure. I just stumbled upon it while working on a tight memory system and found it saves me significant memory. > >> I could combine the patch you list such that >> SECTION_SIZE_BITS defaults to 30 when CONFIG_ARM64_64_PAGES is selected and >> 27 otherwise. Should it default to something else for 16K and 4K pages? > > I haven't done any calculations for 16K yet but we could probably come > up with some formula based on PAGE_SHIFT to cover all cases. Yes, a calculation based on pages that modified SECTION_SIZE_BITS would probably be a better solution. > >> In terms of MAX_PHYSMEM_BITS, if our SoCs only use 40 (or less) bits I would >> also like the configuration functionality. This allows us to make the >> SECTION_SIZE_BITS smaller. > > So how small do you want SECTION_SIZE_BITS to be? As I said above, 128MB > sections should be sufficient in most cases and without the need to > reduce MAX_PHYSMEM_BITS. > 256MB works for my current use case. But appears somebody else was looking for 64MB previously. So that is why adding support for modifying MAX_PHYSMEM_BITS makes sense as it needs to be modified to support the 64MB case: https://lkml.org/lkml/2016/8/11/209
On Thu, Dec 08, 2016 at 11:33:39AM -0800, Scott Branden wrote: > Since I currently have your attention: I do think there is fundamental bug > in the ARM64 mm implementation. If you look at /sys/devices/system/memory > it only shows the last memoryX section created after init. That directory doesn't seem to exist on my arm64 systems. Do I have to enable something specific in the .config? Will
Hi Will, On 16-12-09 02:57 AM, Will Deacon wrote: > On Thu, Dec 08, 2016 at 11:33:39AM -0800, Scott Branden wrote: >> Since I currently have your attention: I do think there is fundamental bug >> in the ARM64 mm implementation. If you look at /sys/devices/system/memory >> it only shows the last memoryX section created after init. > > That directory doesn't seem to exist on my arm64 systems. Do I have to > enable something specific in the .config? I looked in the /sys/devices/system/memory at it doesn't look like it appears until memory hotplug is enabled in the system. This is another patch I'm trying to work through at the same time: https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1284943.html The internals of the memory management subsystem is not something I'm too familiar with at this point. > > Will > Regards, Scott
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 2482fdd..d0a95de 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -121,6 +121,27 @@ config ARCH_PHYS_ADDR_T_64BIT config MMU def_bool y +config MAX_PHYSMEM_BITS + int "Maximum Physical Addressable Bits" + depends on SPARSEMEM + default 48 + help + Maximum number of physical addressable bits. + If not all the physical bits are used this number can be reduced + to save memory when creating page tables. + + If unsure, leave this at the default. + +config SECTION_SIZE_BITS + int "Section Size Bits" + depends on SPARSEMEM + default 30 + help + Size of each memory section. + If you need to be able to allocate smaller + sections in page table this number can be change to save memory. + + If unsure, leave this at the default. config DEBUG_RODATA def_bool y diff --git a/arch/arm64/include/asm/sparsemem.h b/arch/arm64/include/asm/sparsemem.h index 74a9d30..6677582 100644 --- a/arch/arm64/include/asm/sparsemem.h +++ b/arch/arm64/include/asm/sparsemem.h @@ -17,8 +17,8 @@ #define __ASM_SPARSEMEM_H #ifdef CONFIG_SPARSEMEM -#define MAX_PHYSMEM_BITS 48 -#define SECTION_SIZE_BITS 30 +#define MAX_PHYSMEM_BITS CONFIG_MAX_PHYSMEM_BITS +#define SECTION_SIZE_BITS CONFIG_SECTION_SIZE_BITS #endif #endif
Make MAX_PHYSMEM_BITS and SECTIONS_SIZE_BITS configurable by adding config options. Default to current settings currently defined in sparesmem.h. For systems wishing to save memory the config options can be overridden. Example, changing MAX_PHYSMEM_BITS from 48 to 36 at the same time as changing SECTION_SIZE_BITS from 30 to 26 frees 13MB of memory. Signed-off-by: Scott Branden <scott.branden@broadcom.com> --- arch/arm64/Kconfig | 21 +++++++++++++++++++++ arch/arm64/include/asm/sparsemem.h | 4 ++-- 2 files changed, 23 insertions(+), 2 deletions(-)