diff mbox

[v3] mm: make expand_downwards symmetrical to expand_upwards

Message ID 1303235306.3171.33.camel@mulgrave.site (mailing list archive)
State Not Applicable
Headers show

Commit Message

James Bottomley April 19, 2011, 5:48 p.m. UTC 
On Tue, 2011-04-19 at 12:15 -0500, Christoph Lameter wrote:
> On Tue, 19 Apr 2011, James Bottomley wrote:
> 
> > On Tue, 2011-04-19 at 20:05 +0300, Pekka Enberg wrote:
> > > > It seems to be a random intermittent mm crash because the next reboot
> > > > crashed with the same trace but after the fsck had completed and the
> > > > third came up to the login prompt.
> > >
> > > Looks like a genuine SLUB problem on parisc. Christoph?
> >
> > Looking through the slub code, it seems to be making invalid
> > assumptions.  All of the node stuff is dependent on CONFIG_NUMA.
> > However, we're CONFIG_DISCONTIGMEM (with CONFIG_NUMA not set): on the
> > machines I and Dave Anglin have, our physical memory ranges are 0-1GB
> > and 64-65GB, so I think slub crashes when we get a page from the high
> > memory range ... because it's not expecting a non-zero node number.
> 
> Right !NUMA systems only have node 0.

That's rubbish.  Discontigmem uses the nodes field to identify the
discontiguous region.  page_to_nid() returns this value.  Your code
wrongly assumes this is zero for non NUMA.

This simple program triggers the problem instantly because it forces
allocation up into the upper discontigmem range:

#include <stdlib.h>

void main(void)
{
  const long size = 1024*1024*1024;
  char *a = malloc(size);
  int i;

  for (i=0; i < size; i += 4096)
    a[i] = '\0';
}

I can fix the panic by hard coding get_nodes() to return the zero node
for the non-numa case ... however, presumably it's more than just this
that's broken in slub?

James

---



--
To unsubscribe from this list: send the line "unsubscribe linux-parisc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Comments

Christoph Lameter (Ampere) April 19, 2011, 6:10 p.m. UTC | #1 
On Tue, 19 Apr 2011, James Bottomley wrote:

> > Right !NUMA systems only have node 0.
>
> That's rubbish.  Discontigmem uses the nodes field to identify the
> discontiguous region.  page_to_nid() returns this value.  Your code
> wrongly assumes this is zero for non NUMA.

Sorry the kernel has no node awareness if you do not set CONFIG_NUMA

F.e. zone node lookups work the following way

static inline int
zone_to_nid(struct zone *zone)
{
#ifdef CONFIG_NUMA
        return zone->node;
#else
        return 0;
#endif
}

How in the world did you get a zone setup in node 1 with a !NUMA config?


The problem seems to be that the kernel seems to allow a
definition of a page_to_nid() function that returns non zero in the !NUMA
case. And slub relies on page_to_nid returning zero in the !NUMA case.
Because NODES_WIDTH should be 0 in the !NUMA case and therefore
page_to_nid must return 0.

> I can fix the panic by hard coding get_nodes() to return the zero node
> for the non-numa case ... however, presumably it's more than just this
> that's broken in slub?

If you think that is broken then we have brokenness all over the kernel
whenever we determine the node from a page and use that to do a lookup.

--
To unsubscribe from this list: send the line "unsubscribe linux-parisc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
James Bottomley April 19, 2011, 6:20 p.m. UTC | #2 
On Tue, 2011-04-19 at 13:10 -0500, Christoph Lameter wrote:
> On Tue, 19 Apr 2011, James Bottomley wrote:
> 
> > > Right !NUMA systems only have node 0.
> >
> > That's rubbish.  Discontigmem uses the nodes field to identify the
> > discontiguous region.  page_to_nid() returns this value.  Your code
> > wrongly assumes this is zero for non NUMA.
> 
> Sorry the kernel has no node awareness if you do not set CONFIG_NUMA
> 
> F.e. zone node lookups work the following way
> 
> static inline int
> zone_to_nid(struct zone *zone)
> {
> #ifdef CONFIG_NUMA
>         return zone->node;
> #else
>         return 0;
> #endif
> }
> 
> How in the world did you get a zone setup in node 1 with a !NUMA config?

I told you ... I forced an allocation into the first discontiguous
region.  That will return 1 for page_to_nid().

> The problem seems to be that the kernel seems to allow a
> definition of a page_to_nid() function that returns non zero in the !NUMA
> case.

This is called reality, yes.

>  And slub relies on page_to_nid returning zero in the !NUMA case.
> Because NODES_WIDTH should be 0 in the !NUMA case and therefore
> page_to_nid must return 0.

right, that's what I told you: slub is broken because it's making a
wrong assumption.  Look in asm-generic/memory_model.h it shows how the
page_to_nid() is used in finding the pfn array.  DISCONTIGMEM uses some
of the numa properties (including assigning zones to the discontiguous
regions).

> > I can fix the panic by hard coding get_nodes() to return the zero node
> > for the non-numa case ... however, presumably it's more than just this
> > that's broken in slub?
> 
> If you think that is broken then we have brokenness all over the kernel
> whenever we determine the node from a page and use that to do a lookup.

Not really.  The rest of the kernel uses the proper macros.  in
DISCONTIGMEM but !NUMA configs, the numa macros expand correctly.
You've cut across that with all the CONFIG_NUMA checks in slub.

James


--
To unsubscribe from this list: send the line "unsubscribe linux-parisc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
diff mbox

Patch

diff --git a/mm/slub.c b/mm/slub.c
index 94d2a33..243bd9c 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -235,7 +235,11 @@  int slab_is_available(void)
 
 static inline struct kmem_cache_node *get_node(struct kmem_cache *s, int node)
 {
+#ifdef CONFIG_NUMA
 	return s->node[node];
+#else
+	return s->node[0];
+#endif
 }
 
 /* Verify that a pointer has an address that is valid within a slab page */