From patchwork Wed Jun 6 08:04:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 10449795 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A42F86053F for ; Wed, 6 Jun 2018 08:04:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 94147285A6 for ; Wed, 6 Jun 2018 08:04:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8802C297BC; Wed, 6 Jun 2018 08:04:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D692B285A6 for ; Wed, 6 Jun 2018 08:04:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C81096B0005; Wed, 6 Jun 2018 04:04:12 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C30186B0006; Wed, 6 Jun 2018 04:04:12 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AF8F86B0007; Wed, 6 Jun 2018 04:04:12 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm0-f69.google.com (mail-wm0-f69.google.com [74.125.82.69]) by kanga.kvack.org (Postfix) with ESMTP id 55CFD6B0005 for ; Wed, 6 Jun 2018 04:04:12 -0400 (EDT) Received: by mail-wm0-f69.google.com with SMTP id v5-v6so2675946wmh.6 for ; Wed, 06 Jun 2018 01:04:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:date:from:to :cc:subject:message-id:references:mime-version:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=d4mJPCqZm4P9wzDuoWWwASC0GgsNC3nZd2Z9CTHNfjg=; b=r554cCQ/hH1FLBMO+KLlgjPyvtYKHh2XCI5XnN65NO6VFFllFVvx16TWojktjitznN 5AJo4fMPEw5Vb/ACjXu1w50f8LaJTST5XInWQS72PPVGSK9CHrhOHeWH16VCgqCNndEK W4xC1WyT3IiVl7iLkTOqPiVY3n+jCkpMRzki4CkhdDDtlI0k7LVjCCxjDJOoGMkeSsV3 cJppcWW2oSsEJV/8Y0XNKR6iXhlS0SzMk6V3s3Cj2672GwI2fkyZlfDJfe04n0mynNA6 d2YWWrjsUB48gWRsLg8OJuxfFrqXR2HceTY5+t3v16Py+y5wXgDwNdwvOn82QhWQz1zu S8KA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=osalvador.vilardaga@gmail.com X-Gm-Message-State: APt69E3Oz+thUWRF+ZHfpKJBmZEpRbY27N5pUUW/gO2Ix/xsK1E4RyMt F+XRce9t4G19jqLe9Tw6nEIhcBrd9zYLchukftNSJdVnl74X0HJyBNy8vZBRQqeBMa8MabJiqLg Pw4eReXftms4pYNcTLzjrmtSNhWfKYq7azKfr7xvmaUFGpxMAUjSjwN3xkwqZ90XR2Vcb3DmM0/ qPWdL9A1O9i9AstDlQOB3XXiYAPFyykPhgxKkOImRdCPpGySdsGq9PqAq9M+gFCkSP5pfchh6y1 847/fHmXq3//YYweWaRPWsyKm2FOmGBdacZ7QxyOuQwCnwWXqG/7Se5vRDFiVXykzxkZdyuPKhC 2nbsIUKMptgWaFXVrb068DMsp5EdU75p144Dg5cSS/8SEG8Wuja4rZYceMgLida7Vi/zV1IWBA= = X-Received: by 2002:adf:e4cd:: with SMTP id v13-v6mr1415647wrm.238.1528272251844; Wed, 06 Jun 2018 01:04:11 -0700 (PDT) X-Received: by 2002:adf:e4cd:: with SMTP id v13-v6mr1415554wrm.238.1528272250700; Wed, 06 Jun 2018 01:04:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1528272250; cv=none; d=google.com; s=arc-20160816; b=eL51DzRtvsfn8WSvq9QiujpGMPkFZUJyOvi9oJFz/zGdRSInVlnMbpZuxhgnZE1t9s jWr3shjcnMS307GTnYwxY/fOqnjquftQ+FxunKdkrGqt+oiBmEpbYfRXh3pJXSAAoQKQ KQe0of74tyRJITiuhKvYQRwozTu+l+xUpEA8PG3VhensAAvEHgEMnybX0pa+Dcwx/HY8 fvDlL430D8POcLFFdRPWLvMHTJZ7qgCNLjRXJPwp3cpYsefCOSBFXjLSHJRVPl12Xx2Y GPZrXcrwqyVd6vavsCeEfx1ej3417yH65DlBPjfACqeRtRl24GpyzSW0wkSqummb7sls 5n9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=user-agent:in-reply-to:content-transfer-encoding :content-disposition:mime-version:references:message-id:subject:cc :to:from:date:arc-authentication-results; bh=d4mJPCqZm4P9wzDuoWWwASC0GgsNC3nZd2Z9CTHNfjg=; b=R9lsjuJ2MJqTLVcng2fFaT8C3dOsGtP9BG+vmr3soQ6l5HlC0xjeKB1L/wACalyImK w76wjzmLkupkwgevPS+0HBGzAURRO6FVAEnpEG8vXxk7Fal6BNBuQHpcIW5Ot7ybxf/9 3p7uxJ/Nr0o9QjPOwxK9Hqxdvb99mRzHkQol3YSzMGdfMjPfbhogfgfhDyjKkLh3swOA 563TeDsUQ+QiTmk0zAeHioaG/NElgDXsbQA6zULnl7FmT8v9voH2h7FdEoe4UEZ5cbD/ YBNlF+Aez1J1iDr2ke5JfWKNtakwNdC7UKOJWkHkAhxsJimM9cF9+rKAcX5YKBC8B5/7 RY/w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=osalvador.vilardaga@gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id m184-v6sor966151wmd.84.2018.06.06.01.04.10 for (Google Transport Security); Wed, 06 Jun 2018 01:04:10 -0700 (PDT) Received-SPF: pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=osalvador.vilardaga@gmail.com X-Google-Smtp-Source: ADUXVKK4T4ml1uB0y7C5QTM86vpUZze24RowKJqZc7aUi5fmtxlIhkIVFJ5fIUsJnzddOS8Pom72rw== X-Received: by 2002:a1c:b7d4:: with SMTP id h203-v6mr938543wmf.161.1528272250318; Wed, 06 Jun 2018 01:04:10 -0700 (PDT) Received: from techadventures.net (techadventures.net. [62.201.165.239]) by smtp.gmail.com with ESMTPSA id o16-v6sm16331002wrp.8.2018.06.06.01.04.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 06 Jun 2018 01:04:09 -0700 (PDT) Received: by techadventures.net (Postfix, from userid 1000) id 4BFE212322A; Wed, 6 Jun 2018 10:04:08 +0200 (CEST) Date: Wed, 6 Jun 2018 10:04:08 +0200 From: Oscar Salvador To: Naoya Horiguchi Cc: Matthew Wilcox , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , Andrew Morton , Michal Hocko , "mingo@kernel.org" , "dan.j.williams@intel.com" , Huang Ying , Pavel Tatashin Subject: Re: kernel panic in reading /proc/kpageflags when enabling RAM-simulated PMEM Message-ID: <20180606080408.GA31794@techadventures.net> References: <20180605005402.GA22975@hori1.linux.bs1.fc.nec.co.jp> <20180605011836.GA32444@bombadil.infradead.org> <20180605073500.GA23766@hori1.linux.bs1.fc.nec.co.jp> <20180606051624.GA16021@hori1.linux.bs1.fc.nec.co.jp> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20180606051624.GA16021@hori1.linux.bs1.fc.nec.co.jp> User-Agent: Mutt/1.5.23 (2014-03-12) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP On Wed, Jun 06, 2018 at 05:16:24AM +0000, Naoya Horiguchi wrote: > On Tue, Jun 05, 2018 at 07:35:01AM +0000, Horiguchi Naoya(堀口 直也) wrote: > > On Mon, Jun 04, 2018 at 06:18:36PM -0700, Matthew Wilcox wrote: > > > On Tue, Jun 05, 2018 at 12:54:03AM +0000, Naoya Horiguchi wrote: > > > > Reproduction precedure is like this: > > > > - enable RAM based PMEM (with a kernel boot parameter like memmap=1G!4G) > > > > - read /proc/kpageflags (or call tools/vm/page-types with no arguments) > > > > (- my kernel config is attached) > > > > > > > > I spent a few days on this, but didn't reach any solutions. > > > > So let me report this with some details below ... > > > > > > > > In the critial page request, stable_page_flags() is called with an argument > > > > page whose ->compound_head was somehow filled with '0xffffffffffffffff'. > > > > And compound_head() returns (struct page *)(head - 1), which explains the > > > > address 0xfffffffffffffffe in the above message. > > > > > > Hm. compound_head shares with: > > > > > > struct list_head lru; > > > struct list_head slab_list; /* uses lru */ > > > struct { /* Partial pages */ > > > struct page *next; > > > unsigned long _compound_pad_1; /* compound_head */ > > > unsigned long _pt_pad_1; /* compound_head */ > > > struct dev_pagemap *pgmap; > > > struct rcu_head rcu_head; > > > > > > None of them should be -1. > > > > > > > It seems that this kernel panic happens when reading kpageflags of pfn range > > > > [0xbffd7, 0xc0000), which coresponds to a 'reserved' range. > > > > > > > > [ 0.000000] user-defined physical RAM map: > > > > [ 0.000000] user: [mem 0x0000000000000000-0x000000000009fbff] usable > > > > [ 0.000000] user: [mem 0x000000000009fc00-0x000000000009ffff] reserved > > > > [ 0.000000] user: [mem 0x00000000000f0000-0x00000000000fffff] reserved > > > > [ 0.000000] user: [mem 0x0000000000100000-0x00000000bffd6fff] usable > > > > [ 0.000000] user: [mem 0x00000000bffd7000-0x00000000bfffffff] reserved > > > > [ 0.000000] user: [mem 0x00000000feffc000-0x00000000feffffff] reserved > > > > [ 0.000000] user: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved > > > > [ 0.000000] user: [mem 0x0000000100000000-0x000000013fffffff] persistent (type 12) > > > > > > > > So I guess 'memmap=' parameter might badly affect the memory initialization process. > > > > > > > > This problem doesn't reproduce on v4.17, so some pre-released patch introduces it. > > > > I hope this info helps you find the solution/workaround. > > > > > > Can you try bisecting this? It could be one of my patches to reorder struct > > > page, or it could be one of Pavel's deferred page initialisation patches. > > > Or something else ;-) > > > > Thank you for the comment. I'm trying bisecting now, let you know the result later. > > > > And I found that my statement "not reproduce on v4.17" was wrong (I used > > different kvm guests, which made some different test condition and misguided me), > > this seems an older (at least < 4.15) bug. > > (Cc: Pavel) > > Bisection showed that the following commit introduced this issue: > > commit f7f99100d8d95dbcf09e0216a143211e79418b9f > Author: Pavel Tatashin > Date: Wed Nov 15 17:36:44 2017 -0800 > > mm: stop zeroing memory during allocation in vmemmap > > This patch postpones struct page zeroing to later stage of memory initialization. > My kernel config disabled CONFIG_DEFERRED_STRUCT_PAGE_INIT so two callsites of > __init_single_page() were never reached. So in such case, struct pages populated > by vmemmap_pte_populate() could be left uninitialized? > And I'm not sure yet how this issue becomes visible with memmap= setting. I think that this becomes visible because memmap=x!y creates a persistent memory region: parse_memmap_one { ... } else if (*p == '!') { start_at = memparse(p+1, &p); e820__range_add(start_at, mem_size, E820_TYPE_PRAM); ... } and this region it is not added neither in memblock.memory nor in memblock.reserved. Ranges in memblock.memory get zeroed in memmap_init_zone(), while memblock.reserved get zeroed in free_low_memory_core_early(): static unsigned long __init free_low_memory_core_early(void) { ... for_each_reserved_mem_region(i, &start, &end) reserve_bootmem_region(start, end); ... } Maybe I am mistaken, but I think that persistent memory regions should be marked as reserved. A comment in do_mark_busy() suggests this: static bool __init do_mark_busy(enum e820_type type, struct resource *res) { ... /* * Treat persistent memory like device memory, i.e. reserve it * for exclusive use of a driver */ ... } I wonder if something like this could work and if so, if it is right (i haven't tested it yet): Best Regards Oscar Salvador diff --git a/arch/x86/kernel/e820.c b/arch/x86/kernel/e820.c index 71c11ad5643e..3c9686ef74e5 100644 --- a/arch/x86/kernel/e820.c +++ b/arch/x86/kernel/e820.c @@ -1247,6 +1247,11 @@ void __init e820__memblock_setup(void) if (end != (resource_size_t)end) continue; + if (entry->type == E820_TYPE_PRAM || entry->type == E820_TYPE_PMEM) { + memblock_reserve(entry->addr, entry->size); + continue; + } + if (entry->type != E820_TYPE_RAM && entry->type != E820_TYPE_RESERVED_KERN) continue;