diff mbox series

xen/balloon: fix page onlining when populating new zone

Message ID 20220406133229.15979-1-jgross@suse.com (mailing list archive)
State New, archived
Headers show
Series xen/balloon: fix page onlining when populating new zone | expand

Commit Message

Jürgen Groß April 6, 2022, 1:32 p.m. UTC
When onlining a new memory page in a guest the Xen balloon driver is
adding it to the ballooned pages instead making it available to be
used immediately. This is meant to enable to add a new upper memory
limit to a guest via hotplugging memory, without having to assign the
new memory in one go.

In case the upper memory limit will be raised above 4G, the new memory
will populate the ZONE_NORMAL memory zone, which wasn't populated
before. The newly populated zone won't be added to the list of zones
looked at by the page allocator though, as only zones with available
memory are being added, and the memory isn't yet available as it is
ballooned out.

This will result in the new memory being assigned to the guest, but
without the allocator being able to use it.

When running as a PV guest the situation is even worse: when having
been started with less memory than allowed, and the upper limit being
lower than 4G, ballooning up will have the same effect as hotplugging
new memory. This is due to the usage of the zone device functionality
since commit 9e2369c06c8a ("xen: add helpers to allocate unpopulated
memory") for creating mappings of other guest's pages, which as a side
effect is being used for PV guest ballooning, too.

Fix this by checking in xen_online_page() whether the new memory page
will be the first in a new zone. If this is the case, add another page
to the balloon and use the first memory page of the new chunk as a
replacement for this now ballooned out page. This will result in the
newly populated zone containing one page being available for the page
allocator, which in turn will lead to the zone being added to the
allocator.

Cc: stable@vger.kernel.org
Fixes: 9e2369c06c8a ("xen: add helpers to allocate unpopulated memory")
Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
---
 drivers/xen/balloon.c | 72 ++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 65 insertions(+), 7 deletions(-)

Comments

kernel test robot April 6, 2022, 11:31 p.m. UTC | #1
Hi Juergen,

I love your patch! Perhaps something to improve:

[auto build test WARNING on xen-tip/linux-next]
[also build test WARNING on linus/master linux/master v5.18-rc1 next-20220406]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/intel-lab-lkp/linux/commits/Juergen-Gross/xen-balloon-fix-page-onlining-when-populating-new-zone/20220407-000935
base:   https://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git linux-next
config: arm64-randconfig-r011-20220406 (https://download.01.org/0day-ci/archive/20220407/202204070706.PKz2Th7L-lkp@intel.com/config)
compiler: aarch64-linux-gcc (GCC) 11.2.0
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # https://github.com/intel-lab-lkp/linux/commit/b3deb59d5386ade4fb227038f202a9bdb8ade4ab
        git remote add linux-review https://github.com/intel-lab-lkp/linux
        git fetch --no-tags linux-review Juergen-Gross/xen-balloon-fix-page-onlining-when-populating-new-zone/20220407-000935
        git checkout b3deb59d5386ade4fb227038f202a9bdb8ade4ab
        # save the config file to linux build tree
        mkdir build_dir
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross O=build_dir ARCH=arm64 SHELL=/bin/bash drivers/xen/

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All warnings (new ones prefixed by >>):

   drivers/xen/balloon.c: In function 'decrease_reservation':
   drivers/xen/balloon.c:518:24: error: implicit declaration of function 'alloc_page_for_balloon' [-Werror=implicit-function-declaration]
     518 |                 page = alloc_page_for_balloon(gfp);
         |                        ^~~~~~~~~~~~~~~~~~~~~~
>> drivers/xen/balloon.c:518:22: warning: assignment to 'struct page *' from 'int' makes pointer from integer without a cast [-Wint-conversion]
     518 |                 page = alloc_page_for_balloon(gfp);
         |                      ^
   drivers/xen/balloon.c:545:17: error: implicit declaration of function 'add_page_to_balloon' [-Werror=implicit-function-declaration]
     545 |                 add_page_to_balloon(page);
         |                 ^~~~~~~~~~~~~~~~~~~
   cc1: some warnings being treated as errors


vim +518 drivers/xen/balloon.c

   505	
   506	static enum bp_state decrease_reservation(unsigned long nr_pages, gfp_t gfp)
   507	{
   508		enum bp_state state = BP_DONE;
   509		unsigned long i;
   510		struct page *page, *tmp;
   511		int ret;
   512		LIST_HEAD(pages);
   513	
   514		if (nr_pages > ARRAY_SIZE(frame_list))
   515			nr_pages = ARRAY_SIZE(frame_list);
   516	
   517		for (i = 0; i < nr_pages; i++) {
 > 518			page = alloc_page_for_balloon(gfp);
   519			if (page == NULL) {
   520				nr_pages = i;
   521				state = BP_EAGAIN;
   522				break;
   523			}
   524			list_add(&page->lru, &pages);
   525		}
   526	
   527		/*
   528		 * Ensure that ballooned highmem pages don't have kmaps.
   529		 *
   530		 * Do this before changing the p2m as kmap_flush_unused()
   531		 * reads PTEs to obtain pages (and hence needs the original
   532		 * p2m entry).
   533		 */
   534		kmap_flush_unused();
   535	
   536		/*
   537		 * Setup the frame, update direct mapping, invalidate P2M,
   538		 * and add to balloon.
   539		 */
   540		i = 0;
   541		list_for_each_entry_safe(page, tmp, &pages, lru) {
   542			frame_list[i++] = xen_page_to_gfn(page);
   543	
   544			list_del(&page->lru);
   545			add_page_to_balloon(page);
   546		}
   547	
   548		flush_tlb_all();
   549	
   550		ret = xenmem_reservation_decrease(nr_pages, frame_list);
   551		BUG_ON(ret != nr_pages);
   552	
   553		balloon_stats.current_pages -= nr_pages;
   554	
   555		return state;
   556	}
   557
kernel test robot April 7, 2022, 2:06 a.m. UTC | #2
Hi Juergen,

I love your patch! Perhaps something to improve:

[auto build test WARNING on xen-tip/linux-next]
[also build test WARNING on linus/master linux/master v5.18-rc1 next-20220406]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/intel-lab-lkp/linux/commits/Juergen-Gross/xen-balloon-fix-page-onlining-when-populating-new-zone/20220407-000935
base:   https://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git linux-next
config: arm64-randconfig-r036-20220406 (https://download.01.org/0day-ci/archive/20220407/202204070950.mzGBYW2q-lkp@intel.com/config)
compiler: clang version 15.0.0 (https://github.com/llvm/llvm-project c4a1b07d0979e7ff20d7d541af666d822d66b566)
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # install arm64 cross compiling tool for clang build
        # apt-get install binutils-aarch64-linux-gnu
        # https://github.com/intel-lab-lkp/linux/commit/b3deb59d5386ade4fb227038f202a9bdb8ade4ab
        git remote add linux-review https://github.com/intel-lab-lkp/linux
        git fetch --no-tags linux-review Juergen-Gross/xen-balloon-fix-page-onlining-when-populating-new-zone/20220407-000935
        git checkout b3deb59d5386ade4fb227038f202a9bdb8ade4ab
        # save the config file to linux build tree
        mkdir build_dir
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=arm64 SHELL=/bin/bash drivers/media/platform/ drivers/xen/

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All warnings (new ones prefixed by >>):

   drivers/xen/balloon.c:518:10: error: implicit declaration of function 'alloc_page_for_balloon' [-Werror,-Wimplicit-function-declaration]
                   page = alloc_page_for_balloon(gfp);
                          ^
>> drivers/xen/balloon.c:518:8: warning: incompatible integer to pointer conversion assigning to 'struct page *' from 'int' [-Wint-conversion]
                   page = alloc_page_for_balloon(gfp);
                        ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~
   drivers/xen/balloon.c:545:3: error: implicit declaration of function 'add_page_to_balloon' [-Werror,-Wimplicit-function-declaration]
                   add_page_to_balloon(page);
                   ^
   1 warning and 2 errors generated.


vim +518 drivers/xen/balloon.c

   505	
   506	static enum bp_state decrease_reservation(unsigned long nr_pages, gfp_t gfp)
   507	{
   508		enum bp_state state = BP_DONE;
   509		unsigned long i;
   510		struct page *page, *tmp;
   511		int ret;
   512		LIST_HEAD(pages);
   513	
   514		if (nr_pages > ARRAY_SIZE(frame_list))
   515			nr_pages = ARRAY_SIZE(frame_list);
   516	
   517		for (i = 0; i < nr_pages; i++) {
 > 518			page = alloc_page_for_balloon(gfp);
   519			if (page == NULL) {
   520				nr_pages = i;
   521				state = BP_EAGAIN;
   522				break;
   523			}
   524			list_add(&page->lru, &pages);
   525		}
   526	
   527		/*
   528		 * Ensure that ballooned highmem pages don't have kmaps.
   529		 *
   530		 * Do this before changing the p2m as kmap_flush_unused()
   531		 * reads PTEs to obtain pages (and hence needs the original
   532		 * p2m entry).
   533		 */
   534		kmap_flush_unused();
   535	
   536		/*
   537		 * Setup the frame, update direct mapping, invalidate P2M,
   538		 * and add to balloon.
   539		 */
   540		i = 0;
   541		list_for_each_entry_safe(page, tmp, &pages, lru) {
   542			frame_list[i++] = xen_page_to_gfn(page);
   543	
   544			list_del(&page->lru);
   545			add_page_to_balloon(page);
   546		}
   547	
   548		flush_tlb_all();
   549	
   550		ret = xenmem_reservation_decrease(nr_pages, frame_list);
   551		BUG_ON(ret != nr_pages);
   552	
   553		balloon_stats.current_pages -= nr_pages;
   554	
   555		return state;
   556	}
   557
kernel test robot April 7, 2022, 5:09 a.m. UTC | #3
Hi Juergen,

I love your patch! Yet something to improve:

[auto build test ERROR on xen-tip/linux-next]
[also build test ERROR on linus/master linux/master v5.18-rc1 next-20220406]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/intel-lab-lkp/linux/commits/Juergen-Gross/xen-balloon-fix-page-onlining-when-populating-new-zone/20220407-000935
base:   https://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git linux-next
config: arm64-randconfig-r011-20220406 (https://download.01.org/0day-ci/archive/20220407/202204071359.uas4tsu0-lkp@intel.com/config)
compiler: aarch64-linux-gcc (GCC) 11.2.0
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # https://github.com/intel-lab-lkp/linux/commit/b3deb59d5386ade4fb227038f202a9bdb8ade4ab
        git remote add linux-review https://github.com/intel-lab-lkp/linux
        git fetch --no-tags linux-review Juergen-Gross/xen-balloon-fix-page-onlining-when-populating-new-zone/20220407-000935
        git checkout b3deb59d5386ade4fb227038f202a9bdb8ade4ab
        # save the config file to linux build tree
        mkdir build_dir
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross O=build_dir ARCH=arm64 SHELL=/bin/bash drivers/

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

   drivers/xen/balloon.c: In function 'decrease_reservation':
>> drivers/xen/balloon.c:518:24: error: implicit declaration of function 'alloc_page_for_balloon' [-Werror=implicit-function-declaration]
     518 |                 page = alloc_page_for_balloon(gfp);
         |                        ^~~~~~~~~~~~~~~~~~~~~~
   drivers/xen/balloon.c:518:22: warning: assignment to 'struct page *' from 'int' makes pointer from integer without a cast [-Wint-conversion]
     518 |                 page = alloc_page_for_balloon(gfp);
         |                      ^
>> drivers/xen/balloon.c:545:17: error: implicit declaration of function 'add_page_to_balloon' [-Werror=implicit-function-declaration]
     545 |                 add_page_to_balloon(page);
         |                 ^~~~~~~~~~~~~~~~~~~
   cc1: some warnings being treated as errors


vim +/alloc_page_for_balloon +518 drivers/xen/balloon.c

   505	
   506	static enum bp_state decrease_reservation(unsigned long nr_pages, gfp_t gfp)
   507	{
   508		enum bp_state state = BP_DONE;
   509		unsigned long i;
   510		struct page *page, *tmp;
   511		int ret;
   512		LIST_HEAD(pages);
   513	
   514		if (nr_pages > ARRAY_SIZE(frame_list))
   515			nr_pages = ARRAY_SIZE(frame_list);
   516	
   517		for (i = 0; i < nr_pages; i++) {
 > 518			page = alloc_page_for_balloon(gfp);
   519			if (page == NULL) {
   520				nr_pages = i;
   521				state = BP_EAGAIN;
   522				break;
   523			}
   524			list_add(&page->lru, &pages);
   525		}
   526	
   527		/*
   528		 * Ensure that ballooned highmem pages don't have kmaps.
   529		 *
   530		 * Do this before changing the p2m as kmap_flush_unused()
   531		 * reads PTEs to obtain pages (and hence needs the original
   532		 * p2m entry).
   533		 */
   534		kmap_flush_unused();
   535	
   536		/*
   537		 * Setup the frame, update direct mapping, invalidate P2M,
   538		 * and add to balloon.
   539		 */
   540		i = 0;
   541		list_for_each_entry_safe(page, tmp, &pages, lru) {
   542			frame_list[i++] = xen_page_to_gfn(page);
   543	
   544			list_del(&page->lru);
 > 545			add_page_to_balloon(page);
   546		}
   547	
   548		flush_tlb_all();
   549	
   550		ret = xenmem_reservation_decrease(nr_pages, frame_list);
   551		BUG_ON(ret != nr_pages);
   552	
   553		balloon_stats.current_pages -= nr_pages;
   554	
   555		return state;
   556	}
   557
David Hildenbrand April 7, 2022, 8:23 a.m. UTC | #4
On 06.04.22 15:32, Juergen Gross wrote:
> When onlining a new memory page in a guest the Xen balloon driver is
> adding it to the ballooned pages instead making it available to be
> used immediately. This is meant to enable to add a new upper memory
> limit to a guest via hotplugging memory, without having to assign the
> new memory in one go.
> 
> In case the upper memory limit will be raised above 4G, the new memory
> will populate the ZONE_NORMAL memory zone, which wasn't populated
> before. The newly populated zone won't be added to the list of zones
> looked at by the page allocator though, as only zones with available
> memory are being added, and the memory isn't yet available as it is
> ballooned out.

I think we just recently discussed these corner cases on the -mm list.
The issue is having effectively populated zones without manages pages
because everything is inflated in a balloon.

That can theoretically also happen when managing to fully inflate the
balloon in one zone and then, somehow, the zones get rebuilt.

build_zonerefs_node() documents "Add all populated zones of a node to
the zonelist" but checks for managed zones, which is wrong.

See https://lkml.kernel.org/r/20220201070044.zbm3obsoimhz3xd3@master

> 
> This will result in the new memory being assigned to the guest, but
> without the allocator being able to use it.
> 
> When running as a PV guest the situation is even worse: when having
> been started with less memory than allowed, and the upper limit being
> lower than 4G, ballooning up will have the same effect as hotplugging
> new memory. This is due to the usage of the zone device functionality
> since commit 9e2369c06c8a ("xen: add helpers to allocate unpopulated
> memory") for creating mappings of other guest's pages, which as a side
> effect is being used for PV guest ballooning, too.
> 
> Fix this by checking in xen_online_page() whether the new memory page
> will be the first in a new zone. If this is the case, add another page
> to the balloon and use the first memory page of the new chunk as a
> replacement for this now ballooned out page. This will result in the
> newly populated zone containing one page being available for the page
> allocator, which in turn will lead to the zone being added to the
> allocator.

This somehow feels like a hack for something that should be handled in
the core instead :/
Jürgen Groß April 7, 2022, 8:50 a.m. UTC | #5
On 07.04.22 10:23, David Hildenbrand wrote:
> On 06.04.22 15:32, Juergen Gross wrote:
>> When onlining a new memory page in a guest the Xen balloon driver is
>> adding it to the ballooned pages instead making it available to be
>> used immediately. This is meant to enable to add a new upper memory
>> limit to a guest via hotplugging memory, without having to assign the
>> new memory in one go.
>>
>> In case the upper memory limit will be raised above 4G, the new memory
>> will populate the ZONE_NORMAL memory zone, which wasn't populated
>> before. The newly populated zone won't be added to the list of zones
>> looked at by the page allocator though, as only zones with available
>> memory are being added, and the memory isn't yet available as it is
>> ballooned out.
> 
> I think we just recently discussed these corner cases on the -mm list.

Indeed.

> The issue is having effectively populated zones without manages pages
> because everything is inflated in a balloon.

Correct.

> That can theoretically also happen when managing to fully inflate the
> balloon in one zone and then, somehow, the zones get rebuilt.

I think you are right. I didn't think of that scenario.

> build_zonerefs_node() documents "Add all populated zones of a node to
> the zonelist" but checks for managed zones, which is wrong.
> 
> See https://lkml.kernel.org/r/20220201070044.zbm3obsoimhz3xd3@master

I found commit 6aa303defb7454 which introduced this test. I thought
it was needed due to the problem this commit tried to solve. Maybe I
was wrong and that commit shouldn't have changed the condition when
building the zonelist, but just the ones in the allocation paths.

> 
>>
>> This will result in the new memory being assigned to the guest, but
>> without the allocator being able to use it.
>>
>> When running as a PV guest the situation is even worse: when having
>> been started with less memory than allowed, and the upper limit being
>> lower than 4G, ballooning up will have the same effect as hotplugging
>> new memory. This is due to the usage of the zone device functionality
>> since commit 9e2369c06c8a ("xen: add helpers to allocate unpopulated
>> memory") for creating mappings of other guest's pages, which as a side
>> effect is being used for PV guest ballooning, too.
>>
>> Fix this by checking in xen_online_page() whether the new memory page
>> will be the first in a new zone. If this is the case, add another page
>> to the balloon and use the first memory page of the new chunk as a
>> replacement for this now ballooned out page. This will result in the
>> newly populated zone containing one page being available for the page
>> allocator, which in turn will lead to the zone being added to the
>> allocator.
> 
> This somehow feels like a hack for something that should be handled in
> the core instead :/

Okay, I'll rework the patch (better wording might be: replace) to switch
build_zonerefs_node() to use populated_zone() instead of managed_zone().


Juergen
David Hildenbrand April 7, 2022, 9 a.m. UTC | #6
On 07.04.22 10:50, Juergen Gross wrote:
> On 07.04.22 10:23, David Hildenbrand wrote:
>> On 06.04.22 15:32, Juergen Gross wrote:
>>> When onlining a new memory page in a guest the Xen balloon driver is
>>> adding it to the ballooned pages instead making it available to be
>>> used immediately. This is meant to enable to add a new upper memory
>>> limit to a guest via hotplugging memory, without having to assign the
>>> new memory in one go.
>>>
>>> In case the upper memory limit will be raised above 4G, the new memory
>>> will populate the ZONE_NORMAL memory zone, which wasn't populated
>>> before. The newly populated zone won't be added to the list of zones
>>> looked at by the page allocator though, as only zones with available
>>> memory are being added, and the memory isn't yet available as it is
>>> ballooned out.
>>
>> I think we just recently discussed these corner cases on the -mm list.
> 
> Indeed.
> 
>> The issue is having effectively populated zones without manages pages
>> because everything is inflated in a balloon.
> 
> Correct.
> 
>> That can theoretically also happen when managing to fully inflate the
>> balloon in one zone and then, somehow, the zones get rebuilt.
> 
> I think you are right. I didn't think of that scenario.
> 
>> build_zonerefs_node() documents "Add all populated zones of a node to
>> the zonelist" but checks for managed zones, which is wrong.
>>
>> See https://lkml.kernel.org/r/20220201070044.zbm3obsoimhz3xd3@master
> 
> I found commit 6aa303defb7454 which introduced this test. I thought
> it was needed due to the problem this commit tried to solve. Maybe I
> was wrong and that commit shouldn't have changed the condition when
> building the zonelist, but just the ones in the allocation paths.

In regard to kswapd, that is currently being worked on via

https://lkml.kernel.org/r/20220329010901.1654-2-richard.weiyang@gmail.com
Wei Yang April 8, 2022, 11:16 p.m. UTC | #7
On Thu, Apr 07, 2022 at 11:00:33AM +0200, David Hildenbrand wrote:
>On 07.04.22 10:50, Juergen Gross wrote:
>> On 07.04.22 10:23, David Hildenbrand wrote:
>>> On 06.04.22 15:32, Juergen Gross wrote:
>>>> When onlining a new memory page in a guest the Xen balloon driver is
>>>> adding it to the ballooned pages instead making it available to be
>>>> used immediately. This is meant to enable to add a new upper memory
>>>> limit to a guest via hotplugging memory, without having to assign the
>>>> new memory in one go.
>>>>
>>>> In case the upper memory limit will be raised above 4G, the new memory
>>>> will populate the ZONE_NORMAL memory zone, which wasn't populated
>>>> before. The newly populated zone won't be added to the list of zones
>>>> looked at by the page allocator though, as only zones with available
>>>> memory are being added, and the memory isn't yet available as it is
>>>> ballooned out.
>>>
>>> I think we just recently discussed these corner cases on the -mm list.
>> 
>> Indeed.
>> 
>>> The issue is having effectively populated zones without manages pages
>>> because everything is inflated in a balloon.
>> 
>> Correct.
>> 
>>> That can theoretically also happen when managing to fully inflate the
>>> balloon in one zone and then, somehow, the zones get rebuilt.
>> 
>> I think you are right. I didn't think of that scenario.
>> 
>>> build_zonerefs_node() documents "Add all populated zones of a node to
>>> the zonelist" but checks for managed zones, which is wrong.
>>>
>>> See https://lkml.kernel.org/r/20220201070044.zbm3obsoimhz3xd3@master
>> 
>> I found commit 6aa303defb7454 which introduced this test. I thought
>> it was needed due to the problem this commit tried to solve. Maybe I
>> was wrong and that commit shouldn't have changed the condition when
>> building the zonelist, but just the ones in the allocation paths.
>
>In regard to kswapd, that is currently being worked on via
>
>https://lkml.kernel.org/r/20220329010901.1654-2-richard.weiyang@gmail.com
>

Thanks, David

Do you think it is the right time to repost the original fix?

>-- 
>Thanks,
>
>David / dhildenb
diff mbox series

Patch

diff --git a/drivers/xen/balloon.c b/drivers/xen/balloon.c
index dfe26fa17e95..f895c54c4c65 100644
--- a/drivers/xen/balloon.c
+++ b/drivers/xen/balloon.c
@@ -355,14 +355,77 @@  static enum bp_state reserve_additional_memory(void)
 	return BP_ECANCELED;
 }
 
+static struct page *alloc_page_for_balloon(gfp_t gfp)
+{
+	struct page *page;
+
+	page = alloc_page(gfp);
+	if (page == NULL)
+		return NULL;
+
+	adjust_managed_page_count(page, -1);
+	xenmem_reservation_scrub_page(page);
+
+	return page;
+}
+
+static void add_page_to_balloon(struct page *page)
+{
+	xenmem_reservation_va_mapping_reset(1, &page);
+	balloon_append(page);
+}
+
 static void xen_online_page(struct page *page, unsigned int order)
 {
 	unsigned long i, size = (1 << order);
 	unsigned long start_pfn = page_to_pfn(page);
 	struct page *p;
+	struct zone *zone;
 
 	pr_debug("Online %lu pages starting at pfn 0x%lx\n", size, start_pfn);
 	mutex_lock(&balloon_mutex);
+	zone = page_zone(pfn_to_page(start_pfn));
+
+	/*
+	 * In case a new memory zone is going to be populated, we need to
+	 * ensure at least one page is made available for the memory allocator.
+	 * As the number of pages per zone is updated only after a batch of
+	 * pages having been added, use the number of managed pages as an
+	 * additional indicator for a new zone.
+	 * Otherwise this zone won't be added to the zonelist resulting in the
+	 * zone's memory not usable by the kernel.
+	 * Add an already valid page to the balloon and replace it with the
+	 * first page of the to be added new memory chunk.
+	 */
+	if (!populated_zone(zone) && !managed_zone(zone)) {
+		xen_pfn_t frame;
+
+		pr_info("Populating new zone\n");
+
+		p = alloc_page_for_balloon(GFP_ATOMIC);
+		if (!p) {
+			pr_err("Failed to allocate replacement balloon page!\n");
+			pr_err("New onlined memory might not be usable.\n");
+		} else {
+			kmap_flush_unused();
+			add_page_to_balloon(p);
+			flush_tlb_all();
+			frame = xen_page_to_gfn(p);
+			xenmem_reservation_decrease(1, &frame);
+			balloon_stats.current_pages--;
+		}
+
+		p = pfn_to_page(start_pfn);
+		frame = page_to_xen_pfn(p);
+		if (xenmem_reservation_increase(1, &frame) > 0) {
+			xenmem_reservation_va_mapping_update(1, &p, &frame);
+			free_reserved_page(p);
+			balloon_stats.current_pages++;
+
+			start_pfn++;
+			size--;
+		}
+	}
 	for (i = 0; i < size; i++) {
 		p = pfn_to_page(start_pfn + i);
 		balloon_append(p);
@@ -452,14 +515,12 @@  static enum bp_state decrease_reservation(unsigned long nr_pages, gfp_t gfp)
 		nr_pages = ARRAY_SIZE(frame_list);
 
 	for (i = 0; i < nr_pages; i++) {
-		page = alloc_page(gfp);
+		page = alloc_page_for_balloon(gfp);
 		if (page == NULL) {
 			nr_pages = i;
 			state = BP_EAGAIN;
 			break;
 		}
-		adjust_managed_page_count(page, -1);
-		xenmem_reservation_scrub_page(page);
 		list_add(&page->lru, &pages);
 	}
 
@@ -480,11 +541,8 @@  static enum bp_state decrease_reservation(unsigned long nr_pages, gfp_t gfp)
 	list_for_each_entry_safe(page, tmp, &pages, lru) {
 		frame_list[i++] = xen_page_to_gfn(page);
 
-		xenmem_reservation_va_mapping_reset(1, &page);
-
 		list_del(&page->lru);
-
-		balloon_append(page);
+		add_page_to_balloon(page);
 	}
 
 	flush_tlb_all();