From patchwork Thu Sep 20 17:30:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nadav Amit X-Patchwork-Id: 10608343 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E515115E8 for ; Thu, 20 Sep 2018 17:31:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CD6C92E39B for ; Thu, 20 Sep 2018 17:31:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CB8902E303; Thu, 20 Sep 2018 17:31:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C78912E3DB for ; Thu, 20 Sep 2018 17:31:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6CCAB8E0006; Thu, 20 Sep 2018 13:31:51 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5DB3C8E0003; Thu, 20 Sep 2018 13:31:51 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4513C8E0006; Thu, 20 Sep 2018 13:31:51 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f199.google.com (mail-pl1-f199.google.com [209.85.214.199]) by kanga.kvack.org (Postfix) with ESMTP id DC1228E0003 for ; Thu, 20 Sep 2018 13:31:50 -0400 (EDT) Received: by mail-pl1-f199.google.com with SMTP id w11-v6so4674765plq.8 for ; Thu, 20 Sep 2018 10:31:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:mime-version; bh=xQzoQO5W2oX+IozbXrJx+pVdev3tlBnwNPAHg7s+zAo=; b=erdGVChMtpKhKOnLlNpH8xciiEBeKTWHa504Hdx7rD/mSlxa9LPlfEixzRkpqksOCE S9xiaYo9oVE0YjklN9g5DZczYisMP4jQF55DdeVMbtxfpbQSi1A5cHRCqJZ4GYUVSj4p renuW4fFbxdFjtCNodltqp/IP2t4FWoc2IYXvCurOKjABbqNkQY7b98vEhMw8acTvN/u O+jXcCqS8VPQsY8VVk11ffwB70Ql4jZOat/MvQMB8W7YjVcjqFKq6JBbOy+bWT9XeDyS Iy5xqysM3FKK/hJfEGueKWw/rQqRP4iDs695TrlYPMRFoJqafcGnCWc1BFXSQ8gks8RP JbAA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of namit@vmware.com designates 208.91.0.190 as permitted sender) smtp.mailfrom=namit@vmware.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=vmware.com X-Gm-Message-State: APzg51BPVtMxzDRQAwXjtmqJnV2urJubPeSPAasSZf3vKQPYDXNGpzJO nLpCj67Au3pUBcyy2+MCWwIEHS4HNfS0HS8jssHUUr96pmUOFWxAkNBbovTBsLmKzkhIFKKCpnC haTto2KGeun7V2Q5W6UWrsKKhUYlluW2GFbAl7gT6+HYLX0IW5UQTgYZfwAZIZC4ilA== X-Received: by 2002:a17:902:e5:: with SMTP id a92-v6mr40547617pla.273.1537464710550; Thu, 20 Sep 2018 10:31:50 -0700 (PDT) X-Google-Smtp-Source: ANB0VdbqnWR+wpRx3iCHt8nBx6oqtOaE4qK1jFM5hTJCP41cUz+QhOwBfm1TPsCFHDDbZKnGxSQJ X-Received: by 2002:a17:902:e5:: with SMTP id a92-v6mr40547519pla.273.1537464709020; Thu, 20 Sep 2018 10:31:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537464708; cv=none; d=google.com; s=arc-20160816; b=GMRzv4gDg+hqbg6ViMFVT3fNWipafbidg73SGf6Fq15PzCyUrXIP4VfAQsOdrnitya XZzBxnaQABmcTgM4mFd5fuXcHUIa1CeRb7UanGpQ0Og/qav2XEy0F/1C8gydz1EJh56h V/gXNJOAM+kjoFA80+oddvg86pcJFbxjYKNM4eAyU+Wt9RMdbN0w8E6TUSFnyg6JA153 oAo7CIHeWmca/Jy0JcBai/CLD9ExOAk/jnWzwORCXUzqDTGZIo4qvVsxfvKymXcbpqgT yfi+QpiKwPipOPsQYf0c3QHNnLwxcef5JHfhE04z9mJHlB7FYxTszI1ry2WmNe/Io8TV P1Gw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:message-id:date:subject:cc:to:from; bh=xQzoQO5W2oX+IozbXrJx+pVdev3tlBnwNPAHg7s+zAo=; b=mdA2QqTG5DuS/4QzM4HpSoNWppnxBdTJzkAK/fQTCvcV66GyQxMCeWQUkpkz0ImmCv 5qg8gUbwaw0MUP/Zctno6HHu0yzfAahqZvEpZ+IFNU29KXMeYCtDuFqh2ayViizs1U0y 9hp9avTPQCPb4F5T2KePaV0fe9/g/K+GlIdVY6Kz7xmEobjWRSHVuQ95M9d0kyOJfqEv w4iIBbRwbBday9ZV2lWekwUYN/cVlMZQz4nWCH4MA8xnSdCRsUk3LLBoFm3GwfDO4ahi lM1eZHj/nNZHR84epIBv9gvypf81K2hmj5w/ZxJhHOsnTLqEh8O+reKLKal85jpxJs30 QmZQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of namit@vmware.com designates 208.91.0.190 as permitted sender) smtp.mailfrom=namit@vmware.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=vmware.com Received: from EX13-EDG-OU-002.vmware.com (ex13-edg-ou-002.vmware.com. [208.91.0.190]) by mx.google.com with ESMTPS id w3-v6si24769775pgw.135.2018.09.20.10.31.48 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 20 Sep 2018 10:31:48 -0700 (PDT) Received-SPF: pass (google.com: domain of namit@vmware.com designates 208.91.0.190 as permitted sender) client-ip=208.91.0.190; Authentication-Results: mx.google.com; spf=pass (google.com: domain of namit@vmware.com designates 208.91.0.190 as permitted sender) smtp.mailfrom=namit@vmware.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=vmware.com Received: from sc9-mailhost3.vmware.com (10.113.161.73) by EX13-EDG-OU-002.vmware.com (10.113.208.156) with Microsoft SMTP Server id 15.0.1156.6; Thu, 20 Sep 2018 10:31:20 -0700 Received: from sc2-haas01-esx0118.eng.vmware.com (sc2-haas01-esx0118.eng.vmware.com [10.172.44.118]) by sc9-mailhost3.vmware.com (Postfix) with ESMTP id C45B440691; Thu, 20 Sep 2018 10:31:47 -0700 (PDT) From: Nadav Amit To: Greg Kroah-Hartman , Arnd Bergmann CC: , Xavier Deguillard , Nadav Amit , "Michael S. Tsirkin" , Jason Wang , , Subject: [PATCH v2 00/20] vmw_balloon: compaction, shrinker, 64-bit, etc. Date: Thu, 20 Sep 2018 10:30:06 -0700 Message-ID: <20180920173026.141333-1-namit@vmware.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 Received-SPF: None (EX13-EDG-OU-002.vmware.com: namit@vmware.com does not designate permitted sender hosts) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch-set adds the following enhancements to the VMware balloon driver: 1. Balloon compaction support. 2. Report the number of inflated/deflated ballooned pages through vmstat. 3. Memory shrinker to avoid balloon over-inflation (and OOM). 4. Support VMs with memory limit that is greater than 16TB. 5. Faster and more aggressive inflation. To support compaction we wish to use the existing infrastructure. However, we need to make slight adaptions for it. We add a new list interface to balloon-compaction, which is more generic and efficient, since it does not require as many IRQ save/restore operations. We leave the old interface that is used by the virtio balloon. Big parts of this patch-set are cleanup and documentation. Patches 1-13 simplify the balloon code, document its behavior and allow the balloon code to run concurrently. The support for concurrency is required for compaction and the shrinker interface. For documentation we use the kernel-doc format. We are aware that the balloon interface is not public, but following the kernel-doc format may be useful one day. v1->v2: * Fix build error when THP is off (kbuild) * Fix build error on i386 (kbuild) Cc: Xavier Deguillard Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: linux-mm@kvack.org Cc: virtualization@lists.linux-foundation.org Nadav Amit (19): vmw_balloon: handle commands in a single function. vmw_balloon: unify commands tracing and stats vmw_balloon: merge send_lock and send_unlock path vmw_balloon: simplifying batch access vmw_balloon: remove sleeping allocations vmw_balloon: change batch/single lock abstractions vmw_balloon: treat all refused pages equally vmw_balloon: rename VMW_BALLOON_2M_SHIFT to VMW_BALLOON_2M_ORDER vmw_balloon: refactor change size from vmballoon_work vmw_balloon: simplify vmballoon_send_get_target() vmw_balloon: stats rework vmw_balloon: rework the inflate and deflate loops vmw_balloon: general style cleanup vmw_balloon: add reset stat mm/balloon_compaction: suppress allocation warnings mm/balloon_compaction: list interfaces vmw_balloon: compaction support vmw_balloon: memory shrinker vmw_balloon: split refused pages Xavier Deguillard (1): vmw_balloon: support 64-bit memory limit drivers/misc/Kconfig | 1 + drivers/misc/vmw_balloon.c | 2196 +++++++++++++++++++--------- include/linux/balloon_compaction.h | 4 + include/uapi/linux/magic.h | 1 + mm/balloon_compaction.c | 142 +- 5 files changed, 1577 insertions(+), 767 deletions(-)