From patchwork Wed Sep 26 19:13:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nadav Amit X-Patchwork-Id: 10616563 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 461F1175A for ; Wed, 26 Sep 2018 19:15:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3676B2A099 for ; Wed, 26 Sep 2018 19:15:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2A1372A39C; Wed, 26 Sep 2018 19:15:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 667D42A38E for ; Wed, 26 Sep 2018 19:15:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 103408E0003; Wed, 26 Sep 2018 15:15:12 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 09BC68E0001; Wed, 26 Sep 2018 15:15:12 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E94578E0005; Wed, 26 Sep 2018 15:15:11 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by kanga.kvack.org (Postfix) with ESMTP id A842E8E0001 for ; Wed, 26 Sep 2018 15:15:11 -0400 (EDT) Received: by mail-pf1-f198.google.com with SMTP id n17-v6so50568pff.17 for ; Wed, 26 Sep 2018 12:15:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:mime-version; bh=Ez2vr+zejNJ0EFWq87QYQHZXseFFKNLw+kXV03IWx5g=; b=oLoRSG6j7y0CIEoq9a2T2u+iLJPiC8fViMEGsb9GdU7uqMap9qLgaLtQ8wox07aAke jadqaMSrksJbGM5WoWvbUafXONIR2MWaG8hJEhGh2M+2owwmIAC2MRvc5zi7X8ICZyug 0O/tlFbQEn0Ck8cTMoAqz5R01Y0Kpfcl9FV6a9ib1ATaxb3GeSHRFUMgd8hQFudZE9xW akr7xFYc0IMNuPSKKAgW7xUhTMmSsq1QHo3MV+De1rwOq3KQrRnihL5aevY7TPar4yiT /WmVkqX8WKi58c7Oh5obKm1Z76XAOKzV0RYGgMV2ykbrLYHVi2InIestw/f0ovHnPGxF OhrQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of namit@vmware.com designates 208.91.0.190 as permitted sender) smtp.mailfrom=namit@vmware.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=vmware.com X-Gm-Message-State: ABuFfoiwTn9LMBBLpjYcM1yQwqm3YndY5pXoqa6Qdp1wwE5rEIB8eRoV /7NXpH0rVVAi3pjYUEJCRRWtdqOqprLJHyNj823TcyproQnEP8Jjaacur5wHOC8vmJz+stcN9U/ 6rV93EAihGbnNsulVtztkkLNJ5RtDHIhBDH1HT+xpWZK77/t82MwnPcj39NSBlCRaMw== X-Received: by 2002:a17:902:8f93:: with SMTP id z19-v6mr7283148plo.263.1537989311323; Wed, 26 Sep 2018 12:15:11 -0700 (PDT) X-Google-Smtp-Source: ACcGV6066vVysikVp5OL6YPcZyaWtg9QjtyLkhu9Lwmi6846ZcLmqkRW3ot+P3GY3lxaqkc4/ndp X-Received: by 2002:a17:902:8f93:: with SMTP id z19-v6mr7283087plo.263.1537989310320; Wed, 26 Sep 2018 12:15:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537989310; cv=none; d=google.com; s=arc-20160816; b=0V+q39mKmlBm83vrENQgOM1FnGYMVOkEhP1CqEF9jyJOc24tkaORdJe0LIzME9R68F 6m15IdYjcm3f/f/rF1Zr42PYxXFH8npJE3v0M3i0Ddx804s+xk3cfLS8DJbO5ZIdxAyy t0SgGM4N7NOkWpYcOfddSalOQEzGuBGAALcZJyeAj3ZbdtUNfeN80D2J4lUKll9OYvD/ h9SAUmEaGA/0BzZT3lfVMKr/uPiz22l2uPjUxoHpbfLCzXb2U6rQVxF1ECSKbqbQsghd HIuqy41Suph0gmjeOU+mzmmJ63QbO7lPkIZGcqsUuUMe7FK8lXEbg/11OJjuPX33tqDf bFdQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:message-id:date:subject:cc:to:from; bh=Ez2vr+zejNJ0EFWq87QYQHZXseFFKNLw+kXV03IWx5g=; b=QV1x961RpA4nrFVKPD8/LDW8pXOro88bil7ZooeunbVW51kjCV4T6jHQm8y7LLgj/G PPDlUNxDXA0X6UHDW3YTt1S4Z0W1ovQ9d7B/frCQVpt9zrcI3pdtjEsP+MCsK0CstUUu 3ehU4sVFSuKrrZnJwHGaHfGJKLZB2reiw2nHopM1hQJIc12iO9yuKdWv71uYKA5essMB 2eeT1CuB92mnWK9Z77MnOVA9W8eryDCZMRxE6c7VjYKAQftuk3fqKNiEXIuCNbTdgpqT lsByIC0WNJ4xrpvApGh0HMu+NbC+364iEO97gzjhr1XY1DB4+wtVmXb22ScJoVo1ambb xMuQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of namit@vmware.com designates 208.91.0.190 as permitted sender) smtp.mailfrom=namit@vmware.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=vmware.com Received: from EX13-EDG-OU-002.vmware.com (ex13-edg-ou-002.vmware.com. [208.91.0.190]) by mx.google.com with ESMTPS id q90-v6si5539344pfa.272.2018.09.26.12.15.10 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 26 Sep 2018 12:15:10 -0700 (PDT) Received-SPF: pass (google.com: domain of namit@vmware.com designates 208.91.0.190 as permitted sender) client-ip=208.91.0.190; Authentication-Results: mx.google.com; spf=pass (google.com: domain of namit@vmware.com designates 208.91.0.190 as permitted sender) smtp.mailfrom=namit@vmware.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=vmware.com Received: from sc9-mailhost2.vmware.com (10.113.161.72) by EX13-EDG-OU-002.vmware.com (10.113.208.156) with Microsoft SMTP Server id 15.0.1156.6; Wed, 26 Sep 2018 12:15:04 -0700 Received: from sc2-haas01-esx0118.eng.vmware.com (sc2-haas01-esx0118.eng.vmware.com [10.172.44.118]) by sc9-mailhost2.vmware.com (Postfix) with ESMTP id 427DFB05DE; Wed, 26 Sep 2018 15:15:09 -0400 (EDT) From: Nadav Amit To: Arnd Bergmann , CC: Xavier Deguillard , , Nadav Amit , "Michael S. Tsirkin" , Jason Wang , , Subject: [PATCH v3 00/20] vmw_balloon: compaction, shrinker, 64-bit, etc. Date: Wed, 26 Sep 2018 12:13:16 -0700 Message-ID: <20180926191336.101885-1-namit@vmware.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 Received-SPF: None (EX13-EDG-OU-002.vmware.com: namit@vmware.com does not designate permitted sender hosts) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch-set adds the following enhancements to the VMware balloon driver: 1. Balloon compaction support. 2. Report the number of inflated/deflated ballooned pages through vmstat. 3. Memory shrinker to avoid balloon over-inflation (and OOM). 4. Support VMs with memory limit that is greater than 16TB. 5. Faster and more aggressive inflation. To support compaction we wish to use the existing infrastructure. However, we need to make slight adaptions for it. We add a new list interface to balloon-compaction, which is more generic and efficient, since it does not require as many IRQ save/restore operations. We leave the old interface that is used by the virtio balloon. Big parts of this patch-set are cleanup and documentation. Patches 1-13 simplify the balloon code, document its behavior and allow the balloon code to run concurrently. The support for concurrency is required for compaction and the shrinker interface. For documentation we use the kernel-doc format. We are aware that the balloon interface is not public, but following the kernel-doc format may be useful one day. v2->v3: * Moving the balloon magic-number out of uapi (Greg) v1->v2: * Fix build error when THP is off (kbuild) * Fix build error on i386 (kbuild) Cc: Xavier Deguillard Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: linux-mm@kvack.org Cc: virtualization@lists.linux-foundation.org Nadav Amit (19): vmw_balloon: handle commands in a single function. vmw_balloon: unify commands tracing and stats vmw_balloon: merge send_lock and send_unlock path vmw_balloon: simplifying batch access vmw_balloon: remove sleeping allocations vmw_balloon: change batch/single lock abstractions vmw_balloon: treat all refused pages equally vmw_balloon: rename VMW_BALLOON_2M_SHIFT to VMW_BALLOON_2M_ORDER vmw_balloon: refactor change size from vmballoon_work vmw_balloon: simplify vmballoon_send_get_target() vmw_balloon: stats rework vmw_balloon: rework the inflate and deflate loops vmw_balloon: general style cleanup vmw_balloon: add reset stat mm/balloon_compaction: suppress allocation warnings mm/balloon_compaction: list interfaces vmw_balloon: compaction support vmw_balloon: memory shrinker vmw_balloon: split refused pages Xavier Deguillard (1): vmw_balloon: support 64-bit memory limit drivers/misc/Kconfig | 1 + drivers/misc/vmw_balloon.c | 2198 +++++++++++++++++++--------- include/linux/balloon_compaction.h | 4 + mm/balloon_compaction.c | 142 +- 4 files changed, 1578 insertions(+), 767 deletions(-)