From patchwork Tue Jun 12 14:39:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kirill A . Shutemov" X-Patchwork-Id: 10460505 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 2ED146020F for ; Tue, 12 Jun 2018 14:39:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 28063287AB for ; Tue, 12 Jun 2018 14:39:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1C88F287E3; Tue, 12 Jun 2018 14:39:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 29018287AB for ; Tue, 12 Jun 2018 14:39:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 266F66B0010; Tue, 12 Jun 2018 10:39:29 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 259A16B000E; Tue, 12 Jun 2018 10:39:29 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 05EC86B0269; Tue, 12 Jun 2018 10:39:28 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl0-f69.google.com (mail-pl0-f69.google.com [209.85.160.69]) by kanga.kvack.org (Postfix) with ESMTP id B67BA6B000D for ; Tue, 12 Jun 2018 10:39:28 -0400 (EDT) Received: by mail-pl0-f69.google.com with SMTP id q19-v6so14056133plr.22 for ; Tue, 12 Jun 2018 07:39:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=+cvK/InyhjsFStFXxa2lUeGqoVu1PaEPorfN+UkZVIY=; b=d2EVJR280ebSXGgMTpZTjX03S3NykmYvcLZiPXCAOB1eGWIazFk9f2GrPg3yceIoQz SLbw4n8NO+stcZHBoBUG9CyBXjp8urCIdZfTjpZkpJZOb7HM+L4L5qOCbWMjxWEBCX97 E4lgAYjSVhmpB+NGGmU8T/SjykpyyF9VN+6cP2gKKUxEE+Hdrw4uKyw3rLWrNgHA0m4E qVZlscjxhP2pEHlRwuJYoFjB56KMyl8456L6tRPK9xC06tpHkW6/LxJXM3YcqwtIKlxu KW6c7eu2j1tzFoi41SZHYL0PZ4hYAh4abVnXPZAZjhMcega4xZzcLhXPbtN3H8UPpNPX 2PDA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.136 as permitted sender) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: APt69E0wY4EoKHVZs1dKaa4Byj8yNTVO3JFMxs0ZGKK41JMsBPxzs5Hy vth/U06VY6YAMLmgUafsr+6X9olnXG1rVM01JFzt+r68klsNd/P4OZTSjaXIof61R48gXgiFSAS zWp4G7/NIe1wU4qTSygpSulinMmXYsePQGFXFK6IrGAVaUKNtMbjo/TEh1q26Bo3JwA== X-Received: by 2002:a62:574d:: with SMTP id l74-v6mr691605pfb.29.1528814368432; Tue, 12 Jun 2018 07:39:28 -0700 (PDT) X-Google-Smtp-Source: ADUXVKIjMVWXP0rGPfNOJdrARoxJ2tyrbaQCRJhbs8qEz4tr+VBIAlkmghoIhdmA9OhiLwF/zYqL X-Received: by 2002:a62:574d:: with SMTP id l74-v6mr691569pfb.29.1528814367628; Tue, 12 Jun 2018 07:39:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1528814367; cv=none; d=google.com; s=arc-20160816; b=o4JhNgbNLQf+wIYdWGF0aN4XYwtBXhugda3HcdSxJTSpMktGZGiiI+9tAYOF7k+feB o0eVfytda37+/jgEjOUDpMAPxQC590IVV2Z08A0QaYbRJRPiUEfQdH0eY87Kqo+yQB97 udVw3Wg/iQE8EbEfX9zR8vA/hDMDbuZ3XhjTMKQi/v1eGs5kPwHNXgBAadlWEXo7AUaM G7YItPaCz7I8pFwOqADpapBF2wTyTPZPuZOYC6ITKwIYMTqyx1Og5nO5Z0OEwTSbr933 Geo34aJ0/qU7fLkR18hlX5hHyFeW38xLvesjk5eY44FGpCeXICrIYIm7bej/8r7soNZM 7F+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=+cvK/InyhjsFStFXxa2lUeGqoVu1PaEPorfN+UkZVIY=; b=F9KCLeMLgcjEdHcMHJXOyoi7OCa4VipEeWvZCvsd0WoMyxxJ5Dk+Zi/bSjp7CgOYhO bPMUdtKGkkho5OKk5JBsFQXytW1wyaW/7VsFTSVg0OH+A5EDfN0ecD2AC9oOSMmQ6fvm w/fVCLi8ZrD13jB6jKcWZ2uDHM/snGAGKdqNQkJBidJE897LTb32hG4njHRAFKwYOc/b chAVpWDyUoZ8yXBX36KVSRw1pEN4PbtaTNLd2dOuZFIcQiJC4H/H0f8LEuD9DP18H9aD KDvTZX3S27oNOBTtQT3xZS8wenab391T3WPhqoakWBkfsxfru9Zm+kDrJdABP/tcYq/B 4mJQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.136 as permitted sender) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga12.intel.com (mga12.intel.com. [192.55.52.136]) by mx.google.com with ESMTPS id d191-v6si257904pga.192.2018.06.12.07.39.27 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Jun 2018 07:39:27 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.136 as permitted sender) client-ip=192.55.52.136; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.136 as permitted sender) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 12 Jun 2018 07:39:27 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,215,1526367600"; d="scan'208";a="56706818" Received: from black.fi.intel.com ([10.237.72.28]) by FMSMGA003.fm.intel.com with ESMTP; 12 Jun 2018 07:39:24 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id 13AB4477; Tue, 12 Jun 2018 17:39:21 +0300 (EEST) From: "Kirill A. Shutemov" To: Ingo Molnar , x86@kernel.org, Thomas Gleixner , "H. Peter Anvin" , Tom Lendacky Cc: Dave Hansen , Kai Huang , Jacob Pan , linux-kernel@vger.kernel.org, linux-mm@kvack.org, "Kirill A. Shutemov" Subject: [PATCHv3 10/17] x86/mm: Implement prep_encrypted_page() and arch_free_page() Date: Tue, 12 Jun 2018 17:39:08 +0300 Message-Id: <20180612143915.68065-11-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180612143915.68065-1-kirill.shutemov@linux.intel.com> References: <20180612143915.68065-1-kirill.shutemov@linux.intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The hardware/CPU does not enforce coherency between mappings of the same physical page with different KeyIDs or encryption keys. We are responsible for cache management. Flush cache on allocating encrypted page and on returning the page to the free pool. prep_encrypted_page() also takes care about zeroing the page. We have to do this after KeyID is set for the page. Signed-off-by: Kirill A. Shutemov --- arch/x86/include/asm/mktme.h | 6 ++++++ arch/x86/mm/mktme.c | 39 ++++++++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+) diff --git a/arch/x86/include/asm/mktme.h b/arch/x86/include/asm/mktme.h index 0fe0db424e48..ec7036abdb3f 100644 --- a/arch/x86/include/asm/mktme.h +++ b/arch/x86/include/asm/mktme.h @@ -11,6 +11,12 @@ extern phys_addr_t mktme_keyid_mask; extern int mktme_nr_keyids; extern int mktme_keyid_shift; +#define prep_encrypted_page prep_encrypted_page +void prep_encrypted_page(struct page *page, int order, int keyid, bool zero); + +#define HAVE_ARCH_FREE_PAGE +void arch_free_page(struct page *page, int order); + #define vma_is_encrypted vma_is_encrypted bool vma_is_encrypted(struct vm_area_struct *vma); diff --git a/arch/x86/mm/mktme.c b/arch/x86/mm/mktme.c index b02d5b9d4339..1821b87abb2f 100644 --- a/arch/x86/mm/mktme.c +++ b/arch/x86/mm/mktme.c @@ -1,4 +1,5 @@ #include +#include #include phys_addr_t mktme_keyid_mask; @@ -30,6 +31,44 @@ int vma_keyid(struct vm_area_struct *vma) return (prot & mktme_keyid_mask) >> mktme_keyid_shift; } +void prep_encrypted_page(struct page *page, int order, int keyid, bool zero) +{ + int i; + + /* + * The hardware/CPU does not enforce coherency between mappings of the + * same physical page with different KeyIDs or encrypt ion keys. + * We are responsible for cache management. + * + * We flush cache before allocating encrypted page + */ + clflush_cache_range(page_address(page), PAGE_SIZE << order); + + for (i = 0; i < (1 << order); i++) { + WARN_ON_ONCE(lookup_page_ext(page)->keyid); + lookup_page_ext(page)->keyid = keyid; + + /* Clear the page after the KeyID is set. */ + if (zero) + clear_highpage(page); + } +} + +void arch_free_page(struct page *page, int order) +{ + int i; + + if (!page_keyid(page)) + return; + + for (i = 0; i < (1 << order); i++) { + WARN_ON_ONCE(lookup_page_ext(page)->keyid > mktme_nr_keyids); + lookup_page_ext(page)->keyid = 0; + } + + clflush_cache_range(page_address(page), PAGE_SIZE << order); +} + static bool need_page_mktme(void) { /* Make sure keyid doesn't collide with extended page flags */