From patchwork Mon Nov 16 00:59:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907059 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C0D6F138B for ; Mon, 16 Nov 2020 01:02:24 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A379222245 for ; Mon, 16 Nov 2020 01:02:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A379222245 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id AD62021FDD0; Sun, 15 Nov 2020 17:01:45 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 61941308154 for ; Sun, 15 Nov 2020 17:00:15 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 8AF32236C; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 891E82C802; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:59 -0500 Message-Id: <1605488401-981-27-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 26/28] lustre: sec: encryption with different client PAGE_SIZE X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Sebastien Buisson In order to properly handle encryption/decryption on clients that have a PAGE_SIZE != LUSTRE_ENCRYPTION_UNIT_SIZE (typically aarch64/ppc64), a few adjustements are necessary: - when encrypting, do not proceed with PAGE_SIZE as encryption length. Instead, round up to a multiple of LUSTRE_ENCRYPTION_UNIT_SIZE. On aarch64/ppc64, it avoids encrypting way beyond LUSTRE_ENCRYPTION_UNIT_SIZE when the page is not full. - when decrypting, do not proceed with PAGE_SIZE as decryption length. Instead, do LUSTRE_ENCRYPTION_UNIT_SIZE length at a time. It enables proper detection of 'all 0s' sent by servers for content beyond file size. Regarding tests, add sanity-sec test_53 to exercise encryption from clients with different PAGE_SIZE. The trick to achieve this with AT is to expect the client to have 64KB PAGE_SIZE, and the servers to have 4KB PAGE_SIZE, and then mount a client from the MDS node. This also means code running on server side needs to have client encryption support enabled, so CentOS/RHEL 8 at least. WC-bug-id: https://jira.whamcloud.com/browse/LU-12275 Lustre-commit: ac5fcdce025b4 ("LU-12275 sec: encryption with different client PAGE_SIZE") Signed-off-by: Sebastien Buisson Reviewed-on: https://review.whamcloud.com/39315 Reviewed-by: Andreas Dilger Reviewed-by: Wang Shilong Signed-off-by: James Simmons --- fs/lustre/llite/file.c | 28 +++++++++++----- fs/lustre/osc/osc_request.c | 79 +++++++++++++++++++++++++++------------------ 2 files changed, 68 insertions(+), 39 deletions(-) diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 02cc2d6..f7f917b 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -444,16 +444,28 @@ static inline int ll_dom_readpage(void *data, struct page *page) kunmap_atomic(kaddr); if (inode && IS_ENCRYPTED(inode) && S_ISREG(inode->i_mode)) { - if (!llcrypt_has_encryption_key(inode)) + if (!llcrypt_has_encryption_key(inode)) { CDEBUG(D_SEC, "no enc key for " DFID "\n", PFID(ll_inode2fid(inode))); - /* decrypt only if page is not empty */ - else if (memcmp(page_address(page), - page_address(ZERO_PAGE(0)), - PAGE_SIZE) != 0) - rc = llcrypt_decrypt_pagecache_blocks(page, - PAGE_SIZE, - 0); + } else { + unsigned int offs = 0; + + while (offs < PAGE_SIZE) { + /* decrypt only if page is not empty */ + if (memcmp(page_address(page) + offs, + page_address(ZERO_PAGE(0)), + LUSTRE_ENCRYPTION_UNIT_SIZE) == 0) + break; + + rc = llcrypt_decrypt_pagecache_blocks(page, + LUSTRE_ENCRYPTION_UNIT_SIZE, + 0); + if (rc) + break; + + offs += LUSTRE_ENCRYPTION_UNIT_SIZE; + } + } } unlock_page(page); diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index bf9ce44..746b695 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -1421,8 +1421,12 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli, struct page *data_page = NULL; bool retried = false; bool lockedbymyself; + u32 nunits = (pg->off & ~PAGE_MASK) + pg->count; retry_encrypt: + if (nunits & ~LUSTRE_ENCRYPTION_MASK) + nunits = (nunits & LUSTRE_ENCRYPTION_MASK) + + LUSTRE_ENCRYPTION_UNIT_SIZE; /* The page can already be locked when we arrive here. * This is possible when cl_page_assume/vvp_page_assume * is stuck on wait_on_page_writeback with page lock @@ -1435,7 +1439,7 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli, lockedbymyself = trylock_page(pg->pg); data_page = llcrypt_encrypt_pagecache_blocks(pg->pg, - PAGE_SIZE, 0, + nunits, 0, GFP_NOFS); if (lockedbymyself) unlock_page(pg->pg); @@ -1458,24 +1462,29 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli, oap->oap_obj_off + oap->oap_page_off; } - /* len is forced to PAGE_SIZE, and poff to 0 + /* len is forced to nunits, and relative offset to 0 * so store the old, clear text info */ - pg->bp_count_diff = PAGE_SIZE - pg->count; - pg->count = PAGE_SIZE; + pg->bp_count_diff = nunits - pg->count; + pg->count = nunits; pg->bp_off_diff = pg->off & ~PAGE_MASK; pg->off = pg->off & PAGE_MASK; } } else if (opc == OST_READ && inode && IS_ENCRYPTED(inode)) { for (i = 0; i < page_count; i++) { struct brw_page *pg = pga[i]; - - /* count/off are forced to cover the whole page so that - * all encrypted data is stored on the OST, so adjust - * bp_{count,off}_diff for the size of the clear text. + u32 nunits = (pg->off & ~PAGE_MASK) + pg->count; + + if (nunits & ~LUSTRE_ENCRYPTION_MASK) + nunits = (nunits & LUSTRE_ENCRYPTION_MASK) + + LUSTRE_ENCRYPTION_UNIT_SIZE; + /* count/off are forced to cover the whole encryption + * unit size so that all encrypted data is stored on the + * OST, so adjust bp_{count,off}_diff for the size of + * the clear text. */ - pg->bp_count_diff = PAGE_SIZE - pg->count; - pg->count = PAGE_SIZE; + pg->bp_count_diff = nunits - pg->count; + pg->count = nunits; pg->bp_off_diff = pg->off & ~PAGE_MASK; pg->off = pg->off & PAGE_MASK; } @@ -2096,30 +2105,38 @@ static int osc_brw_fini_request(struct ptlrpc_request *req, int rc) } for (idx = 0; idx < aa->aa_page_count; idx++) { struct brw_page *pg = aa->aa_ppga[idx]; + unsigned int offs = 0; + + while (offs < PAGE_SIZE) { + /* do not decrypt if page is all 0s */ + if (memchr_inv(page_address(pg->pg) + offs, 0, + LUSTRE_ENCRYPTION_UNIT_SIZE) == NULL) { + /* if page is empty forward info to + * upper layers (ll_io_zero_page) by + * clearing PagePrivate2 + */ + if (!offs) + ClearPagePrivate2(pg->pg); + break; + } - /* do not decrypt if page is all 0s */ - if (memchr_inv(page_address(pg->pg), 0, - PAGE_SIZE) == NULL) { - /* if page is empty forward info to upper layers - * (ll_io_zero_page) by clearing PagePrivate2 + /* The page is already locked when we arrive + * here, except when we deal with a twisted + * page for specific Direct IO support, in + * which case PageChecked flag is set on page. */ - ClearPagePrivate2(pg->pg); - continue; + if (PageChecked(pg->pg)) + lock_page(pg->pg); + rc = llcrypt_decrypt_pagecache_blocks(pg->pg, + LUSTRE_ENCRYPTION_UNIT_SIZE, + offs); + if (PageChecked(pg->pg)) + unlock_page(pg->pg); + if (rc) + goto out; + + offs += LUSTRE_ENCRYPTION_UNIT_SIZE; } - - /* The page is already locked when we arrive here, - * except when we deal with a twisted page for - * specific Direct IO support, in which case - * PageChecked flag is set on page. - */ - if (PageChecked(pg->pg)) - lock_page(pg->pg); - rc = llcrypt_decrypt_pagecache_blocks(pg->pg, - PAGE_SIZE, 0); - if (PageChecked(pg->pg)) - unlock_page(pg->pg); - if (rc) - goto out; } }