| Message ID | 20230613052424.254540-1-xiubli@redhat.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | ceph+fscrypt: full support | expand |
Looks good to me. Reviewed-by: Milind Changire <mchangir@redhat.com> On Tue, Jun 13, 2023 at 10:56 AM <xiubli@redhat.com> wrote: > > From: Xiubo Li <xiubli@redhat.com> > > > This patch series is based on Jeff Layton's previous great work and effort > on this and all the patches have been in the testing branch and well tested: > > The qa teuthology tests results: > https://pulpito.ceph.com/xiubli-2023-06-09_03:59:14-fs:fscrypt-wip-lxb-fscrypt-20230607-0901-distro-default-smithi/ > > > Since v15 we have added the ceph qa teuthology test cases for this [1][2], > which will test both the file name and contents encryption features and at > the same time they will also test the IO benchmarks. > > To support the fscrypt we also have some other work in ceph [3][4][5][6][7][8][9]: > > [1] https://github.com/ceph/ceph/pull/48628 > [2] https://github.com/ceph/ceph/pull/49934 > [3] https://github.com/ceph/ceph/pull/43588 > [4] https://github.com/ceph/ceph/pull/37297 > [5] https://github.com/ceph/ceph/pull/45192 > [6] https://github.com/ceph/ceph/pull/45312 > [7] https://github.com/ceph/ceph/pull/40828 > [8] https://github.com/ceph/ceph/pull/45224 > [9] https://github.com/ceph/ceph/pull/45073 > > The [9] is still undering testing and will soon be merged after that. All > the others had been merged. > > > The main changes since v19: > > - s/ceph_open/ceph_lookup/ for [PATCH v20 70/71]. > - Updated the patches to fix use-after-free bug for fscrypt_destroy_keyring, > the link https://patchwork.kernel.org/project/ceph-devel/list/?series=755131. > These new changes have been > > > > Jeff Layton (47): > libceph: add spinlock around osd->o_requests > libceph: define struct ceph_sparse_extent and add some helpers > libceph: add sparse read support to msgr2 crc state machine > libceph: add sparse read support to OSD client > libceph: support sparse reads on msgr2 secure codepath > libceph: add sparse read support to msgr1 > ceph: add new mount option to enable sparse reads > ceph: preallocate inode for ops that may create one > ceph: make ceph_msdc_build_path use ref-walk > libceph: add new iov_iter-based ceph_msg_data_type and > ceph_osd_data_type > ceph: use osd_req_op_extent_osd_iter for netfs reads > ceph: fscrypt_auth handling for ceph > ceph: ensure that we accept a new context from MDS for new inodes > ceph: add support for fscrypt_auth/fscrypt_file to cap messages > ceph: implement -o test_dummy_encryption mount option > ceph: decode alternate_name in lease info > ceph: add fscrypt ioctls > ceph: add encrypted fname handling to ceph_mdsc_build_path > ceph: send altname in MClientRequest > ceph: encode encrypted name in dentry release > ceph: properly set DCACHE_NOKEY_NAME flag in lookup > ceph: set DCACHE_NOKEY_NAME in atomic open > ceph: make d_revalidate call fscrypt revalidator for encrypted > dentries > ceph: add helpers for converting names for userland presentation > ceph: add fscrypt support to ceph_fill_trace > ceph: create symlinks with encrypted and base64-encoded targets > ceph: make ceph_get_name decrypt filenames > ceph: add a new ceph.fscrypt.auth vxattr > ceph: add some fscrypt guardrails > libceph: add CEPH_OSD_OP_ASSERT_VER support > ceph: size handling for encrypted inodes in cap updates > ceph: fscrypt_file field handling in MClientRequest messages > ceph: handle fscrypt fields in cap messages from MDS > ceph: update WARN_ON message to pr_warn > ceph: add infrastructure for file encryption and decryption > libceph: allow ceph_osdc_new_request to accept a multi-op read > ceph: disable fallocate for encrypted inodes > ceph: disable copy offload on encrypted inodes > ceph: don't use special DIO path for encrypted inodes > ceph: align data in pages in ceph_sync_write > ceph: add read/modify/write to ceph_sync_write > ceph: plumb in decryption during sync reads > ceph: add fscrypt decryption support to ceph_netfs_issue_op > ceph: set i_blkbits to crypto block size for encrypted inodes > ceph: add encryption support to writepage > ceph: fscrypt support for writepages > ceph: report STATX_ATTR_ENCRYPTED on encrypted inodes > > Luís Henriques (11): > ceph: add base64 endcoding routines for encrypted names > ceph: allow encrypting a directory while not having Ax caps > ceph: mark directory as non-complete after loading key > ceph: don't allow changing layout on encrypted files/directories > ceph: invalidate pages when doing direct/sync writes > ceph: add support for encrypted snapshot names > ceph: add support for handling encrypted snapshot names > ceph: update documentation regarding snapshot naming limitations > ceph: prevent snapshots to be created in encrypted locked directories > ceph: switch ceph_lookup() to use new fscrypt helper > ceph: switch ceph_open_atomic() to use the new fscrypt helper > > Xiubo Li (13): > ceph: make the ioctl cmd more readable in debug log > ceph: fix base64 encoded name's length check in ceph_fname_to_usr() > ceph: pass the request to parse_reply_info_readdir() > ceph: add ceph_encode_encrypted_dname() helper > ceph: add support to readdir for encrypted filenames > ceph: get file size from fscrypt_file when present in inode traces > ceph: add __ceph_get_caps helper support > ceph: add __ceph_sync_read helper support > ceph: add object version support for sync read > ceph: add truncate size handling support for fscrypt > ceph: drop the messages from MDS when unmounting > ceph: just wait the osd requests' callbacks to finish when unmounting > ceph: fix updating the i_truncate_pagecache_size for fscrypt > > Documentation/filesystems/ceph.rst | 10 + > fs/ceph/Makefile | 1 + > fs/ceph/acl.c | 4 +- > fs/ceph/addr.c | 192 +++++++-- > fs/ceph/caps.c | 227 ++++++++-- > fs/ceph/crypto.c | 659 +++++++++++++++++++++++++++++ > fs/ceph/crypto.h | 270 ++++++++++++ > fs/ceph/dir.c | 188 ++++++-- > fs/ceph/export.c | 44 +- > fs/ceph/file.c | 593 ++++++++++++++++++++++---- > fs/ceph/inode.c | 613 ++++++++++++++++++++++++--- > fs/ceph/ioctl.c | 126 +++++- > fs/ceph/mds_client.c | 479 +++++++++++++++++---- > fs/ceph/mds_client.h | 29 +- > fs/ceph/quota.c | 14 +- > fs/ceph/snap.c | 10 +- > fs/ceph/super.c | 191 ++++++++- > fs/ceph/super.h | 46 +- > fs/ceph/xattr.c | 29 ++ > include/linux/ceph/ceph_fs.h | 21 +- > include/linux/ceph/messenger.h | 40 ++ > include/linux/ceph/osd_client.h | 93 +++- > include/linux/ceph/rados.h | 4 + > net/ceph/messenger.c | 79 ++++ > net/ceph/messenger_v1.c | 98 ++++- > net/ceph/messenger_v2.c | 286 ++++++++++++- > net/ceph/osd_client.c | 332 ++++++++++++++- > 27 files changed, 4261 insertions(+), 417 deletions(-) > create mode 100644 fs/ceph/crypto.c > create mode 100644 fs/ceph/crypto.h > > -- > 2.40.1 >
On Tue, Jun 13, 2023 at 10:56 AM <xiubli@redhat.com> wrote: > > From: Xiubo Li <xiubli@redhat.com> > > > This patch series is based on Jeff Layton's previous great work and effort > on this and all the patches have been in the testing branch and well tested: > > The qa teuthology tests results: > https://pulpito.ceph.com/xiubli-2023-06-09_03:59:14-fs:fscrypt-wip-lxb-fscrypt-20230607-0901-distro-default-smithi/ > > > Since v15 we have added the ceph qa teuthology test cases for this [1][2], > which will test both the file name and contents encryption features and at > the same time they will also test the IO benchmarks. > > To support the fscrypt we also have some other work in ceph [3][4][5][6][7][8][9]: > > [1] https://github.com/ceph/ceph/pull/48628 > [2] https://github.com/ceph/ceph/pull/49934 > [3] https://github.com/ceph/ceph/pull/43588 > [4] https://github.com/ceph/ceph/pull/37297 > [5] https://github.com/ceph/ceph/pull/45192 > [6] https://github.com/ceph/ceph/pull/45312 > [7] https://github.com/ceph/ceph/pull/40828 > [8] https://github.com/ceph/ceph/pull/45224 > [9] https://github.com/ceph/ceph/pull/45073 > > The [9] is still undering testing and will soon be merged after that. All > the others had been merged. > > > The main changes since v19: > > - s/ceph_open/ceph_lookup/ for [PATCH v20 70/71]. > - Updated the patches to fix use-after-free bug for fscrypt_destroy_keyring, > the link https://patchwork.kernel.org/project/ceph-devel/list/?series=755131. > These new changes have been > > > > Jeff Layton (47): > libceph: add spinlock around osd->o_requests > libceph: define struct ceph_sparse_extent and add some helpers > libceph: add sparse read support to msgr2 crc state machine > libceph: add sparse read support to OSD client > libceph: support sparse reads on msgr2 secure codepath > libceph: add sparse read support to msgr1 > ceph: add new mount option to enable sparse reads > ceph: preallocate inode for ops that may create one > ceph: make ceph_msdc_build_path use ref-walk > libceph: add new iov_iter-based ceph_msg_data_type and > ceph_osd_data_type > ceph: use osd_req_op_extent_osd_iter for netfs reads > ceph: fscrypt_auth handling for ceph > ceph: ensure that we accept a new context from MDS for new inodes > ceph: add support for fscrypt_auth/fscrypt_file to cap messages > ceph: implement -o test_dummy_encryption mount option > ceph: decode alternate_name in lease info > ceph: add fscrypt ioctls > ceph: add encrypted fname handling to ceph_mdsc_build_path > ceph: send altname in MClientRequest > ceph: encode encrypted name in dentry release > ceph: properly set DCACHE_NOKEY_NAME flag in lookup > ceph: set DCACHE_NOKEY_NAME in atomic open > ceph: make d_revalidate call fscrypt revalidator for encrypted > dentries > ceph: add helpers for converting names for userland presentation > ceph: add fscrypt support to ceph_fill_trace > ceph: create symlinks with encrypted and base64-encoded targets > ceph: make ceph_get_name decrypt filenames > ceph: add a new ceph.fscrypt.auth vxattr > ceph: add some fscrypt guardrails > libceph: add CEPH_OSD_OP_ASSERT_VER support > ceph: size handling for encrypted inodes in cap updates > ceph: fscrypt_file field handling in MClientRequest messages > ceph: handle fscrypt fields in cap messages from MDS > ceph: update WARN_ON message to pr_warn > ceph: add infrastructure for file encryption and decryption > libceph: allow ceph_osdc_new_request to accept a multi-op read > ceph: disable fallocate for encrypted inodes > ceph: disable copy offload on encrypted inodes > ceph: don't use special DIO path for encrypted inodes > ceph: align data in pages in ceph_sync_write > ceph: add read/modify/write to ceph_sync_write > ceph: plumb in decryption during sync reads > ceph: add fscrypt decryption support to ceph_netfs_issue_op > ceph: set i_blkbits to crypto block size for encrypted inodes > ceph: add encryption support to writepage > ceph: fscrypt support for writepages > ceph: report STATX_ATTR_ENCRYPTED on encrypted inodes > > Luís Henriques (11): > ceph: add base64 endcoding routines for encrypted names > ceph: allow encrypting a directory while not having Ax caps > ceph: mark directory as non-complete after loading key > ceph: don't allow changing layout on encrypted files/directories > ceph: invalidate pages when doing direct/sync writes > ceph: add support for encrypted snapshot names > ceph: add support for handling encrypted snapshot names > ceph: update documentation regarding snapshot naming limitations > ceph: prevent snapshots to be created in encrypted locked directories > ceph: switch ceph_lookup() to use new fscrypt helper > ceph: switch ceph_open_atomic() to use the new fscrypt helper > > Xiubo Li (13): > ceph: make the ioctl cmd more readable in debug log > ceph: fix base64 encoded name's length check in ceph_fname_to_usr() > ceph: pass the request to parse_reply_info_readdir() > ceph: add ceph_encode_encrypted_dname() helper > ceph: add support to readdir for encrypted filenames > ceph: get file size from fscrypt_file when present in inode traces > ceph: add __ceph_get_caps helper support > ceph: add __ceph_sync_read helper support > ceph: add object version support for sync read > ceph: add truncate size handling support for fscrypt > ceph: drop the messages from MDS when unmounting > ceph: just wait the osd requests' callbacks to finish when unmounting > ceph: fix updating the i_truncate_pagecache_size for fscrypt > > Documentation/filesystems/ceph.rst | 10 + > fs/ceph/Makefile | 1 + > fs/ceph/acl.c | 4 +- > fs/ceph/addr.c | 192 +++++++-- > fs/ceph/caps.c | 227 ++++++++-- > fs/ceph/crypto.c | 659 +++++++++++++++++++++++++++++ > fs/ceph/crypto.h | 270 ++++++++++++ > fs/ceph/dir.c | 188 ++++++-- > fs/ceph/export.c | 44 +- > fs/ceph/file.c | 593 ++++++++++++++++++++++---- > fs/ceph/inode.c | 613 ++++++++++++++++++++++++--- > fs/ceph/ioctl.c | 126 +++++- > fs/ceph/mds_client.c | 479 +++++++++++++++++---- > fs/ceph/mds_client.h | 29 +- > fs/ceph/quota.c | 14 +- > fs/ceph/snap.c | 10 +- > fs/ceph/super.c | 191 ++++++++- > fs/ceph/super.h | 46 +- > fs/ceph/xattr.c | 29 ++ > include/linux/ceph/ceph_fs.h | 21 +- > include/linux/ceph/messenger.h | 40 ++ > include/linux/ceph/osd_client.h | 93 +++- > include/linux/ceph/rados.h | 4 + > net/ceph/messenger.c | 79 ++++ > net/ceph/messenger_v1.c | 98 ++++- > net/ceph/messenger_v2.c | 286 ++++++++++++- > net/ceph/osd_client.c | 332 ++++++++++++++- > 27 files changed, 4261 insertions(+), 417 deletions(-) > create mode 100644 fs/ceph/crypto.c > create mode 100644 fs/ceph/crypto.h > > -- > 2.40.1 > Tested-by: Venky Shankar <vshankar@redhat.com>
From: Xiubo Li <xiubli@redhat.com> This patch series is based on Jeff Layton's previous great work and effort on this and all the patches have been in the testing branch and well tested: The qa teuthology tests results: https://pulpito.ceph.com/xiubli-2023-06-09_03:59:14-fs:fscrypt-wip-lxb-fscrypt-20230607-0901-distro-default-smithi/ Since v15 we have added the ceph qa teuthology test cases for this [1][2], which will test both the file name and contents encryption features and at the same time they will also test the IO benchmarks. To support the fscrypt we also have some other work in ceph [3][4][5][6][7][8][9]: [1] https://github.com/ceph/ceph/pull/48628 [2] https://github.com/ceph/ceph/pull/49934 [3] https://github.com/ceph/ceph/pull/43588 [4] https://github.com/ceph/ceph/pull/37297 [5] https://github.com/ceph/ceph/pull/45192 [6] https://github.com/ceph/ceph/pull/45312 [7] https://github.com/ceph/ceph/pull/40828 [8] https://github.com/ceph/ceph/pull/45224 [9] https://github.com/ceph/ceph/pull/45073 The [9] is still undering testing and will soon be merged after that. All the others had been merged. The main changes since v19: - s/ceph_open/ceph_lookup/ for [PATCH v20 70/71]. - Updated the patches to fix use-after-free bug for fscrypt_destroy_keyring, the link https://patchwork.kernel.org/project/ceph-devel/list/?series=755131. These new changes have been Jeff Layton (47): libceph: add spinlock around osd->o_requests libceph: define struct ceph_sparse_extent and add some helpers libceph: add sparse read support to msgr2 crc state machine libceph: add sparse read support to OSD client libceph: support sparse reads on msgr2 secure codepath libceph: add sparse read support to msgr1 ceph: add new mount option to enable sparse reads ceph: preallocate inode for ops that may create one ceph: make ceph_msdc_build_path use ref-walk libceph: add new iov_iter-based ceph_msg_data_type and ceph_osd_data_type ceph: use osd_req_op_extent_osd_iter for netfs reads ceph: fscrypt_auth handling for ceph ceph: ensure that we accept a new context from MDS for new inodes ceph: add support for fscrypt_auth/fscrypt_file to cap messages ceph: implement -o test_dummy_encryption mount option ceph: decode alternate_name in lease info ceph: add fscrypt ioctls ceph: add encrypted fname handling to ceph_mdsc_build_path ceph: send altname in MClientRequest ceph: encode encrypted name in dentry release ceph: properly set DCACHE_NOKEY_NAME flag in lookup ceph: set DCACHE_NOKEY_NAME in atomic open ceph: make d_revalidate call fscrypt revalidator for encrypted dentries ceph: add helpers for converting names for userland presentation ceph: add fscrypt support to ceph_fill_trace ceph: create symlinks with encrypted and base64-encoded targets ceph: make ceph_get_name decrypt filenames ceph: add a new ceph.fscrypt.auth vxattr ceph: add some fscrypt guardrails libceph: add CEPH_OSD_OP_ASSERT_VER support ceph: size handling for encrypted inodes in cap updates ceph: fscrypt_file field handling in MClientRequest messages ceph: handle fscrypt fields in cap messages from MDS ceph: update WARN_ON message to pr_warn ceph: add infrastructure for file encryption and decryption libceph: allow ceph_osdc_new_request to accept a multi-op read ceph: disable fallocate for encrypted inodes ceph: disable copy offload on encrypted inodes ceph: don't use special DIO path for encrypted inodes ceph: align data in pages in ceph_sync_write ceph: add read/modify/write to ceph_sync_write ceph: plumb in decryption during sync reads ceph: add fscrypt decryption support to ceph_netfs_issue_op ceph: set i_blkbits to crypto block size for encrypted inodes ceph: add encryption support to writepage ceph: fscrypt support for writepages ceph: report STATX_ATTR_ENCRYPTED on encrypted inodes Luís Henriques (11): ceph: add base64 endcoding routines for encrypted names ceph: allow encrypting a directory while not having Ax caps ceph: mark directory as non-complete after loading key ceph: don't allow changing layout on encrypted files/directories ceph: invalidate pages when doing direct/sync writes ceph: add support for encrypted snapshot names ceph: add support for handling encrypted snapshot names ceph: update documentation regarding snapshot naming limitations ceph: prevent snapshots to be created in encrypted locked directories ceph: switch ceph_lookup() to use new fscrypt helper ceph: switch ceph_open_atomic() to use the new fscrypt helper Xiubo Li (13): ceph: make the ioctl cmd more readable in debug log ceph: fix base64 encoded name's length check in ceph_fname_to_usr() ceph: pass the request to parse_reply_info_readdir() ceph: add ceph_encode_encrypted_dname() helper ceph: add support to readdir for encrypted filenames ceph: get file size from fscrypt_file when present in inode traces ceph: add __ceph_get_caps helper support ceph: add __ceph_sync_read helper support ceph: add object version support for sync read ceph: add truncate size handling support for fscrypt ceph: drop the messages from MDS when unmounting ceph: just wait the osd requests' callbacks to finish when unmounting ceph: fix updating the i_truncate_pagecache_size for fscrypt Documentation/filesystems/ceph.rst | 10 + fs/ceph/Makefile | 1 + fs/ceph/acl.c | 4 +- fs/ceph/addr.c | 192 +++++++-- fs/ceph/caps.c | 227 ++++++++-- fs/ceph/crypto.c | 659 +++++++++++++++++++++++++++++ fs/ceph/crypto.h | 270 ++++++++++++ fs/ceph/dir.c | 188 ++++++-- fs/ceph/export.c | 44 +- fs/ceph/file.c | 593 ++++++++++++++++++++++---- fs/ceph/inode.c | 613 ++++++++++++++++++++++++--- fs/ceph/ioctl.c | 126 +++++- fs/ceph/mds_client.c | 479 +++++++++++++++++---- fs/ceph/mds_client.h | 29 +- fs/ceph/quota.c | 14 +- fs/ceph/snap.c | 10 +- fs/ceph/super.c | 191 ++++++++- fs/ceph/super.h | 46 +- fs/ceph/xattr.c | 29 ++ include/linux/ceph/ceph_fs.h | 21 +- include/linux/ceph/messenger.h | 40 ++ include/linux/ceph/osd_client.h | 93 +++- include/linux/ceph/rados.h | 4 + net/ceph/messenger.c | 79 ++++ net/ceph/messenger_v1.c | 98 ++++- net/ceph/messenger_v2.c | 286 ++++++++++++- net/ceph/osd_client.c | 332 ++++++++++++++- 27 files changed, 4261 insertions(+), 417 deletions(-) create mode 100644 fs/ceph/crypto.c create mode 100644 fs/ceph/crypto.h