| Message ID | 20210215174002.2376333-1-dan@kernelim.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | sysfs files for multipath transport control | expand |
Hi Dan, On Mon, Feb 15, 2021 at 12:43 PM Dan Aloni <dan@kernelim.com> wrote: > > Hi Anna, > > This patchset builds ontop v2 of your 'sysfs files for changing IP' changeset. > > - The patchset adds two more sysfs objects, for one for transport and another > for multipath. > - Also, `net` renamed to `client`, and `client` now has symlink to its principal > transport. A client also has a symlink to its `multipath` object. > - The transport interface lets you change `dstaddr` of individual transports, > when `nconnect` is used (or if it wasn't used and these were added with the > new interface). > - The interface to add a transport is using a single string written to 'add', > for example: > > echo 'dstaddr 192.168.40.8 kind rdma' \ > > /sys/kernel/sunrpc/client/0/multipath/add > > These changes are independent of the method used to obtain a sunrpc ID for a > mountpoint. For that I've sent a concept patch showing an fspick-based > implementation: https://marc.info/?l=linux-nfs&m=161332454821849&w=4 I'm confused: does this allow adding arbitrary connections between a client and some server IP to an existing RPC client? Given the above description, that's how it reads to me, can you clarify please. I thought it was something specifically for v3 (because it has no concept of trunking). As for NFSv4 there is a notion of getting server locations via FS_LOCATION and doing trunking (ie multipathing)? I don't see how this code restricts the addition of transports to v3. > > Thanks > > Dan Aloni (8): > sunrpc: rename 'net' to 'client' > sunrpc: add xprt id > sunrpc: add a directory per sunrpc xprt > sunrpc: have client directory a symlink to the root transport > sunrpc: add IDs to multipath > sunrpc: add multipath directory and symlink from client > sunrpc: change rpc_clnt_add_xprt() to rpc_add_xprt() > sunrpc: introduce an 'add' node to 'multipath' sysfs directory > > fs/nfs/pnfs_nfs.c | 12 +- > include/linux/sunrpc/clnt.h | 12 +- > include/linux/sunrpc/xprt.h | 3 + > include/linux/sunrpc/xprtmultipath.h | 6 + > net/sunrpc/clnt.c | 39 +-- > net/sunrpc/sunrpc_syms.c | 2 + > net/sunrpc/sysfs.c | 403 +++++++++++++++++++++++---- > net/sunrpc/sysfs.h | 21 +- > net/sunrpc/xprt.c | 29 ++ > net/sunrpc/xprtmultipath.c | 37 +++ > 10 files changed, 487 insertions(+), 77 deletions(-) > > -- > 2.26.2 >
On Mon, Mar 01, 2021 at 10:56:22PM -0500, Olga Kornievskaia wrote: > Hi Dan, > > On Mon, Feb 15, 2021 at 12:43 PM Dan Aloni <dan@kernelim.com> wrote: > > > > Hi Anna, > > > > This patchset builds ontop v2 of your 'sysfs files for changing IP' changeset. > > > > - The patchset adds two more sysfs objects, for one for transport and another > > for multipath. > > - Also, `net` renamed to `client`, and `client` now has symlink to its principal > > transport. A client also has a symlink to its `multipath` object. > > - The transport interface lets you change `dstaddr` of individual transports, > > when `nconnect` is used (or if it wasn't used and these were added with the > > new interface). > > - The interface to add a transport is using a single string written to 'add', > > for example: > > > > echo 'dstaddr 192.168.40.8 kind rdma' \ > > > /sys/kernel/sunrpc/client/0/multipath/add > > > > These changes are independent of the method used to obtain a sunrpc ID for a > > mountpoint. For that I've sent a concept patch showing an fspick-based > > implementation: https://marc.info/?l=linux-nfs&m=161332454821849&w=4 > > I'm confused: does this allow adding arbitrary connections between a > client and some server IP to an existing RPC client? Given the above > description, that's how it reads to me, can you clarify please. I > thought it was something specifically for v3 (because it has no > concept of trunking). As for NFSv4 there is a notion of getting server > locations via FS_LOCATION and doing trunking (ie multipathing)? I > don't see how this code restricts the addition of transports to v3. Indeed, there's no restriction to NFSv3. There can be potential uses for this for NFSv4 too. FS_LOCATIONS serving as recommendation to which hosts the client can connect, while smart load-balancing logic in userspace can determine to which subset of these servers each client in a cluster should actually connect (a full mesh is not always desired). At any case, if this restriction is desired, we can add a new sunrpc client flag for that and pass it only in NFSv3 client init.
> On Mar 4, 2021, at 6:58 AM, Dan Aloni <dan@kernelim.com> wrote: > > On Mon, Mar 01, 2021 at 10:56:22PM -0500, Olga Kornievskaia wrote: >> Hi Dan, >> >> On Mon, Feb 15, 2021 at 12:43 PM Dan Aloni <dan@kernelim.com> wrote: >>> >>> Hi Anna, >>> >>> This patchset builds ontop v2 of your 'sysfs files for changing IP' changeset. >>> >>> - The patchset adds two more sysfs objects, for one for transport and another >>> for multipath. >>> - Also, `net` renamed to `client`, and `client` now has symlink to its principal >>> transport. A client also has a symlink to its `multipath` object. >>> - The transport interface lets you change `dstaddr` of individual transports, >>> when `nconnect` is used (or if it wasn't used and these were added with the >>> new interface). >>> - The interface to add a transport is using a single string written to 'add', >>> for example: >>> >>> echo 'dstaddr 192.168.40.8 kind rdma' \ >>>> /sys/kernel/sunrpc/client/0/multipath/add >>> >>> These changes are independent of the method used to obtain a sunrpc ID for a >>> mountpoint. For that I've sent a concept patch showing an fspick-based >>> implementation: https://marc.info/?l=linux-nfs&m=161332454821849&w=4 >> >> I'm confused: does this allow adding arbitrary connections between a >> client and some server IP to an existing RPC client? Given the above >> description, that's how it reads to me, can you clarify please. I >> thought it was something specifically for v3 (because it has no >> concept of trunking). As for NFSv4 there is a notion of getting server >> locations via FS_LOCATION and doing trunking (ie multipathing)? I >> don't see how this code restricts the addition of transports to v3. > > Indeed, there's no restriction to NFSv3. > > There can be potential uses for this for NFSv4 too. FS_LOCATIONS serving > as recommendation to which hosts the client can connect, while smart > load-balancing logic in userspace can determine to which subset of these > servers each client in a cluster should actually connect (a full mesh > is not always desired). > > At any case, if this restriction is desired, we can add a new sunrpc > client flag for that and pass it only in NFSv3 client init. IMO an NFSv3-only policy should not be built into this API. This is a user-space / kernel API, not something that is an administrative interface. The administrative interface, which is the place to apply an NFSv3-only policy, would use this sysfs API. So would smart load- balancing logic based on fs_locations. If the API is under /sys/kernel/sunrpc/client, then including NFS-specific controls is a layering violation. Consider that the kernel can send multiple protocols over the same connection (NFSv3 and NFSACL, eg). -- Chuck Lever
Hi Anna, This patchset builds ontop v2 of your 'sysfs files for changing IP' changeset. - The patchset adds two more sysfs objects, for one for transport and another for multipath. - Also, `net` renamed to `client`, and `client` now has symlink to its principal transport. A client also has a symlink to its `multipath` object. - The transport interface lets you change `dstaddr` of individual transports, when `nconnect` is used (or if it wasn't used and these were added with the new interface). - The interface to add a transport is using a single string written to 'add', for example: echo 'dstaddr 192.168.40.8 kind rdma' \ > /sys/kernel/sunrpc/client/0/multipath/add These changes are independent of the method used to obtain a sunrpc ID for a mountpoint. For that I've sent a concept patch showing an fspick-based implementation: https://marc.info/?l=linux-nfs&m=161332454821849&w=4 Thanks Dan Aloni (8): sunrpc: rename 'net' to 'client' sunrpc: add xprt id sunrpc: add a directory per sunrpc xprt sunrpc: have client directory a symlink to the root transport sunrpc: add IDs to multipath sunrpc: add multipath directory and symlink from client sunrpc: change rpc_clnt_add_xprt() to rpc_add_xprt() sunrpc: introduce an 'add' node to 'multipath' sysfs directory fs/nfs/pnfs_nfs.c | 12 +- include/linux/sunrpc/clnt.h | 12 +- include/linux/sunrpc/xprt.h | 3 + include/linux/sunrpc/xprtmultipath.h | 6 + net/sunrpc/clnt.c | 39 +-- net/sunrpc/sunrpc_syms.c | 2 + net/sunrpc/sysfs.c | 403 +++++++++++++++++++++++---- net/sunrpc/sysfs.h | 21 +- net/sunrpc/xprt.c | 29 ++ net/sunrpc/xprtmultipath.c | 37 +++ 10 files changed, 487 insertions(+), 77 deletions(-)