mbox series

[v4,0/6] add multichannel support

Message ID 20191103012112.12212-1-aaptel@suse.com (mailing list archive)
Headers show
Series add multichannel support | expand

Message

Aurélien Aptel Nov. 3, 2019, 1:21 a.m. UTC 
Hello,

I have very little time to work on this during the work week due to
annoying^W lovely customers, company stuff and backporting work but
during this long weekend of festivities I somehow mustered the courage
to dig in again...

So without further ado, here is another iteration of the multichannel
patchset.

Couple of changes since last time:
* do round-robin on the channels (cycle over them on requests)
* reuse master client guid for new channel (fixes binding against samba)
* simplification of crypto key lookup and potential race condition fix
* properly discard channels on session closing
* better display of channels info in DebugData

Usage
=====

Make sure your server exposes multiple interfaces and mount with:

    -o multichannel,max_channels=2

To have a total of 2 connections. One master and one extra channel.
* Each call to ...send_recv() will alternate between the 2 channels.
* You can check /proc/fs/cifs/DebugData for channel infos

Overview
========

Multichannel is part of SMB3 and above. It lets the client "share" the
same SMB session over multiple TCP (or RDMA) connections.

* First, a regular connection is made.

client                                                       server
  |							       |
  | ----------- negprot req (unsigned, guid=0xABC)--------->   |
  |							       |
  | <---------- negprot res (unsigned)	-------------------    |
  |							       |
  | ----------  sesssetup req (sesid=0x000, unsigned) ------>  |
  |							       |
  | <---------  sesssetup res (sesid=0x123, signed) ---------  |
  |                                                            |
  |  (client now has signing/encryption/decryption keys)       |
  |                                                            |
  |							       |
  | ----------- ... (signed) ------------------------------->  |
  | <-------------------------------------------------------   |
  |							       |

* The client then queries the network interface available on the
  server via an FSCTL (this already implemented in cifs.ko)

* For each new channel, the client opens another connection, but
  reuses the client guid, and at session setup it sets a binding flag
  to say "this is a new channel, please make this session an alias to
  sess id 0x123"

* Since the session is an alias, you can use TreeCons, files, etc
  interchangeably between channels as if it was the same transport.

* Channels use the same encryption/decryption keys from the session
  BUT each channel has its own signing key.

* When you see "signed" in lowercase, it's signed using the session/master key.
  When you see "SIGNED" in uppercase, it's signed with the CHANNEL key.

Here is the traffic for opening and using new channels:
                                                              |
  | ---------  negprot req (unsigned, guid=0xABC) ----------> |  o
  |                                                           |  o
  | <--------  negprog res (unsigned) ----------------------  |  o BINDING
  |                                                           |  o  PART
  | ---------  sesssetup req (sesid=0x123, binding, signed)-> |  o
  |                                                           |  o
  | <--------  sesssetup res (sesid=0x123, SIGNED) ---------  |  o
  |                                                           |
  | (from then on, client needs to use CHANNEL key            |
  |  for signing)                                             |
  |                                                           |
  | ---------  ... (SIGNED) --------------------------------> |
  | <-------------------------------------------------------- |
  |                                                           |

This signing key business is tricky to get right. As usual refer to
MS-SMB2 for more boring details.

cifs.ko implementation
======================

Data structure
--------------

We add an array of struct cifs_chan to cifs_ses. A channel has a
server pointer and a signing key.

struct cifs_ses {
    struct TCP_Server_Info *server; <-- master con
    u8 smb3signingkey[...];         <-- master sign key
    u8 smb3encryptionkey[...];      <-- extra channels & master
    u8 smb3decryptionkey[...];      <-- extra channels & master
    bool binding;                   <--- true during BINDING PART
    ...
    struct cifs_chan {
           struct TCP_Server_Info *server; <-- channel con
           u8 signkey[SMB3_SIGN_KEY_SIZE]; <-- channel sign key
    } chans[16];
};

New channel connections (TCP_Server_Info) end up in the usual global
cifs_tcp_ses_list linked list but also in the ses channel array.

 ,-------------> global cifs_tcp_ses_list <-------------------------.
 |                                                                  |
 '- TCP_Server_Info  <-->  TCP_Server_Info  <-->  TCP_Server_Info <-'
       (master con)           (chan#1 con)         (chan#2 con)
       |      ^                    ^                    ^
       v      '--------------------|--------------------'
    cifs_ses                       |
    - chan_count = 3               |
    - chans[].server --------------'

Some important details:

* ses->server still points to the master con.
* ses->server == chans[0].server
* chan#1 and chan#2 have an empty server->smb_ses_list

Channel/session establishment
-----------------------------

* When opening channels we check ses->binding to know that we are in the
  BINDING PART. In which case we don't create a new session object but
  merely reuse the existing one.

* chan_count only keeps track of fully established channels. During
  the BINDING PART there is actually chan_count+1 channels, the last
  one being the channel in the process of being created.

NOTE: we could add a state field to cifs_chan and make cifs_count
      always accurate. Not sure if it's better.

Channel dispatch
----------------

On syscalls, as we get lower and lower in the bowels of cifs.ko we
eventually reach the transport layer. There we often pass a ses
pointer to code that assumes the transport can be reached via
ses->server.

* A big change was to decouple ses from server in those codepaths. So
  that we can select a channel and make the code use that one instead
  of the master.
* That often means passing ses AND server and replacing ses->server by
  server.

Signing and encryption
----------------------

* When generating the keys we make sure to not overwrite enc/dec keys
  for new channels and store the sign key in cifs_chan.

* When sending and receiving a message we have to search for the right
  keys by searching the whole cifs_tcp_ses_list instead of just
  looking at the master con sessions.


Aurelien Aptel (6):
  cifs: sort interface list by speed
  cifs: add multichannel mount options and data structs
  cifs: add server param
  cifs: switch servers depending on binding state
  cifs: try opening channels after mounting
  cifs: dump channel info in DebugData

 fs/cifs/cifs_debug.c    |  35 +++++++-
 fs/cifs/cifs_spnego.c   |   2 +-
 fs/cifs/cifsfs.c        |   4 +
 fs/cifs/cifsglob.h      |  45 +++++++++-
 fs/cifs/cifsproto.h     |   8 ++
 fs/cifs/connect.c       |  92 ++++++++++++++++----
 fs/cifs/sess.c          | 218 +++++++++++++++++++++++++++++++++++++++++++++++-
 fs/cifs/smb2misc.c      |  37 +++++---
 fs/cifs/smb2ops.c       |  31 +++++--
 fs/cifs/smb2pdu.c       | 109 ++++++++++++++----------
 fs/cifs/smb2proto.h     |   3 +-
 fs/cifs/smb2transport.c | 165 +++++++++++++++++++++++++++---------
 fs/cifs/transport.c     |  18 +++-
 13 files changed, 636 insertions(+), 131 deletions(-)

--
2.16.4