mbox series

[v18,00/83] sg: add v4 interface, request sharing

Message ID 20210427215733.417746-1-dgilbert@interlog.com (mailing list archive)
Headers show
Series sg: add v4 interface, request sharing | expand

Message

Douglas Gilbert April 27, 2021, 9:56 p.m. UTC 
This is the combined patchset showing the additions in the
second half patchset, that will be presented after the first
half patchset is accepted.

Patches 1 to 45 (inclusive) are a new version (v18) of the
first half patchset and have their own cover letter whose
subject line is:
    [PATCH v18 00/45] sg: add v4 interface

The idea behind releasing the combined patchset is so they can
be run through the kernel's code sanity checking mechanisms.
So expect some noise.

The additions in the second half patchset are more fully described
in:    https://sg.danny.cz/sg/sg_v40.html  which is mirrored at:
       https://doug-gilbert.github.io/sg_v40.html
       

The following list is a summary of features in the second patchset:

    - add (sg) file descriptor sharing; this is used by:
      - request sharing
      - multiple requests (mrq) use of SGV4_FLAG_DO_ON_OTHER flag

    - add request sharing, mainly to expedite copying. READ bio
      handed off to paired WRITE with no data to user space unless
      requested. Also capable of using VERIFY(BytChk=1) instead of
      the WRITE (what NVMe does in its Compare NVM command).

    - extend the request sharing logic so SGV4_FLAG_KEEP_SHARE
      will keep bio after a WRITE. This allows for a single
      source, multiple destinations copy 

    - add an extensible SG_SET_GET_EXTENDED ioctl(2) that takes a
      fixed size structure (96 byte).

    - add multiple requests capability (mrq) in a single ioctl(SG_IO)
      or ioctl(SG_IOSUBMIT) invocation. Can be combined with request
      sharing.

    - add a SGV4_FLAG_IMMED flag for ioctl(SG_IORECEIVE) or
      ioctl(SG_IORECEIVE_V3) so they don't wait

    - add logic for (block layer generated) tag handling and keep
      existing pack_id (packet id) logic which plays a similar role

    - add ioctl(SG_IOABORT) to abort an inflight command/request
      using its pack-id or tag.

    - add shared variable blocking (svb) method to the mrq. Assumes
      it is doing copy-like request sharing. By default WRITEs are
      unordered (wrt to each other). With SGV4_FLAG_ORDERED_WR flag
      WRITEs are ordered as required for ZBC disks.

    - add support to pass a fd generated by eventfd(2) to the driver
      via an ioctl(2).

    - use iopoll/hipri/blk_poll with mrq, especially svb.

    - bump the driver version number to 4.0.47

Most of the above are _only_ implemented for the sg version 4
(i.e. based on struct sg_io_v4) interface.


Douglas Gilbert (83):
  sg: move functions around
  sg: remove typedefs, type+formatting cleanup
  sg: sg_log and is_enabled
  sg: rework sg_poll(), minor changes
  sg: bitops in sg_device
  sg: make open count an atomic
  sg: move header to uapi section
  sg: speed sg_poll and sg_get_num_waiting
  sg: sg_allow_if_err_recovery and renames
  sg: improve naming
  sg: change rwlock to spinlock
  sg: ioctl handling
  sg: split sg_read
  sg: sg_common_write add structure for arguments
  sg: rework sg_vma_fault
  sg: rework sg_mmap
  sg: replace sg_allow_access
  sg: rework scatter gather handling
  sg: introduce request state machine
  sg: sg_find_srp_by_id
  sg: sg_fill_request_element
  sg: printk change %p to %pK
  sg: xarray for fds in device
  sg: xarray for reqs in fd
  sg: replace rq array with xarray
  sg: sense buffer rework
  sg: add sg v4 interface support
  sg: rework debug info
  sg: add 8 byte SCSI LUN to sg_scsi_id
  sg: expand sg_comm_wr_t
  sg: add sg_iosubmit_v3 and sg_ioreceive_v3 ioctls
  sg: add some __must_hold macros
  sg: move procfs objects to avoid forward decls
  sg: protect multiple receivers
  sg: first debugfs support
  sg: rework mmap support
  sg: defang allow_dio
  sg: warn v3 write system call users
  sg: add mmap_sz tracking
  sg: remove rcv_done request state
  sg: track lowest inactive and await indexes
  sg: remove unit attention check for device changed
  sg: no_dxfer: move to/from kernel buffers
  sg: add blk_poll support
  sg: bump version to 4.0.12
  sg: add sg_ioabort ioctl
  sg: add sg_set_get_extended ioctl
  sg: sgat_elem_sz and sum_fd_dlens
  sg: tag and more_async
  sg: add fd sharing , change, unshare
  sg: add shared requests
  sg: add multiple request support
  sg: rename some mrq variables
  sg: unlikely likely
  sg: mrq abort
  sg: reduce atomic operations
  sg: add excl_wait flag
  sg: tweak sg_find_sfp_by_fd()
  sg: add snap_dev flag and snapped in debugfs
  sg: compress usercontext to uc
  sg: optionally output sg_request.frq_bm flags
  sg: work on sg_mrq_sanity()
  sg: shared variable blocking
  sg: device timestamp
  sg: condition met is not an error
  sg: split sg_setup_req
  sg: finish after read-side request
  sg: keep share and dout offset flags
  sg: add dlen to sg_comm_wr_t
  sg: make use of struct sg_mrq_hold
  sg: add mmap IO option for mrq metadata
  sg: add eventfd support
  sg: table of error number explanations
  sg: add ordered write flag
  sg: expand source line length to 100 characters
  sg: add no_attach_msg parameter
  sg: add SGV4_FLAG_REC_ORDER
  sg: max to read for mrq sg_ioreceive
  sg: mrq: if uniform svb then re-use bio_s
  sg: expand bvec usage; re-use bio_s
  sg: blk_poll/hipri work for mrq
  sg: pollable and non-pollable requests
  sg: bump version to 4.0.47

 drivers/scsi/sg.c      | 10003 +++++++++++++++++++++++++++++++--------
 include/scsi/sg.h      |   273 +-
 include/uapi/scsi/sg.h |   491 ++
 3 files changed, 8440 insertions(+), 2327 deletions(-)
 create mode 100644 include/uapi/scsi/sg.h