| Message ID | 20241111234006.5942-1-ouster@cs.stanford.edu (mailing list archive) |
|---|---|
| Headers | show |
| Series | Begin upstreaming Homa transport protocol | expand |
On Mon, Nov 11, 2024 at 03:39:53PM -0800, John Ousterhout wrote: > This patch series begins the process of upstreaming the Homa transport > protocol. Homa is an alternative to TCP for use in datacenter > environments. It provides 10-100x reductions in tail latency for short > messages relative to TCP. Its benefits are greatest for mixed workloads > containing both short and long messages running under high network loads. > Homa is not API-compatible with TCP: it is connectionless and message- > oriented (but still reliable and flow-controlled). Homa's new API not > only contributes to its performance gains, but it also eliminates the > massive amount of connection state required by TCP for highly connected > datacenter workloads. Quick note for future versions: Consider adding a changelog to the cover letter and to the individual patches which change from revision to revision to make it easier for folks to follow along with what changed. You can find more information about this here: https://www.kernel.org/doc/html/v6.11/process/submitting-patches.html#respond-to-review-comments And it might be helpful to take a look at other series on the list which have multiple revisions [1] to get a sense of how others typically do this. [1]: https://lore.kernel.org/netdev/20241030-feature_ptp_netnext-v19-0-94f8aadc9d5c@bootlin.com/
On Mon, Nov 11, 2024 at 4:15 PM Joe Damato <jdamato@fastly.com> wrote: > > Quick note for future versions: > > Consider adding a changelog to the cover letter and to the > individual patches which change from revision to revision to make > it easier for folks to follow along with what changed. Oops, sorry for the omission. I'll get this right in v3 and beyond. In the meantime, here is the changelog for v2: - Remove sockaddr_in_union declaration from public API in homa.h - Remove kernel wrapper functions (homa_send, etc.) from homa.h - Fix many sparse warnings (still more work to do here) and other issues uncovered by test robot - Fix checkpatch.pl issues - Remove residual code related to unit tests - Remove references to tt_record from comments - Make it safe to delete sockets during homa_socktab scans - Use uintptr_t for portability fo 32-bit platforms - Use do_div instead of "/" for portability - Remove homa->busy_usecs and homa->gro_busy_usecs (not needed in this stripped down version of Homa) - Eliminate usage of cpu_khz, use sched_clock instead of get_cycles - Add missing checks of kmalloc return values - Remove "inline" qualifier from functions in .c files - Document that pad fields must be zero - Use more precise type "uint32_t" rather than "int" - Remove unneeded #include of linux/version.h > You can find more information about this here: > https://www.kernel.org/doc/html/v6.11/process/submitting-patches.html#respond-to-review-comments > > And it might be helpful to take a look at other series on the list > which have multiple revisions [1] to get a sense of how others > typically do this. > > [1]: https://lore.kernel.org/netdev/20241030-feature_ptp_netnext-v19-0-94f8aadc9d5c@bootlin.com/
On Mon, 11 Nov 2024 15:39:53 -0800 John Ousterhout wrote: > This patch series begins the process of upstreaming the Homa transport > protocol. Homa is an alternative to TCP for use in datacenter > environments. It provides 10-100x reductions in tail latency for short > messages relative to TCP. Its benefits are greatest for mixed workloads > containing both short and long messages running under high network loads. > Homa is not API-compatible with TCP: it is connectionless and message- > oriented (but still reliable and flow-controlled). Homa's new API not > only contributes to its performance gains, but it also eliminates the > massive amount of connection state required by TCP for highly connected > datacenter workloads. Hi John! Thanks for pushing forward! I wanted to give you a heads up that we're operating at 50% maintainer capacity for the next 2 weeks so the reviews may be more muted than usual. Don't hesitate to post new versions (typically ~once a week) if you want to address any incoming feedback and/or the build issues. HTH
On Mon, Nov 11, 2024 at 03:39:53PM -0800, John Ousterhout wrote: > MAINTAINERS | 7 + > include/uapi/linux/homa.h | 165 ++++++ > net/Kconfig | 1 + > net/Makefile | 1 + > net/homa/Kconfig | 19 + > net/homa/Makefile | 14 + > net/homa/homa_impl.h | 767 ++++++++++++++++++++++++++ > net/homa/homa_incoming.c | 1076 +++++++++++++++++++++++++++++++++++++ > net/homa/homa_outgoing.c | 854 +++++++++++++++++++++++++++++ > net/homa/homa_peer.c | 319 +++++++++++ > net/homa/homa_peer.h | 234 ++++++++ > net/homa/homa_plumbing.c | 965 +++++++++++++++++++++++++++++++++ > net/homa/homa_pool.c | 420 +++++++++++++++ > net/homa/homa_pool.h | 152 ++++++ > net/homa/homa_rpc.c | 488 +++++++++++++++++ > net/homa/homa_rpc.h | 446 +++++++++++++++ > net/homa/homa_sock.c | 380 +++++++++++++ > net/homa/homa_sock.h | 426 +++++++++++++++ > net/homa/homa_stub.h | 80 +++ > net/homa/homa_timer.c | 156 ++++++ > net/homa/homa_utils.c | 150 ++++++ > net/homa/homa_wire.h | 378 +++++++++++++ Hi John, Thanks for your efforts to push them upstream! Just some very high-level comments: 1. Please run scripts/checkpatch.pl to make sure the coding style is aligned with upstream, since I noticed there are still some C++ style comments in your patchset. 2. Please consider adding socket diagnostics, see net/ipv4/inet_diag.c. 3. Please consider adding test cases, you can pretty much follow tools/testing/vsock/vsock_test.c. Regards, Cong
On Tue, Nov 12, 2024 at 5:48 PM Jakub Kicinski <kuba@kernel.org> wrote: > > On Mon, 11 Nov 2024 15:39:53 -0800 John Ousterhout wrote: > > This patch series begins the process of upstreaming the Homa transport > > protocol. Homa is an alternative to TCP for use in datacenter > > environments. It provides 10-100x reductions in tail latency for short > > messages relative to TCP. Its benefits are greatest for mixed workloads > > containing both short and long messages running under high network loads. > > Homa is not API-compatible with TCP: it is connectionless and message- > > oriented (but still reliable and flow-controlled). Homa's new API not > > only contributes to its performance gains, but it also eliminates the > > massive amount of connection state required by TCP for highly connected > > datacenter workloads. > > Hi John! > > Thanks for pushing forward! I wanted to give you a heads up that we're > operating at 50% maintainer capacity for the next 2 weeks so the > reviews may be more muted than usual. Don't hesitate to post new > versions (typically ~once a week) if you want to address any incoming > feedback and/or the build issues. Thanks for letting me know. -John-
On Wed, Nov 13, 2024 at 9:08 AM Cong Wang <xiyou.wangcong@gmail.com> wrote: > > On Mon, Nov 11, 2024 at 03:39:53PM -0800, John Ousterhout wrote: > > MAINTAINERS | 7 + > > include/uapi/linux/homa.h | 165 ++++++ > > net/Kconfig | 1 + > > net/Makefile | 1 + > > net/homa/Kconfig | 19 + > > net/homa/Makefile | 14 + > > net/homa/homa_impl.h | 767 ++++++++++++++++++++++++++ > > net/homa/homa_incoming.c | 1076 +++++++++++++++++++++++++++++++++++++ > > net/homa/homa_outgoing.c | 854 +++++++++++++++++++++++++++++ > > net/homa/homa_peer.c | 319 +++++++++++ > > net/homa/homa_peer.h | 234 ++++++++ > > net/homa/homa_plumbing.c | 965 +++++++++++++++++++++++++++++++++ > > net/homa/homa_pool.c | 420 +++++++++++++++ > > net/homa/homa_pool.h | 152 ++++++ > > net/homa/homa_rpc.c | 488 +++++++++++++++++ > > net/homa/homa_rpc.h | 446 +++++++++++++++ > > net/homa/homa_sock.c | 380 +++++++++++++ > > net/homa/homa_sock.h | 426 +++++++++++++++ > > net/homa/homa_stub.h | 80 +++ > > net/homa/homa_timer.c | 156 ++++++ > > net/homa/homa_utils.c | 150 ++++++ > > net/homa/homa_wire.h | 378 +++++++++++++ > > Hi John, > > Thanks for your efforts to push them upstream! > > Just some very high-level comments: > > 1. Please run scripts/checkpatch.pl to make sure the coding style is > aligned with upstream, since I noticed there are still some C++ style > comments in your patchset. I have been running checkpatch.pl, but it didn't complain about C++ style comments. Those comments really shouldn't be there, though: they are for pieces of code I've temporarily commented out, and those chunks shouldn't be in the upstream version of Homa. I'll fix things so they don't appear in the future. > 2. Please consider adding socket diagnostics, see net/ipv4/inet_diag.c. I wasn't familiar with them before your email; I'll take a look. > 3. Please consider adding test cases, you can pretty much follow > tools/testing/vsock/vsock_test.c. Homa has extensive unit tests in the GitHub repo, but I thought I'd wait to try upstreaming them until all of Homa has been upstreamed (they are based on a modified version of kselftest.h, so they may be a bit controversial). One way or another, though, I'm committed to having good unit tests for Homa. Thank you for your feedback. -John-
On 11/14/24 8:59 AM, John Ousterhout wrote: > On Wed, Nov 13, 2024 at 9:08 AM Cong Wang <xiyou.wangcong@gmail.com> wrote: >> >> On Mon, Nov 11, 2024 at 03:39:53PM -0800, John Ousterhout wrote: >>> MAINTAINERS | 7 + >>> include/uapi/linux/homa.h | 165 ++++++ >>> net/Kconfig | 1 + >>> net/Makefile | 1 + >>> net/homa/Kconfig | 19 + >>> net/homa/Makefile | 14 + >>> net/homa/homa_impl.h | 767 ++++++++++++++++++++++++++ >>> net/homa/homa_incoming.c | 1076 +++++++++++++++++++++++++++++++++++++ >>> net/homa/homa_outgoing.c | 854 +++++++++++++++++++++++++++++ >>> net/homa/homa_peer.c | 319 +++++++++++ >>> net/homa/homa_peer.h | 234 ++++++++ >>> net/homa/homa_plumbing.c | 965 +++++++++++++++++++++++++++++++++ >>> net/homa/homa_pool.c | 420 +++++++++++++++ >>> net/homa/homa_pool.h | 152 ++++++ >>> net/homa/homa_rpc.c | 488 +++++++++++++++++ >>> net/homa/homa_rpc.h | 446 +++++++++++++++ >>> net/homa/homa_sock.c | 380 +++++++++++++ >>> net/homa/homa_sock.h | 426 +++++++++++++++ >>> net/homa/homa_stub.h | 80 +++ >>> net/homa/homa_timer.c | 156 ++++++ >>> net/homa/homa_utils.c | 150 ++++++ >>> net/homa/homa_wire.h | 378 +++++++++++++ >> >> Hi John, >> >> Thanks for your efforts to push them upstream! >> >> Just some very high-level comments: >> >> 1. Please run scripts/checkpatch.pl to make sure the coding style is >> aligned with upstream, since I noticed there are still some C++ style >> comments in your patchset. > > I have been running checkpatch.pl, but it didn't complain about C++ > style comments. Those comments really shouldn't be there, though: they > are for pieces of code I've temporarily commented out, and those > chunks shouldn't be in the upstream version of Homa. I'll fix things > so they don't appear in the future. > John, you should be OK using // comments if that's what you want to use. https://lkml.iu.edu/hypermail/linux/kernel/1607.1/00627.html or https://lore.kernel.org/lkml/CA+55aFyQYJerovMsSoSKS7PessZBr4vNp-3QUUwhqk4A4_jcbg@mail.gmail.com/ are still current AFAIK.
On Thu, Nov 14, 2024 at 8:59 AM John Ousterhout <ouster@cs.stanford.edu> wrote: > > On Wed, Nov 13, 2024 at 9:08 AM Cong Wang <xiyou.wangcong@gmail.com> wrote: > > > 2. Please consider adding socket diagnostics, see net/ipv4/inet_diag.c. > > I wasn't familiar with them before your email; I'll take a look. I have taken a look at socket diagnostics, and I agree that Homa should support them. However, this looks like a nontrivial task: there appears to be a fair amount of complexity there, not much documentation, and the current diagnostics appear pretty TCP-specific so it may take some work (and advice) to figure out how to map them onto Homa. Thus it feels like it will take a fair amount of back-and-forth to get this right. I would prefer to defer this until after the main body of Homa has been upstreamed, then have a patch series that is dedicated to socket diagnostics support, rather than lumping socket diagnostics into the current patch series. Is that OK with you? -John-
On Tue, Nov 19, 2024 at 01:13:36PM -0800, John Ousterhout wrote: > On Thu, Nov 14, 2024 at 8:59 AM John Ousterhout <ouster@cs.stanford.edu> wrote: > > > > On Wed, Nov 13, 2024 at 9:08 AM Cong Wang <xiyou.wangcong@gmail.com> wrote: > > > > > 2. Please consider adding socket diagnostics, see net/ipv4/inet_diag.c. > > > > I wasn't familiar with them before your email; I'll take a look. > > I have taken a look at socket diagnostics, and I agree that Homa > should support them. However, this looks like a nontrivial task: there > appears to be a fair amount of complexity there, not much > documentation, and the current diagnostics appear pretty TCP-specific > so it may take some work (and advice) to figure out how to map them > onto Homa. Thus it feels like it will take a fair amount of > back-and-forth to get this right. I would prefer to defer this until > after the main body of Homa has been upstreamed, then have a patch > series that is dedicated to socket diagnostics support, rather than > lumping socket diagnostics into the current patch series. Is that OK > with you? > Sure, I don't think it is a blocker here, it is just a nice feature to promote Homa. Thanks.