| Message ID | 20240513125346.764076-3-haakon.bugge@oracle.com (mailing list archive) |
|---|---|
| State | Superseded |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | rds: rdma: Add ability to force GFP_NOIO | expand |
Hi Håkon,
kernel test robot noticed the following build warnings:
[auto build test WARNING on tj-wq/for-next]
[also build test WARNING on rdma/for-next net/main net-next/main linus/master v6.9 next-20240513]
[cannot apply to horms-ipvs/master]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/H-kon-Bugge/workqueue-Inherit-NOIO-and-NOFS-alloc-flags/20240513-205927
base: https://git.kernel.org/pub/scm/linux/kernel/git/tj/wq.git for-next
patch link: https://lore.kernel.org/r/20240513125346.764076-3-haakon.bugge%40oracle.com
patch subject: [PATCH 2/6] rds: Brute force GFP_NOIO
config: s390-defconfig (https://download.01.org/0day-ci/archive/20240514/202405140136.LE0Nk9by-lkp@intel.com/config)
compiler: clang version 19.0.0git (https://github.com/llvm/llvm-project b910bebc300dafb30569cecc3017b446ea8eafa0)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240514/202405140136.LE0Nk9by-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202405140136.LE0Nk9by-lkp@intel.com/
All warnings (new ones prefixed by >>):
In file included from net/rds/af_rds.c:33:
In file included from include/linux/module.h:19:
In file included from include/linux/elf.h:6:
In file included from arch/s390/include/asm/elf.h:173:
In file included from arch/s390/include/asm/mmu_context.h:11:
In file included from arch/s390/include/asm/pgalloc.h:18:
In file included from include/linux/mm.h:2208:
include/linux/vmstat.h:508:43: warning: arithmetic between different enumeration types ('enum zone_stat_item' and 'enum numa_stat_item') [-Wenum-enum-conversion]
508 | return vmstat_text[NR_VM_ZONE_STAT_ITEMS +
| ~~~~~~~~~~~~~~~~~~~~~ ^
509 | item];
| ~~~~
include/linux/vmstat.h:515:43: warning: arithmetic between different enumeration types ('enum zone_stat_item' and 'enum numa_stat_item') [-Wenum-enum-conversion]
515 | return vmstat_text[NR_VM_ZONE_STAT_ITEMS +
| ~~~~~~~~~~~~~~~~~~~~~ ^
516 | NR_VM_NUMA_EVENT_ITEMS +
| ~~~~~~~~~~~~~~~~~~~~~~
include/linux/vmstat.h:522:36: warning: arithmetic between different enumeration types ('enum node_stat_item' and 'enum lru_list') [-Wenum-enum-conversion]
522 | return node_stat_name(NR_LRU_BASE + lru) + 3; // skip "nr_"
| ~~~~~~~~~~~ ^ ~~~
include/linux/vmstat.h:527:43: warning: arithmetic between different enumeration types ('enum zone_stat_item' and 'enum numa_stat_item') [-Wenum-enum-conversion]
527 | return vmstat_text[NR_VM_ZONE_STAT_ITEMS +
| ~~~~~~~~~~~~~~~~~~~~~ ^
528 | NR_VM_NUMA_EVENT_ITEMS +
| ~~~~~~~~~~~~~~~~~~~~~~
include/linux/vmstat.h:536:43: warning: arithmetic between different enumeration types ('enum zone_stat_item' and 'enum numa_stat_item') [-Wenum-enum-conversion]
536 | return vmstat_text[NR_VM_ZONE_STAT_ITEMS +
| ~~~~~~~~~~~~~~~~~~~~~ ^
537 | NR_VM_NUMA_EVENT_ITEMS +
| ~~~~~~~~~~~~~~~~~~~~~~
In file included from net/rds/af_rds.c:38:
In file included from include/linux/ipv6.h:101:
In file included from include/linux/tcp.h:17:
In file included from include/linux/skbuff.h:28:
In file included from include/linux/dma-mapping.h:11:
In file included from include/linux/scatterlist.h:9:
In file included from arch/s390/include/asm/io.h:78:
include/asm-generic/io.h:547:31: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
547 | val = __raw_readb(PCI_IOBASE + addr);
| ~~~~~~~~~~ ^
include/asm-generic/io.h:560:61: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
560 | val = __le16_to_cpu((__le16 __force)__raw_readw(PCI_IOBASE + addr));
| ~~~~~~~~~~ ^
include/uapi/linux/byteorder/big_endian.h:37:59: note: expanded from macro '__le16_to_cpu'
37 | #define __le16_to_cpu(x) __swab16((__force __u16)(__le16)(x))
| ^
include/uapi/linux/swab.h:102:54: note: expanded from macro '__swab16'
102 | #define __swab16(x) (__u16)__builtin_bswap16((__u16)(x))
| ^
In file included from net/rds/af_rds.c:38:
In file included from include/linux/ipv6.h:101:
In file included from include/linux/tcp.h:17:
In file included from include/linux/skbuff.h:28:
In file included from include/linux/dma-mapping.h:11:
In file included from include/linux/scatterlist.h:9:
In file included from arch/s390/include/asm/io.h:78:
include/asm-generic/io.h:573:61: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
573 | val = __le32_to_cpu((__le32 __force)__raw_readl(PCI_IOBASE + addr));
| ~~~~~~~~~~ ^
include/uapi/linux/byteorder/big_endian.h:35:59: note: expanded from macro '__le32_to_cpu'
35 | #define __le32_to_cpu(x) __swab32((__force __u32)(__le32)(x))
| ^
include/uapi/linux/swab.h:115:54: note: expanded from macro '__swab32'
115 | #define __swab32(x) (__u32)__builtin_bswap32((__u32)(x))
| ^
In file included from net/rds/af_rds.c:38:
In file included from include/linux/ipv6.h:101:
In file included from include/linux/tcp.h:17:
In file included from include/linux/skbuff.h:28:
In file included from include/linux/dma-mapping.h:11:
In file included from include/linux/scatterlist.h:9:
In file included from arch/s390/include/asm/io.h:78:
include/asm-generic/io.h:584:33: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
584 | __raw_writeb(value, PCI_IOBASE + addr);
| ~~~~~~~~~~ ^
include/asm-generic/io.h:594:59: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
594 | __raw_writew((u16 __force)cpu_to_le16(value), PCI_IOBASE + addr);
| ~~~~~~~~~~ ^
include/asm-generic/io.h:604:59: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
604 | __raw_writel((u32 __force)cpu_to_le32(value), PCI_IOBASE + addr);
| ~~~~~~~~~~ ^
include/asm-generic/io.h:692:20: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
692 | readsb(PCI_IOBASE + addr, buffer, count);
| ~~~~~~~~~~ ^
include/asm-generic/io.h:700:20: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
700 | readsw(PCI_IOBASE + addr, buffer, count);
| ~~~~~~~~~~ ^
include/asm-generic/io.h:708:20: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
708 | readsl(PCI_IOBASE + addr, buffer, count);
| ~~~~~~~~~~ ^
include/asm-generic/io.h:717:21: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
717 | writesb(PCI_IOBASE + addr, buffer, count);
| ~~~~~~~~~~ ^
include/asm-generic/io.h:726:21: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
726 | writesw(PCI_IOBASE + addr, buffer, count);
| ~~~~~~~~~~ ^
include/asm-generic/io.h:735:21: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
735 | writesl(PCI_IOBASE + addr, buffer, count);
| ~~~~~~~~~~ ^
>> net/rds/af_rds.c:315:15: warning: variable 'noio_flags' set but not used [-Wunused-but-set-variable]
315 | unsigned int noio_flags;
| ^
18 warnings generated.
vim +/noio_flags +315 net/rds/af_rds.c
310
311 static int rds_cancel_sent_to(struct rds_sock *rs, sockptr_t optval, int len)
312 {
313 struct sockaddr_in6 sin6;
314 struct sockaddr_in sin;
> 315 unsigned int noio_flags;
316 int ret = 0;
317
318 if (rds_force_noio)
319 noio_flags = memalloc_noio_save();
320
321 /* racing with another thread binding seems ok here */
322 if (ipv6_addr_any(&rs->rs_bound_addr)) {
323 ret = -ENOTCONN; /* XXX not a great errno */
324 goto out;
325 }
326
327 if (len < sizeof(struct sockaddr_in)) {
328 ret = -EINVAL;
329 goto out;
330 } else if (len < sizeof(struct sockaddr_in6)) {
331 /* Assume IPv4 */
332 if (copy_from_sockptr(&sin, optval,
333 sizeof(struct sockaddr_in))) {
334 ret = -EFAULT;
335 goto out;
336 }
337 ipv6_addr_set_v4mapped(sin.sin_addr.s_addr, &sin6.sin6_addr);
338 sin6.sin6_port = sin.sin_port;
339 } else {
340 if (copy_from_sockptr(&sin6, optval,
341 sizeof(struct sockaddr_in6))) {
342 ret = -EFAULT;
343 goto out;
344 }
345 }
346
347 rds_send_drop_to(rs, &sin6);
348 out:
349 if (rds_force_noio)
350 noio_flags = memalloc_noio_save();
351 return ret;
352 }
353
On Mon, May 13, 2024 at 02:53:42PM +0200, Håkon Bugge wrote: > For most entry points to RDS, we call memalloc_noio_{save,restore} in > a parenthetic fashion when enabled by the module parameter force_noio. > > We skip the calls to memalloc_noio_{save,restore} in rds_ioctl(), as > no memory allocations are executed in this function or its callees. > > The reason we execute memalloc_noio_{save,restore} in rds_poll(), is > due to the following call chain: > > rds_poll() > poll_wait() > __pollwait() > poll_get_entry() > __get_free_page(GFP_KERNEL) > > The function rds_setsockopt() allocates memory in its callee's > rds_get_mr() and rds_get_mr_for_dest(). Hence, we need > memalloc_noio_{save,restore} in rds_setsockopt(). > > In rds_getsockopt(), we have rds_info_getsockopt() that allocates > memory. Hence, we need memalloc_noio_{save,restore} in > rds_getsockopt(). > > All the above, in order to conditionally enable RDS to become a block I/O > device. > > Signed-off-by: Håkon Bugge <haakon.bugge@oracle.com> Hi Håkon, Some minor feedback from my side. > --- > net/rds/af_rds.c | 60 +++++++++++++++++++++++++++++++++++++++++++++--- > 1 file changed, 57 insertions(+), 3 deletions(-) > > diff --git a/net/rds/af_rds.c b/net/rds/af_rds.c > index 8435a20968ef5..a89d192aabc0b 100644 > --- a/net/rds/af_rds.c > +++ b/net/rds/af_rds.c > @@ -37,10 +37,16 @@ > #include <linux/in.h> > #include <linux/ipv6.h> > #include <linux/poll.h> > +#include <linux/sched/mm.h> > #include <net/sock.h> > > #include "rds.h" > > +bool rds_force_noio; > +EXPORT_SYMBOL(rds_force_noio); rds_force_noio seems to be only used within this file. I wonder if it should it be static and not EXPORTed? Flagged by Sparse. > +module_param_named(force_noio, rds_force_noio, bool, 0444); > +MODULE_PARM_DESC(force_noio, "Force the use of GFP_NOIO (Y/N)"); > + > /* this is just used for stats gathering :/ */ > static DEFINE_SPINLOCK(rds_sock_lock); > static unsigned long rds_sock_count; > @@ -60,6 +66,10 @@ static int rds_release(struct socket *sock) > { > struct sock *sk = sock->sk; > struct rds_sock *rs; > + unsigned int noio_flags; Please consider using reverse xmas tree order - longest line to shortest - for local variable declarations in Networking code. This tool can be of assistance: https://github.com/ecree-solarflare/xmastree > + > + if (rds_force_noio) > + noio_flags = memalloc_noio_save(); > > if (!sk) > goto out; ... > @@ -324,6 +346,8 @@ static int rds_cancel_sent_to(struct rds_sock *rs, sockptr_t optval, int len) > > rds_send_drop_to(rs, &sin6); > out: > + if (rds_force_noio) > + noio_flags = memalloc_noio_save(); noio_flags appears to be set but otherwise unused in this function. Flagged by W=1 builds. > return ret; > } > ...
> On Mon, May 13, 2024 at 02:53:42PM +0200, Håkon Bugge wrote: > For most entry points to RDS, we call memalloc_noio_{save,restore} in > a parenthetic fashion when enabled by the module parameter force_noio. > > We skip the calls to memalloc_noio_{save,restore} in rds_ioctl(), as > no memory allocations are executed in this function or its callees. > > The reason we execute memalloc_noio_{save,restore} in rds_poll(), is > due to the following call chain: > > rds_poll() > poll_wait() > __pollwait() > poll_get_entry() > __get_free_page(GFP_KERNEL) > > The function rds_setsockopt() allocates memory in its callee's > rds_get_mr() and rds_get_mr_for_dest(). Hence, we need > memalloc_noio_{save,restore} in rds_setsockopt(). > > In rds_getsockopt(), we have rds_info_getsockopt() that allocates > memory. Hence, we need memalloc_noio_{save,restore} in > rds_getsockopt(). > > All the above, in order to conditionally enable RDS to become a block I/O > device. > > Signed-off-by: Håkon Bugge <haakon.bugge@oracle.com> > > Hi Håkon, > > Some minor feedback from my side. > > --- > net/rds/af_rds.c | 60 +++++++++++++++++++++++++++++++++++++++++++++--- > 1 file changed, 57 insertions(+), 3 deletions(-) > > diff --git a/net/rds/af_rds.c b/net/rds/af_rds.c > index 8435a20968ef5..a89d192aabc0b 100644 > --- a/net/rds/af_rds.c > +++ b/net/rds/af_rds.c > @@ -37,10 +37,16 @@ > #include <linux/in.h> > #include <linux/ipv6.h> > #include <linux/poll.h> > +#include <linux/sched/mm.h> > #include <net/sock.h> > > #include "rds.h" > > +bool rds_force_noio; > +EXPORT_SYMBOL(rds_force_noio); > > rds_force_noio seems to be only used within this file. > I wonder if it should it be static and not EXPORTed? > > Flagged by Sparse. Hi Simon, You are quite right. Had an earlier version where the symbol was used in several files, but in this version, static is the right choice. Fixed in v2. > +module_param_named(force_noio, rds_force_noio, bool, 0444); > +MODULE_PARM_DESC(force_noio, "Force the use of GFP_NOIO (Y/N)"); > + > /* this is just used for stats gathering :/ */ > static DEFINE_SPINLOCK(rds_sock_lock); > static unsigned long rds_sock_count; > @@ -60,6 +66,10 @@ static int rds_release(struct socket *sock) > { > struct sock *sk = sock->sk; > struct rds_sock *rs; > + unsigned int noio_flags; > > Please consider using reverse xmas tree order - longest line to shortest - > for local variable declarations in Networking code. > > This tool can be of assistance: https://github.com/ecree-solarflare/xmastree Will fix. > > + > + if (rds_force_noio) > + noio_flags = memalloc_noio_save(); > > if (!sk) > goto out; > > ... > > @@ -324,6 +346,8 @@ static int rds_cancel_sent_to(struct rds_sock *rs, sockptr_t optval, int len) > > rds_send_drop_to(rs, &sin6); > out: > + if (rds_force_noio) > + noio_flags = memalloc_noio_save(); > > noio_flags appears to be set but otherwise unused in this function. Bummer. C/P error. This should be the restore. Fixed in v2. Will add W=1 for builds in the future :-) Thxs, Håkon
diff --git a/net/rds/af_rds.c b/net/rds/af_rds.c index 8435a20968ef5..a89d192aabc0b 100644 --- a/net/rds/af_rds.c +++ b/net/rds/af_rds.c @@ -37,10 +37,16 @@ #include <linux/in.h> #include <linux/ipv6.h> #include <linux/poll.h> +#include <linux/sched/mm.h> #include <net/sock.h> #include "rds.h" +bool rds_force_noio; +EXPORT_SYMBOL(rds_force_noio); +module_param_named(force_noio, rds_force_noio, bool, 0444); +MODULE_PARM_DESC(force_noio, "Force the use of GFP_NOIO (Y/N)"); + /* this is just used for stats gathering :/ */ static DEFINE_SPINLOCK(rds_sock_lock); static unsigned long rds_sock_count; @@ -60,6 +66,10 @@ static int rds_release(struct socket *sock) { struct sock *sk = sock->sk; struct rds_sock *rs; + unsigned int noio_flags; + + if (rds_force_noio) + noio_flags = memalloc_noio_save(); if (!sk) goto out; @@ -90,6 +100,8 @@ static int rds_release(struct socket *sock) sock->sk = NULL; sock_put(sk); out: + if (rds_force_noio) + memalloc_noio_restore(noio_flags); return 0; } @@ -214,9 +226,13 @@ static __poll_t rds_poll(struct file *file, struct socket *sock, { struct sock *sk = sock->sk; struct rds_sock *rs = rds_sk_to_rs(sk); + unsigned int noio_flags; __poll_t mask = 0; unsigned long flags; + if (rds_force_noio) + noio_flags = memalloc_noio_save(); + poll_wait(file, sk_sleep(sk), wait); if (rs->rs_seen_congestion) @@ -249,6 +265,8 @@ static __poll_t rds_poll(struct file *file, struct socket *sock, if (mask) rs->rs_seen_congestion = 0; + if (rds_force_noio) + memalloc_noio_restore(noio_flags); return mask; } @@ -294,8 +312,12 @@ static int rds_cancel_sent_to(struct rds_sock *rs, sockptr_t optval, int len) { struct sockaddr_in6 sin6; struct sockaddr_in sin; + unsigned int noio_flags; int ret = 0; + if (rds_force_noio) + noio_flags = memalloc_noio_save(); + /* racing with another thread binding seems ok here */ if (ipv6_addr_any(&rs->rs_bound_addr)) { ret = -ENOTCONN; /* XXX not a great errno */ @@ -324,6 +346,8 @@ static int rds_cancel_sent_to(struct rds_sock *rs, sockptr_t optval, int len) rds_send_drop_to(rs, &sin6); out: + if (rds_force_noio) + noio_flags = memalloc_noio_save(); return ret; } @@ -485,8 +509,12 @@ static int rds_getsockopt(struct socket *sock, int level, int optname, { struct rds_sock *rs = rds_sk_to_rs(sock->sk); int ret = -ENOPROTOOPT, len; + unsigned int noio_flags; int trans; + if (rds_force_noio) + noio_flags = memalloc_noio_save(); + if (level != SOL_RDS) goto out; @@ -529,6 +557,8 @@ static int rds_getsockopt(struct socket *sock, int level, int optname, } out: + if (rds_force_noio) + memalloc_noio_restore(noio_flags); return ret; } @@ -538,12 +568,16 @@ static int rds_connect(struct socket *sock, struct sockaddr *uaddr, { struct sock *sk = sock->sk; struct sockaddr_in *sin; + unsigned int noio_flags; struct rds_sock *rs = rds_sk_to_rs(sk); int ret = 0; if (addr_len < offsetofend(struct sockaddr, sa_family)) return -EINVAL; + if (rds_force_noio) + noio_flags = memalloc_noio_save(); + lock_sock(sk); switch (uaddr->sa_family) { @@ -626,6 +660,8 @@ static int rds_connect(struct socket *sock, struct sockaddr *uaddr, } release_sock(sk); + if (rds_force_noio) + memalloc_noio_restore(noio_flags); return ret; } @@ -697,16 +733,28 @@ static int __rds_create(struct socket *sock, struct sock *sk, int protocol) static int rds_create(struct net *net, struct socket *sock, int protocol, int kern) { + unsigned int noio_flags; struct sock *sk; + int ret; if (sock->type != SOCK_SEQPACKET || protocol) return -ESOCKTNOSUPPORT; + if (rds_force_noio) + noio_flags = memalloc_noio_save(); + sk = sk_alloc(net, AF_RDS, GFP_KERNEL, &rds_proto, kern); - if (!sk) - return -ENOMEM; + if (!sk) { + ret = -ENOMEM; + goto out; + } - return __rds_create(sock, sk, protocol); + ret = __rds_create(sock, sk, protocol); +out: + if (rds_force_noio) + memalloc_noio_restore(noio_flags); + + return ret; } void rds_sock_addref(struct rds_sock *rs) @@ -895,8 +943,12 @@ u32 rds_gen_num; static int __init rds_init(void) { + unsigned int noio_flags; int ret; + if (rds_force_noio) + noio_flags = memalloc_noio_save(); + net_get_random_once(&rds_gen_num, sizeof(rds_gen_num)); ret = rds_bind_lock_init(); @@ -947,6 +999,8 @@ static int __init rds_init(void) out_bind: rds_bind_lock_destroy(); out: + if (rds_force_noio) + memalloc_noio_restore(noio_flags); return ret; } module_init(rds_init);
For most entry points to RDS, we call memalloc_noio_{save,restore} in a parenthetic fashion when enabled by the module parameter force_noio. We skip the calls to memalloc_noio_{save,restore} in rds_ioctl(), as no memory allocations are executed in this function or its callees. The reason we execute memalloc_noio_{save,restore} in rds_poll(), is due to the following call chain: rds_poll() poll_wait() __pollwait() poll_get_entry() __get_free_page(GFP_KERNEL) The function rds_setsockopt() allocates memory in its callee's rds_get_mr() and rds_get_mr_for_dest(). Hence, we need memalloc_noio_{save,restore} in rds_setsockopt(). In rds_getsockopt(), we have rds_info_getsockopt() that allocates memory. Hence, we need memalloc_noio_{save,restore} in rds_getsockopt(). All the above, in order to conditionally enable RDS to become a block I/O device. Signed-off-by: Håkon Bugge <haakon.bugge@oracle.com> --- net/rds/af_rds.c | 60 +++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 57 insertions(+), 3 deletions(-)