diff mbox series

[1/3] dma-direct: provide the ability to reserve per-numa CMA

Message ID 20200603024231.61748-2-song.bao.hua@hisilicon.com (mailing list archive)
State New, archived
Headers show
Series support per-numa CMA for ARM server | expand

Commit Message

Song Bao Hua (Barry Song) June 3, 2020, 2:42 a.m. UTC 
This is useful for two scenarios:
1. ARM64 smmu will get memory from local numa node, it can save its
command queues and page tables locally. Tests show it can decrease
dma_unmap latency at lot. For example, without this patch, smmu on
node2 will get memory from node0 by calling dma_alloc_coherent(),
typically, it has to wait for more than 560ns for the completion of
CMD_SYNC in an empty command queue; with this patch, it needs 240ns
only.
2. when we set iommu passthrough, drivers will get memory from CMA,
local memory means much less latency.

Cc: Will Deacon <will@kernel.org>
Cc: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Barry Song <song.bao.hua@hisilicon.com>
---
 include/linux/dma-contiguous.h |  4 ++++
 kernel/dma/Kconfig             | 10 +++++++++
 kernel/dma/contiguous.c        | 41 +++++++++++++++++++++++++++++++++-
 3 files changed, 54 insertions(+), 1 deletion(-)

Comments

kernel test robot June 3, 2020, 6:55 a.m. UTC | #1 
Hi Barry,

I love your patch! Perhaps something to improve:

[auto build test WARNING on arm64/for-next/core]
[also build test WARNING on linus/master v5.7]
[cannot apply to hch-configfs/for-next next-20200602]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Barry-Song/support-per-numa-CMA-for-ARM-server/20200603-104821
base:   https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/core
config: i386-randconfig-r033-20200602 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-13) 9.3.0
reproduce (this is a W=1 build):
        # save the attached .config to linux build tree
        make W=1 ARCH=i386 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kbuild test robot <lkp@intel.com>

All warnings (new ones prefixed by >>, old ones prefixed by <<):

In file included from include/linux/kernel.h:15,
from include/asm-generic/bug.h:19,
from arch/x86/include/asm/bug.h:83,
from include/linux/bug.h:5,
from include/linux/mmdebug.h:5,
from include/linux/mm.h:9,
from include/linux/memblock.h:13,
from kernel/dma/contiguous.c:21:
kernel/dma/contiguous.c: In function 'dma_pernuma_cma_reserve':
>> include/linux/kern_levels.h:5:18: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 3 has type 'unsigned int' [-Wformat=]
5 | #define KERN_SOH "001"  /* ASCII Start Of Header */
|                  ^~~~~~
include/linux/printk.h:137:10: note: in definition of macro 'no_printk'
137 |   printk(fmt, ##__VA_ARGS__);           |          ^~~
include/linux/kern_levels.h:15:20: note: in expansion of macro 'KERN_SOH'
15 | #define KERN_DEBUG KERN_SOH "7" /* debug-level messages */
|                    ^~~~~~~~
include/linux/printk.h:336:12: note: in expansion of macro 'KERN_DEBUG'
336 |  no_printk(KERN_DEBUG pr_fmt(fmt), ##__VA_ARGS__)
|            ^~~~~~~~~~
kernel/dma/contiguous.c:128:3: note: in expansion of macro 'pr_debug'
128 |   pr_debug("%s: reserved %llu MiB on node %dn", __func__,
|   ^~~~~~~~
kernel/dma/contiguous.c:128:29: note: format string is defined here
128 |   pr_debug("%s: reserved %llu MiB on node %dn", __func__,
|                          ~~~^
|                             |
|                             long long unsigned int
|                          %u

vim +5 include/linux/kern_levels.h

314ba3520e513a Joe Perches 2012-07-30  4  
04d2c8c83d0e3a Joe Perches 2012-07-30 @5  #define KERN_SOH	"\001"		/* ASCII Start Of Header */
04d2c8c83d0e3a Joe Perches 2012-07-30  6  #define KERN_SOH_ASCII	'\001'
04d2c8c83d0e3a Joe Perches 2012-07-30  7  

:::::: The code at line 5 was first introduced by commit
:::::: 04d2c8c83d0e3ac5f78aeede51babb3236200112 printk: convert the format for KERN_<LEVEL> to a 2 byte pattern

:::::: TO: Joe Perches <joe@perches.com>
:::::: CC: Linus Torvalds <torvalds@linux-foundation.org>

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
kernel test robot June 3, 2020, 7:18 a.m. UTC | #2 
Hi Barry,

I love your patch! Perhaps something to improve:

[auto build test WARNING on arm64/for-next/core]
[also build test WARNING on linus/master v5.7]
[cannot apply to next-20200602]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Barry-Song/support-per-numa-CMA-for-ARM-server/20200603-104821
base:   https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/core
config: i386-randconfig-s002-20200602 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-13) 9.3.0
reproduce:
        # apt-get install sparse
        # sparse version: v0.6.1-244-g0ee050a8-dirty
        # save the attached .config to linux build tree
        make W=1 C=1 ARCH=i386 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__'

If you fix the issue, kindly add following tag as appropriate
Reported-by: kbuild test robot <lkp@intel.com>


sparse warnings: (new ones prefixed by >>)

>> kernel/dma/contiguous.c:40:12: sparse: sparse: symbol 'dma_contiguous_pernuma_area' was not declared. Should it be static?
>> kernel/dma/contiguous.c:278:50: sparse: sparse: invalid access below 'dma_contiguous_pernuma_area' (-4 4)

Please review and possibly fold the followup patch.

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
Dan Carpenter June 4, 2020, 11:36 a.m. UTC | #3 
Hi Barry,

url:    https://github.com/0day-ci/linux/commits/Barry-Song/support-per-numa-CMA-for-ARM-server/20200603-104821
base:   https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/core
config: x86_64-randconfig-m001-20200603 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-13) 9.3.0

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>

smatch warnings:
kernel/dma/contiguous.c:274 dma_alloc_contiguous() warn: variable dereferenced before check 'dev' (see line 272)

# https://github.com/0day-ci/linux/commit/adb919e972c1cac3d8b11905d5258d23d3aac6a4
git remote add linux-review https://github.com/0day-ci/linux
git remote update linux-review
git checkout adb919e972c1cac3d8b11905d5258d23d3aac6a4
vim +/dev +274 kernel/dma/contiguous.c

b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  267  struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp)
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  268  {
90ae409f9eb3bc kernel/dma/contiguous.c       Christoph Hellwig 2019-08-20  269  	size_t count = size >> PAGE_SHIFT;
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  270  	struct page *page = NULL;
bd2e75633c8012 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  271  	struct cma *cma = NULL;
adb919e972c1ca kernel/dma/contiguous.c       Barry Song        2020-06-03 @272  	int nid = dev_to_node(dev);
                                                                                                              ^^^
Dereferenced inside function.

bd2e75633c8012 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  273  
bd2e75633c8012 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23 @274  	if (dev && dev->cma_area)
                                                                                            ^^^
Too late.

bd2e75633c8012 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  275  		cma = dev->cma_area;
adb919e972c1ca kernel/dma/contiguous.c       Barry Song        2020-06-03  276  	else if ((nid != NUMA_NO_NODE) && dma_contiguous_pernuma_area[nid]
adb919e972c1ca kernel/dma/contiguous.c       Barry Song        2020-06-03  277  		&& !(gfp & (GFP_DMA | GFP_DMA32)))
adb919e972c1ca kernel/dma/contiguous.c       Barry Song        2020-06-03  278  		cma = dma_contiguous_pernuma_area[nid];
bd2e75633c8012 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  279  	else if (count > 1)
bd2e75633c8012 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  280  		cma = dma_contiguous_default_area;
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  281  
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  282  	/* CMA can be used only in the context which permits sleeping */
b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen      2019-05-23  283  	if (cma && gfpflags_allow_blocking(gfp)) {

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
Song Bao Hua (Barry Song) June 5, 2020, 6:04 a.m. UTC | #4 
> -----Original Message-----
> From: Dan Carpenter [mailto:dan.carpenter@oracle.com]
> Sent: Thursday, June 4, 2020 11:37 PM
> To: kbuild@lists.01.org; Song Bao Hua (Barry Song)
> <song.bao.hua@hisilicon.com>; hch@lst.de; m.szyprowski@samsung.com;
> robin.murphy@arm.com; catalin.marinas@arm.com
> Cc: lkp@intel.com; Dan Carpenter <error27@gmail.com>;
> kbuild-all@lists.01.org; iommu@lists.linux-foundation.org;
> linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org; Linuxarm
> <linuxarm@huawei.com>; Jonathan Cameron
> <jonathan.cameron@huawei.com>; John Garry <john.garry@huawei.com>
> Subject: Re: [PATCH 1/3] dma-direct: provide the ability to reserve per-numa
> CMA
> 
> Hi Barry,
> 
> url:
> https://github.com/0day-ci/linux/commits/Barry-Song/support-per-numa-CM
> A-for-ARM-server/20200603-104821
> base:   https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git
> for-next/core
> config: x86_64-randconfig-m001-20200603 (attached as .config)
> compiler: gcc-9 (Debian 9.3.0-13) 9.3.0
> 
> If you fix the issue, kindly add following tag as appropriate
> Reported-by: kernel test robot <lkp@intel.com>
> Reported-by: Dan Carpenter <dan.carpenter@oracle.com>

Dan, thanks! Good catch!
as this patch has not been in mainline yet, is it correct to add these "reported-by" in patch v2?

Barry

> 
> smatch warnings:
> kernel/dma/contiguous.c:274 dma_alloc_contiguous() warn: variable
> dereferenced before check 'dev' (see line 272)
> 
> #
> https://github.com/0day-ci/linux/commit/adb919e972c1cac3d8b11905d525
> 8d23d3aac6a4
> git remote add linux-review https://github.com/0day-ci/linux git remote
> update linux-review git checkout
> adb919e972c1cac3d8b11905d5258d23d3aac6a4
> vim +/dev +274 kernel/dma/contiguous.c
> 
> b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen
> 2019-05-23  267  struct page *dma_alloc_contiguous(struct device *dev,
> size_t size, gfp_t gfp)
> b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen
> 2019-05-23  268  {
> 90ae409f9eb3bc kernel/dma/contiguous.c       Christoph Hellwig
> 2019-08-20  269  	size_t count = size >> PAGE_SHIFT;
> b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen
> 2019-05-23  270  	struct page *page = NULL;
> bd2e75633c8012 kernel/dma/contiguous.c       Nicolin Chen
> 2019-05-23  271  	struct cma *cma = NULL;
> adb919e972c1ca kernel/dma/contiguous.c       Barry Song
> 2020-06-03 @272  	int nid = dev_to_node(dev);
> 
> ^^^ Dereferenced inside function.
> 
> bd2e75633c8012 kernel/dma/contiguous.c       Nicolin Chen
> 2019-05-23  273
> bd2e75633c8012 kernel/dma/contiguous.c       Nicolin Chen
> 2019-05-23 @274  	if (dev && dev->cma_area)
> 
> ^^^ Too late.
> 
> bd2e75633c8012 kernel/dma/contiguous.c       Nicolin Chen
> 2019-05-23  275  		cma = dev->cma_area;
> adb919e972c1ca kernel/dma/contiguous.c       Barry Song
> 2020-06-03  276  	else if ((nid != NUMA_NO_NODE) &&
> dma_contiguous_pernuma_area[nid]
> adb919e972c1ca kernel/dma/contiguous.c       Barry Song
> 2020-06-03  277  		&& !(gfp & (GFP_DMA | GFP_DMA32)))
> adb919e972c1ca kernel/dma/contiguous.c       Barry Song
> 2020-06-03  278  		cma = dma_contiguous_pernuma_area[nid];
> bd2e75633c8012 kernel/dma/contiguous.c       Nicolin Chen
> 2019-05-23  279  	else if (count > 1)
> bd2e75633c8012 kernel/dma/contiguous.c       Nicolin Chen
> 2019-05-23  280  		cma = dma_contiguous_default_area;
> b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen
> 2019-05-23  281
> b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen
> 2019-05-23  282  	/* CMA can be used only in the context which permits
> sleeping */
> b1d2dc009dece4 kernel/dma/contiguous.c       Nicolin Chen
> 2019-05-23  283  	if (cma && gfpflags_allow_blocking(gfp)) {
> 
> ---
> 0-DAY CI Kernel Test Service, Intel Corporation
> https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
Dan Carpenter June 5, 2020, 8:57 a.m. UTC | #5 
On Fri, Jun 05, 2020 at 06:04:31AM +0000, Song Bao Hua (Barry Song) wrote:
> 
> 
> > -----Original Message-----
> > From: Dan Carpenter [mailto:dan.carpenter@oracle.com]
> > Sent: Thursday, June 4, 2020 11:37 PM
> > To: kbuild@lists.01.org; Song Bao Hua (Barry Song)
> > <song.bao.hua@hisilicon.com>; hch@lst.de; m.szyprowski@samsung.com;
> > robin.murphy@arm.com; catalin.marinas@arm.com
> > Cc: lkp@intel.com; Dan Carpenter <error27@gmail.com>;
> > kbuild-all@lists.01.org; iommu@lists.linux-foundation.org;
> > linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org; Linuxarm
> > <linuxarm@huawei.com>; Jonathan Cameron
> > <jonathan.cameron@huawei.com>; John Garry <john.garry@huawei.com>
> > Subject: Re: [PATCH 1/3] dma-direct: provide the ability to reserve per-numa
> > CMA
> > 
> > Hi Barry,
> > 
> > url:
> > https://github.com/0day-ci/linux/commits/Barry-Song/support-per-numa-CM
> > A-for-ARM-server/20200603-104821
> > base:   https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git
> > for-next/core
> > config: x86_64-randconfig-m001-20200603 (attached as .config)
> > compiler: gcc-9 (Debian 9.3.0-13) 9.3.0
> > 
> > If you fix the issue, kindly add following tag as appropriate
> > Reported-by: kernel test robot <lkp@intel.com>
> > Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
> 
> Dan, thanks! Good catch!
> as this patch has not been in mainline yet, is it correct to add these "reported-by" in patch v2?

These are just an automated email from the zero day bot.  I look over
the Smatch warnings and then forward them on.

regards,
dan carpenter
kernel test robot June 5, 2020, 1:57 p.m. UTC | #6 
Hi Barry,

I love your patch! Perhaps something to improve:

[auto build test WARNING on arm64/for-next/core]
[also build test WARNING on linus/master v5.7]
[cannot apply to next-20200605]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Barry-Song/support-per-numa-CMA-for-ARM-server/20200603-104821
base:   https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/core
config: i386-randconfig-r016-20200605 (attached as .config)
compiler: gcc-4.9 (Ubuntu 4.9.3-13ubuntu2) 4.9.3
reproduce (this is a W=1 build):
        # save the attached .config to linux build tree
        make W=1 ARCH=i386 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All warnings (new ones prefixed by >>, old ones prefixed by <<):

In file included from include/linux/printk.h:326:0,
from include/linux/kernel.h:15,
from include/asm-generic/bug.h:19,
from arch/x86/include/asm/bug.h:83,
from include/linux/bug.h:5,
from include/linux/mmdebug.h:5,
from include/linux/mm.h:9,
from include/linux/memblock.h:13,
from kernel/dma/contiguous.c:21:
kernel/dma/contiguous.c: In function 'dma_pernuma_cma_reserve':
>> include/linux/dynamic_debug.h:82:16: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 4 has type 'unsigned int' [-Wformat=]
static struct _ddebug  __aligned(8)                      ^
include/linux/dynamic_debug.h:123:2: note: in expansion of macro 'DEFINE_DYNAMIC_DEBUG_METADATA'
DEFINE_DYNAMIC_DEBUG_METADATA(id, fmt);       ^
include/linux/dynamic_debug.h:143:2: note: in expansion of macro '__dynamic_func_call'
__dynamic_func_call(__UNIQUE_ID(ddebug), fmt, func, ##__VA_ARGS__)
^
include/linux/dynamic_debug.h:153:2: note: in expansion of macro '_dynamic_func_call'
_dynamic_func_call(fmt, __dynamic_pr_debug,       ^
include/linux/printk.h:330:2: note: in expansion of macro 'dynamic_pr_debug'
dynamic_pr_debug(fmt, ##__VA_ARGS__)
^
kernel/dma/contiguous.c:128:3: note: in expansion of macro 'pr_debug'
pr_debug("%s: reserved %llu MiB on node %dn", __func__,
^

vim +82 include/linux/dynamic_debug.h

923abb9d797ba0 Gal Pressman     2019-05-01  75  
923abb9d797ba0 Gal Pressman     2019-05-01  76  extern __printf(3, 4)
923abb9d797ba0 Gal Pressman     2019-05-01  77  void __dynamic_ibdev_dbg(struct _ddebug *descriptor,
923abb9d797ba0 Gal Pressman     2019-05-01  78  			 const struct ib_device *ibdev,
923abb9d797ba0 Gal Pressman     2019-05-01  79  			 const char *fmt, ...);
923abb9d797ba0 Gal Pressman     2019-05-01  80  
2bdde670beedf7 Rasmus Villemoes 2019-03-07  81  #define DEFINE_DYNAMIC_DEBUG_METADATA(name, fmt)		\
c0d2af63786394 Joe Perches      2012-10-18 @82  	static struct _ddebug  __aligned(8)			\
07613b0b5ef857 Jason Baron      2011-10-04  83  	__attribute__((section("__verbose"))) name = {		\
07613b0b5ef857 Jason Baron      2011-10-04  84  		.modname = KBUILD_MODNAME,			\
07613b0b5ef857 Jason Baron      2011-10-04  85  		.function = __func__,				\
07613b0b5ef857 Jason Baron      2011-10-04  86  		.filename = __FILE__,				\
07613b0b5ef857 Jason Baron      2011-10-04  87  		.format = (fmt),				\
07613b0b5ef857 Jason Baron      2011-10-04  88  		.lineno = __LINE__,				\
07613b0b5ef857 Jason Baron      2011-10-04  89  		.flags = _DPRINTK_FLAGS_DEFAULT,		\
2bdde670beedf7 Rasmus Villemoes 2019-03-07  90  		_DPRINTK_KEY_INIT				\
07613b0b5ef857 Jason Baron      2011-10-04  91  	}
07613b0b5ef857 Jason Baron      2011-10-04  92  

:::::: The code at line 82 was first introduced by commit
:::::: c0d2af637863940b1a4fb208224ca7acb905c39f dynamic_debug: Remove unnecessary __used

:::::: TO: Joe Perches <joe@perches.com>
:::::: CC: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
Philip Li June 6, 2020, 3:46 a.m. UTC | #7 
On Fri, Jun 05, 2020 at 11:57:51AM +0300, Dan Carpenter wrote:
> On Fri, Jun 05, 2020 at 06:04:31AM +0000, Song Bao Hua (Barry Song) wrote:
> > 
> > 
> > > -----Original Message-----
> > > From: Dan Carpenter [mailto:dan.carpenter@oracle.com]
> > > Sent: Thursday, June 4, 2020 11:37 PM
> > > To: kbuild@lists.01.org; Song Bao Hua (Barry Song)
> > > <song.bao.hua@hisilicon.com>; hch@lst.de; m.szyprowski@samsung.com;
> > > robin.murphy@arm.com; catalin.marinas@arm.com
> > > Cc: lkp@intel.com; Dan Carpenter <error27@gmail.com>;
> > > kbuild-all@lists.01.org; iommu@lists.linux-foundation.org;
> > > linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org; Linuxarm
> > > <linuxarm@huawei.com>; Jonathan Cameron
> > > <jonathan.cameron@huawei.com>; John Garry <john.garry@huawei.com>
> > > Subject: Re: [PATCH 1/3] dma-direct: provide the ability to reserve per-numa
> > > CMA
> > > 
> > > Hi Barry,
> > > 
> > > url:
> > > https://github.com/0day-ci/linux/commits/Barry-Song/support-per-numa-CM
> > > A-for-ARM-server/20200603-104821
> > > base:   https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git
> > > for-next/core
> > > config: x86_64-randconfig-m001-20200603 (attached as .config)
> > > compiler: gcc-9 (Debian 9.3.0-13) 9.3.0
> > > 
> > > If you fix the issue, kindly add following tag as appropriate
> > > Reported-by: kernel test robot <lkp@intel.com>
> > > Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
> > 
> > Dan, thanks! Good catch!
> > as this patch has not been in mainline yet, is it correct to add these "reported-by" in patch v2?
Hi Barry, we provides the suggestion here, but you may decide
to add or not as appropriate for your situation. For the
patch still under development, it is not that necessary to add i think.

> 
> These are just an automated email from the zero day bot.  I look over
> the Smatch warnings and then forward them on.
> 
> regards,
> dan carpenter
> _______________________________________________
> kbuild-all mailing list -- kbuild-all@lists.01.org
> To unsubscribe send an email to kbuild-all-leave@lists.01.org
Song Bao Hua (Barry Song) June 6, 2020, 10:15 a.m. UTC | #8 
> -----Original Message-----
> From: Philip Li [mailto:philip.li@intel.com]
> Sent: Saturday, June 6, 2020 3:47 PM
> To: Dan Carpenter <dan.carpenter@oracle.com>
> Cc: Song Bao Hua (Barry Song) <song.bao.hua@hisilicon.com>;
> kbuild@lists.01.org; hch@lst.de; m.szyprowski@samsung.com;
> robin.murphy@arm.com; catalin.marinas@arm.com; lkp@intel.com; Dan
> Carpenter <error27@gmail.com>; kbuild-all@lists.01.org;
> iommu@lists.linux-foundation.org; linux-arm-kernel@lists.infradead.org;
> linux-kernel@vger.kernel.org; Linuxarm <linuxarm@huawei.com>; Jonathan
> Cameron <jonathan.cameron@huawei.com>; John Garry
> <john.garry@huawei.com>
> Subject: Re: [kbuild-all] Re: [PATCH 1/3] dma-direct: provide the ability to
> reserve per-numa CMA
> 
> On Fri, Jun 05, 2020 at 11:57:51AM +0300, Dan Carpenter wrote:
> > On Fri, Jun 05, 2020 at 06:04:31AM +0000, Song Bao Hua (Barry Song)
> wrote:
> > >
> > >
> > > > -----Original Message-----
> > > > From: Dan Carpenter [mailto:dan.carpenter@oracle.com]
> > > > Sent: Thursday, June 4, 2020 11:37 PM
> > > > To: kbuild@lists.01.org; Song Bao Hua (Barry Song)
> > > > <song.bao.hua@hisilicon.com>; hch@lst.de;
> > > > m.szyprowski@samsung.com; robin.murphy@arm.com;
> > > > catalin.marinas@arm.com
> > > > Cc: lkp@intel.com; Dan Carpenter <error27@gmail.com>;
> > > > kbuild-all@lists.01.org; iommu@lists.linux-foundation.org;
> > > > linux-arm-kernel@lists.infradead.org;
> > > > linux-kernel@vger.kernel.org; Linuxarm <linuxarm@huawei.com>;
> > > > Jonathan Cameron <jonathan.cameron@huawei.com>; John Garry
> > > > <john.garry@huawei.com>
> > > > Subject: Re: [PATCH 1/3] dma-direct: provide the ability to
> > > > reserve per-numa CMA
> > > >
> > > > Hi Barry,
> > > >
> > > > url:
> > > > https://github.com/0day-ci/linux/commits/Barry-Song/support-per-nu
> > > > ma-CM
> > > > A-for-ARM-server/20200603-104821
> > > > base:   https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git
> > > > for-next/core
> > > > config: x86_64-randconfig-m001-20200603 (attached as .config)
> > > > compiler: gcc-9 (Debian 9.3.0-13) 9.3.0
> > > >
> > > > If you fix the issue, kindly add following tag as appropriate
> > > > Reported-by: kernel test robot <lkp@intel.com>
> > > > Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
> > >
> > > Dan, thanks! Good catch!
> > > as this patch has not been in mainline yet, is it correct to add these
> "reported-by" in patch v2?
> Hi Barry, we provides the suggestion here, but you may decide to add or not as
> appropriate for your situation. For the patch still under development, it is not
> that necessary to add i think.

Hi Philip, Dan,
Thanks for clarification.
> 
> >
> > These are just an automated email from the zero day bot.  I look over
> > the Smatch warnings and then forward them on.
> >
> > regards,
> > dan carpenter

Best regards
Barry
diff mbox series

Patch

diff --git a/include/linux/dma-contiguous.h b/include/linux/dma-contiguous.h
index 03f8e98e3bcc..278a80a40456 100644
--- a/include/linux/dma-contiguous.h
+++ b/include/linux/dma-contiguous.h
@@ -79,6 +79,8 @@  static inline void dma_contiguous_set_default(struct cma *cma)
 
 void dma_contiguous_reserve(phys_addr_t addr_limit);
 
+void dma_pernuma_cma_reserve(void);
+
 int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
 				       phys_addr_t limit, struct cma **res_cma,
 				       bool fixed);
@@ -128,6 +130,8 @@  static inline void dma_contiguous_set_default(struct cma *cma) { }
 
 static inline void dma_contiguous_reserve(phys_addr_t limit) { }
 
+static inline void dma_pernuma_cma_reserve(void) { }
+
 static inline int dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
 				       phys_addr_t limit, struct cma **res_cma,
 				       bool fixed)
diff --git a/kernel/dma/Kconfig b/kernel/dma/Kconfig
index 4c103a24e380..14f4a1fe4285 100644
--- a/kernel/dma/Kconfig
+++ b/kernel/dma/Kconfig
@@ -100,6 +100,16 @@  config DMA_CMA
 if  DMA_CMA
 comment "Default contiguous memory area size:"
 
+config CMA_PERNUMA_SIZE_MBYTES
+	int "Size in Mega Bytes for per-numa CMA areas"
+	depends on NUMA
+	default 16 if ARM64
+	default 0
+	help
+	  Defines the size (in MiB) of the per-numa memory area for Contiguous
+	  Memory Allocator. Every numa node will get a separate CMA with this
+	  size. If the size of 0 is selected, per-numa CMA is disabled.
+
 config CMA_SIZE_MBYTES
 	int "Size in Mega Bytes"
 	depends on !CMA_SIZE_SEL_PERCENTAGE
diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c
index 8bc6f2d670f9..4b10d0ca0456 100644
--- a/kernel/dma/contiguous.c
+++ b/kernel/dma/contiguous.c
@@ -30,7 +30,14 @@ 
 #define CMA_SIZE_MBYTES 0
 #endif
 
+#ifdef CONFIG_CMA_PERNUMA_SIZE_MBYTES
+#define CMA_SIZE_PERNUMA_MBYTES CONFIG_CMA_PERNUMA_SIZE_MBYTES
+#else
+#define CMA_SIZE_PERNUMA_MBYTES 0
+#endif
+
 struct cma *dma_contiguous_default_area;
+struct cma *dma_contiguous_pernuma_area[MAX_NUMNODES];
 
 /*
  * Default global CMA area size can be defined in kernel's .config.
@@ -44,6 +51,8 @@  struct cma *dma_contiguous_default_area;
  */
 static const phys_addr_t size_bytes __initconst =
 	(phys_addr_t)CMA_SIZE_MBYTES * SZ_1M;
+static const phys_addr_t pernuma_size_bytes __initconst =
+	(phys_addr_t)CMA_SIZE_PERNUMA_MBYTES * SZ_1M;
 static phys_addr_t  size_cmdline __initdata = -1;
 static phys_addr_t base_cmdline __initdata;
 static phys_addr_t limit_cmdline __initdata;
@@ -96,6 +105,31 @@  static inline __maybe_unused phys_addr_t cma_early_percent_memory(void)
 
 #endif
 
+void __init dma_pernuma_cma_reserve(void)
+{
+	int nid;
+
+	if (!pernuma_size_bytes || nr_online_nodes <= 1)
+		return;
+
+	for_each_node_state(nid, N_ONLINE) {
+		int ret;
+
+		ret = cma_declare_contiguous_nid(0, pernuma_size_bytes, 0, 0,
+						 0, false, "pernuma",
+						 &dma_contiguous_pernuma_area[nid],
+						 nid);
+		if (ret) {
+			pr_warn("%s: reservation failed: err %d, node %d", __func__,
+				ret, nid);
+			continue;
+		}
+
+		pr_debug("%s: reserved %llu MiB on node %d\n", __func__,
+			pernuma_size_bytes / SZ_1M, nid);
+	}
+}
+
 /**
  * dma_contiguous_reserve() - reserve area(s) for contiguous memory handling
  * @limit: End address of the reserved memory (optional, 0 for any).
@@ -223,7 +257,8 @@  bool dma_release_from_contiguous(struct device *dev, struct page *pages,
  *
  * This function allocates contiguous memory buffer for specified device. It
  * first tries to use device specific contiguous memory area if available or
- * the default global one, then tries a fallback allocation of normal pages.
+ * the per-numa ones and default global one, then tries a fallback allocation
+ * of normal pages. per-numa memory areas don't support address limit
  *
  * Note that it byapss one-page size of allocations from the global area as
  * the addresses within one page are always contiguous, so there is no need
@@ -234,9 +269,13 @@  struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp)
 	size_t count = size >> PAGE_SHIFT;
 	struct page *page = NULL;
 	struct cma *cma = NULL;
+	int nid = dev_to_node(dev);
 
 	if (dev && dev->cma_area)
 		cma = dev->cma_area;
+	else if ((nid != NUMA_NO_NODE) && dma_contiguous_pernuma_area[nid]
+		&& !(gfp & (GFP_DMA | GFP_DMA32)))
+		cma = dma_contiguous_pernuma_area[nid];
 	else if (count > 1)
 		cma = dma_contiguous_default_area;