| Message ID | faad7d7f58ff45285eaac9af7fae9a5fcca98977.1610170479.git.asml.silence@gmail.com (mailing list archive) |
|---|---|
| State | Not Applicable |
| Headers | show |
| Series | no-copy bvec | expand |
On Sat, Jan 09, 2021 at 04:02:59PM +0000, Pavel Begunkov wrote: > Direct IO does not operate on the current working set of pages managed > by the kernel, so it should not be accounted as memory stall to PSI > infrastructure. > > The block layer and iomap direct IO use bio_iov_iter_get_pages() > to build bios, and they are the only users of it, so to avoid PSI > tracking for them clear out BIO_WORKINGSET flag. Do same for > dio_bio_submit() because fs/direct_io constructs bios by hand directly > calling bio_add_page(). > > Reported-by: Christoph Hellwig <hch@infradead.org> > Suggested-by: Christoph Hellwig <hch@infradead.org> > Suggested-by: Johannes Weiner <hannes@cmpxchg.org> > Reviewed-by: Christoph Hellwig <hch@lst.de> > Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> > --- > block/bio.c | 6 ++++++ > fs/direct-io.c | 2 ++ > 2 files changed, 8 insertions(+) > > diff --git a/block/bio.c b/block/bio.c > index 1f2cc1fbe283..9f26984af643 100644 > --- a/block/bio.c > +++ b/block/bio.c > @@ -1099,6 +1099,9 @@ static int __bio_iov_append_get_pages(struct bio *bio, struct iov_iter *iter) > * fit into the bio, or are requested in @iter, whatever is smaller. If > * MM encounters an error pinning the requested pages, it stops. Error > * is returned only if 0 pages could be pinned. > + * > + * It's intended for direct IO, so doesn't do PSI tracking, the caller is > + * responsible for setting BIO_WORKINGSET if necessary. > */ > int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) > { > @@ -1123,6 +1126,9 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) > > if (is_bvec) > bio_set_flag(bio, BIO_NO_PAGE_REF); > + > + /* don't account direct I/O as memory stall */ > + bio_clear_flag(bio, BIO_WORKINGSET); > return bio->bi_vcnt ? 0 : ret; > } > EXPORT_SYMBOL_GPL(bio_iov_iter_get_pages); > diff --git a/fs/direct-io.c b/fs/direct-io.c > index d53fa92a1ab6..0e689233f2c7 100644 > --- a/fs/direct-io.c > +++ b/fs/direct-io.c > @@ -426,6 +426,8 @@ static inline void dio_bio_submit(struct dio *dio, struct dio_submit *sdio) > unsigned long flags; > > bio->bi_private = dio; > + /* don't account direct I/O as memory stall */ > + bio_clear_flag(bio, BIO_WORKINGSET); > > spin_lock_irqsave(&dio->bio_lock, flags); > dio->refcount++; > -- > 2.24.0 > Reviewed-by: Ming Lei <ming.lei@redhat.com>
diff --git a/block/bio.c b/block/bio.c index 1f2cc1fbe283..9f26984af643 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1099,6 +1099,9 @@ static int __bio_iov_append_get_pages(struct bio *bio, struct iov_iter *iter) * fit into the bio, or are requested in @iter, whatever is smaller. If * MM encounters an error pinning the requested pages, it stops. Error * is returned only if 0 pages could be pinned. + * + * It's intended for direct IO, so doesn't do PSI tracking, the caller is + * responsible for setting BIO_WORKINGSET if necessary. */ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) { @@ -1123,6 +1126,9 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) if (is_bvec) bio_set_flag(bio, BIO_NO_PAGE_REF); + + /* don't account direct I/O as memory stall */ + bio_clear_flag(bio, BIO_WORKINGSET); return bio->bi_vcnt ? 0 : ret; } EXPORT_SYMBOL_GPL(bio_iov_iter_get_pages); diff --git a/fs/direct-io.c b/fs/direct-io.c index d53fa92a1ab6..0e689233f2c7 100644 --- a/fs/direct-io.c +++ b/fs/direct-io.c @@ -426,6 +426,8 @@ static inline void dio_bio_submit(struct dio *dio, struct dio_submit *sdio) unsigned long flags; bio->bi_private = dio; + /* don't account direct I/O as memory stall */ + bio_clear_flag(bio, BIO_WORKINGSET); spin_lock_irqsave(&dio->bio_lock, flags); dio->refcount++;