| Message ID | 20200414150233.24495-6-willy@infradead.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Change readahead API | expand |
On Tue, 14 Apr 2020 08:02:13 -0700 Matthew Wilcox <willy@infradead.org> wrote: > From: "Matthew Wilcox (Oracle)" <willy@infradead.org> > > Filesystems which implement the upcoming ->readahead method will get > their pages by calling readahead_page() or readahead_page_batch(). > These functions support large pages, even though none of the filesystems > to be converted do yet. > > +static inline struct page *readahead_page(struct readahead_control *rac) > +static inline unsigned int __readahead_batch(struct readahead_control *rac, > + struct page **array, unsigned int array_sz) These are large functions. Was it correct to inline them? The batching API only appears to be used by fuse? If so, do we really need it? Does it provide some functional need, or is it a performance thing? If the latter, how significant is it? The code adds quite a few (inlined!) VM_BUG_ONs. Can we plan to remove them at some stage? Such as, before Linus shouts at us :)
On Tue, Apr 14, 2020 at 06:17:05PM -0700, Andrew Morton wrote: > On Tue, 14 Apr 2020 08:02:13 -0700 Matthew Wilcox <willy@infradead.org> wrote: > > From: "Matthew Wilcox (Oracle)" <willy@infradead.org> > > > > Filesystems which implement the upcoming ->readahead method will get > > their pages by calling readahead_page() or readahead_page_batch(). > > These functions support large pages, even though none of the filesystems > > to be converted do yet. > > > > +static inline struct page *readahead_page(struct readahead_control *rac) > > +static inline unsigned int __readahead_batch(struct readahead_control *rac, > > + struct page **array, unsigned int array_sz) > > These are large functions. Was it correct to inline them? Hmm. They don't seem that big to me. readahead_page, stripped of its sanity checks: + rac->_nr_pages -= rac->_batch_count; + rac->_index += rac->_batch_count; + if (!rac->_nr_pages) { + rac->_batch_count = 0; + return NULL; + } + page = xa_load(&rac->mapping->i_pages, rac->_index); + rac->_batch_count = hpage_nr_pages(page); __readahead_batch is much bigger, but it's only used by btrfs and fuse, and it seemed unfair to make everybody pay the cost for a function only used by two filesystems. > The batching API only appears to be used by fuse? If so, do we really > need it? Does it provide some functional need, or is it a performance > thing? If the latter, how significant is it? I must confess to not knowing the performance impact. If the code uses xa_load() repeatedly, it costs O(log n) each time as we walk down the tree (mitigated to a large extent by cache, of course). Using xas_for_each() keeps us at the bottom of the tree and each iteration is O(1). I'm interested to see if filesystem maintainers start to use the batch function or if they're happier sticking with the individual lookups. The batch API was originally written for use with btrfs, but it was a significant simplification to convert fuse to use it. > The code adds quite a few (inlined!) VM_BUG_ONs. Can we plan to remove > them at some stage? Such as, before Linus shouts at us :) I'd be happy to remove them. Various reviewers said things like "are you sure this can't happen?"
On Tue, 14 Apr 2020 19:18:08 -0700 Matthew Wilcox <willy@infradead.org> wrote: > On Tue, Apr 14, 2020 at 06:17:05PM -0700, Andrew Morton wrote: > > On Tue, 14 Apr 2020 08:02:13 -0700 Matthew Wilcox <willy@infradead.org> wrote: > > > From: "Matthew Wilcox (Oracle)" <willy@infradead.org> > > > > > > Filesystems which implement the upcoming ->readahead method will get > > > their pages by calling readahead_page() or readahead_page_batch(). > > > These functions support large pages, even though none of the filesystems > > > to be converted do yet. > > > > > > +static inline struct page *readahead_page(struct readahead_control *rac) > > > +static inline unsigned int __readahead_batch(struct readahead_control *rac, > > > + struct page **array, unsigned int array_sz) > > > > These are large functions. Was it correct to inline them? > > Hmm. They don't seem that big to me. They're really big! > readahead_page, stripped of its sanity checks: Well, the sanity checks still count for cache footprint. otoh, I think a function which is expected to be called from a single site per filesystem is OK to be inlined, because there's not likely to be much icache benefit unless different filesystem types are simultaneously being used heavily, which sounds unlikely. Although there's still a bit of overall code size bloat. > + rac->_index += rac->_batch_count; > + if (!rac->_nr_pages) { > + rac->_batch_count = 0; > + return NULL; > + } > + page = xa_load(&rac->mapping->i_pages, rac->_index); > + rac->_batch_count = hpage_nr_pages(page); > > __readahead_batch is much bigger, but it's only used by btrfs and fuse, > and it seemed unfair to make everybody pay the cost for a function only > used by two filesystems. Do we expect more filesystems to use these in the future? These function are really big! > > The batching API only appears to be used by fuse? If so, do we really > > need it? Does it provide some functional need, or is it a performance > > thing? If the latter, how significant is it? > > I must confess to not knowing the performance impact. If the code uses > xa_load() repeatedly, it costs O(log n) each time as we walk down the tree > (mitigated to a large extent by cache, of course). Using xas_for_each() > keeps us at the bottom of the tree and each iteration is O(1). > I'm interested to see if filesystem maintainers start to use the batch > function or if they're happier sticking with the individual lookups. > > The batch API was originally written for use with btrfs, but it was a > significant simplification to convert fuse to use it. hm, OK. It's not clear that its inclusion is justified? > > The code adds quite a few (inlined!) VM_BUG_ONs. Can we plan to remove > > them at some stage? Such as, before Linus shouts at us :) > > I'd be happy to remove them. Various reviewers said things like "are you > sure this can't happen?" Yeah, these things tend to live for ever. Please add a todo to remove them after the code has matured?
On Tue, Apr 14, 2020 at 09:56:16PM -0700, Andrew Morton wrote: > On Tue, 14 Apr 2020 19:18:08 -0700 Matthew Wilcox <willy@infradead.org> wrote: > > Hmm. They don't seem that big to me. > > They're really big! v5.7-rc1: 11636 636 224 12496 30d0 fs/iomap/buffered-io.o readahead_v11: 11528 636 224 12388 3064 fs/iomap/buffered-io.o > > __readahead_batch is much bigger, but it's only used by btrfs and fuse, > > and it seemed unfair to make everybody pay the cost for a function only > > used by two filesystems. > > Do we expect more filesystems to use these in the future? I'm honestly not sure. I think it'd be nice to be able to fill a bvec from the page cache directly, but I haven't tried to write that function yet. If so, then it'd be appropriate to move that functionality into the core. > > > The code adds quite a few (inlined!) VM_BUG_ONs. Can we plan to remove > > > them at some stage? Such as, before Linus shouts at us :) > > > > I'd be happy to remove them. Various reviewers said things like "are you > > sure this can't happen?" > > Yeah, these things tend to live for ever. Please add a todo to remove > them after the code has matured? Sure! I'm touching this code some more in the large pages patch set, so I can get rid of it there.
diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 6c61535aa7ff..a6eccfd2c80b 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -639,6 +639,146 @@ static inline int add_to_page_cache(struct page *page, return error; } +/** + * struct readahead_control - Describes a readahead request. + * + * A readahead request is for consecutive pages. Filesystems which + * implement the ->readahead method should call readahead_page() or + * readahead_page_batch() in a loop and attempt to start I/O against + * each page in the request. + * + * Most of the fields in this struct are private and should be accessed + * by the functions below. + * + * @file: The file, used primarily by network filesystems for authentication. + * May be NULL if invoked internally by the filesystem. + * @mapping: Readahead this filesystem object. + */ +struct readahead_control { + struct file *file; + struct address_space *mapping; +/* private: use the readahead_* accessors instead */ + pgoff_t _index; + unsigned int _nr_pages; + unsigned int _batch_count; +}; + +/** + * readahead_page - Get the next page to read. + * @rac: The current readahead request. + * + * Context: The page is locked and has an elevated refcount. The caller + * should decreases the refcount once the page has been submitted for I/O + * and unlock the page once all I/O to that page has completed. + * Return: A pointer to the next page, or %NULL if we are done. + */ +static inline struct page *readahead_page(struct readahead_control *rac) +{ + struct page *page; + + BUG_ON(rac->_batch_count > rac->_nr_pages); + rac->_nr_pages -= rac->_batch_count; + rac->_index += rac->_batch_count; + + if (!rac->_nr_pages) { + rac->_batch_count = 0; + return NULL; + } + + page = xa_load(&rac->mapping->i_pages, rac->_index); + VM_BUG_ON_PAGE(!PageLocked(page), page); + rac->_batch_count = hpage_nr_pages(page); + + return page; +} + +static inline unsigned int __readahead_batch(struct readahead_control *rac, + struct page **array, unsigned int array_sz) +{ + unsigned int i = 0; + XA_STATE(xas, &rac->mapping->i_pages, 0); + struct page *page; + + BUG_ON(rac->_batch_count > rac->_nr_pages); + rac->_nr_pages -= rac->_batch_count; + rac->_index += rac->_batch_count; + rac->_batch_count = 0; + + xas_set(&xas, rac->_index); + rcu_read_lock(); + xas_for_each(&xas, page, rac->_index + rac->_nr_pages - 1) { + VM_BUG_ON_PAGE(!PageLocked(page), page); + VM_BUG_ON_PAGE(PageTail(page), page); + array[i++] = page; + rac->_batch_count += hpage_nr_pages(page); + + /* + * The page cache isn't using multi-index entries yet, + * so the xas cursor needs to be manually moved to the + * next index. This can be removed once the page cache + * is converted. + */ + if (PageHead(page)) + xas_set(&xas, rac->_index + rac->_batch_count); + + if (i == array_sz) + break; + } + rcu_read_unlock(); + + return i; +} + +/** + * readahead_page_batch - Get a batch of pages to read. + * @rac: The current readahead request. + * @array: An array of pointers to struct page. + * + * Context: The pages are locked and have an elevated refcount. The caller + * should decreases the refcount once the page has been submitted for I/O + * and unlock the page once all I/O to that page has completed. + * Return: The number of pages placed in the array. 0 indicates the request + * is complete. + */ +#define readahead_page_batch(rac, array) \ + __readahead_batch(rac, array, ARRAY_SIZE(array)) + +/** + * readahead_pos - The byte offset into the file of this readahead request. + * @rac: The readahead request. + */ +static inline loff_t readahead_pos(struct readahead_control *rac) +{ + return (loff_t)rac->_index * PAGE_SIZE; +} + +/** + * readahead_length - The number of bytes in this readahead request. + * @rac: The readahead request. + */ +static inline loff_t readahead_length(struct readahead_control *rac) +{ + return (loff_t)rac->_nr_pages * PAGE_SIZE; +} + +/** + * readahead_index - The index of the first page in this readahead request. + * @rac: The readahead request. + */ +static inline pgoff_t readahead_index(struct readahead_control *rac) +{ + return rac->_index; +} + +/** + * readahead_count - The number of pages in this readahead request. + * @rac: The readahead request. + */ +static inline unsigned int readahead_count(struct readahead_control *rac) +{ + return rac->_nr_pages; +} + static inline unsigned long dir_pages(struct inode *inode) { return (unsigned long)(inode->i_size + PAGE_SIZE - 1) >>