| Message ID | 20220608150249.3033815-4-willy@infradead.org (mailing list archive) |
|---|---|
| State | Deferred, archived |
| Headers | show |
| Series | Convert aops->migratepage to aops->migrate_folio | expand |
On 08.06.22 17:02, Matthew Wilcox (Oracle) wrote: > Provide a folio-based replacement for aops->migratepage. Update the > documentation to document migrate_folio instead of migratepage. > > Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> > Reviewed-by: Christoph Hellwig <hch@lst.de> > --- > Documentation/filesystems/locking.rst | 5 ++-- > Documentation/filesystems/vfs.rst | 13 ++++++----- > Documentation/vm/page_migration.rst | 33 ++++++++++++++------------- > include/linux/fs.h | 4 +++- > mm/compaction.c | 4 +++- > mm/migrate.c | 11 +++++---- > 6 files changed, 40 insertions(+), 30 deletions(-) > > diff --git a/Documentation/filesystems/locking.rst b/Documentation/filesystems/locking.rst > index c0fe711f14d3..3d28b23676bd 100644 > --- a/Documentation/filesystems/locking.rst > +++ b/Documentation/filesystems/locking.rst > @@ -253,7 +253,8 @@ prototypes:: > void (*free_folio)(struct folio *); > int (*direct_IO)(struct kiocb *, struct iov_iter *iter); > bool (*isolate_page) (struct page *, isolate_mode_t); > - int (*migratepage)(struct address_space *, struct page *, struct page *); > + int (*migrate_folio)(struct address_space *, struct folio *dst, > + struct folio *src, enum migrate_mode); > void (*putback_page) (struct page *); isolate_page/putback_page are leftovers from the previous patch, no? > int (*launder_folio)(struct folio *); > bool (*is_partially_uptodate)(struct folio *, size_t from, size_t count); > @@ -281,7 +282,7 @@ release_folio: yes > free_folio: yes > direct_IO: > isolate_page: yes > -migratepage: yes (both) > +migrate_folio: yes (both) > putback_page: yes Dito. > launder_folio: yes > is_partially_uptodate: yes > diff --git a/Documentation/filesystems/vfs.rst b/Documentation/filesystems/vfs.rst > index a08c652467d7..3ae1b039b03f 100644 > --- a/Documentation/filesystems/vfs.rst > +++ b/Documentation/filesystems/vfs.rst > @@ -740,7 +740,8 @@ cache in your filesystem. The following members are defined: > /* isolate a page for migration */ > bool (*isolate_page) (struct page *, isolate_mode_t); > /* migrate the contents of a page to the specified target */ > - int (*migratepage) (struct page *, struct page *); > + int (*migrate_folio)(struct mapping *, struct folio *dst, > + struct folio *src, enum migrate_mode); > /* put migration-failed page back to right list */ > void (*putback_page) (struct page *); Dito. > int (*launder_folio) (struct folio *); > @@ -935,12 +936,12 @@ cache in your filesystem. The following members are defined: > is successfully isolated, VM marks the page as PG_isolated via > __SetPageIsolated. > > -``migrate_page`` > +``migrate_folio`` > This is used to compact the physical memory usage. If the VM > - wants to relocate a page (maybe off a memory card that is > - signalling imminent failure) it will pass a new page and an old > - page to this function. migrate_page should transfer any private > - data across and update any references that it has to the page. > + wants to relocate a folio (maybe from a memory device that is > + signalling imminent failure) it will pass a new folio and an old > + folio to this function. migrate_folio should transfer any private > + data across and update any references that it has to the folio. > > ``putback_page`` > Called by the VM when isolated page's migration fails. Dito. > diff --git a/Documentation/vm/page_migration.rst b/Documentation/vm/page_migration.rst > index 8c5cb8147e55..e0f73ddfabb1 100644 > --- a/Documentation/vm/page_migration.rst > +++ b/Documentation/vm/page_migration.rst > @@ -181,22 +181,23 @@ which are function pointers of struct address_space_operations. > Once page is successfully isolated, VM uses page.lru fields so driver > shouldn't expect to preserve values in those fields. > > -2. ``int (*migratepage) (struct address_space *mapping,`` > -| ``struct page *newpage, struct page *oldpage, enum migrate_mode);`` > - > - After isolation, VM calls migratepage() of driver with the isolated page. > - The function of migratepage() is to move the contents of the old page to the > - new page > - and set up fields of struct page newpage. Keep in mind that you should > - indicate to the VM the oldpage is no longer movable via __ClearPageMovable() > - under page_lock if you migrated the oldpage successfully and returned > - MIGRATEPAGE_SUCCESS. If driver cannot migrate the page at the moment, driver > - can return -EAGAIN. On -EAGAIN, VM will retry page migration in a short time > - because VM interprets -EAGAIN as "temporary migration failure". On returning > - any error except -EAGAIN, VM will give up the page migration without > - retrying. > - > - Driver shouldn't touch the page.lru field while in the migratepage() function. > +2. ``int (*migrate_folio) (struct address_space *mapping,`` > +| ``struct folio *dst, struct folio *src, enum migrate_mode);`` > + > + After isolation, VM calls the driver's migrate_folio() with the > + isolated folio. The purpose of migrate_folio() is to move the contents > + of the source folio to the destination folio and set up the fields > + of destination folio. Keep in mind that you should indicate to the > + VM the source folio is no longer movable via __ClearPageMovable() > + under folio if you migrated the source successfully and returned > + MIGRATEPAGE_SUCCESS. If driver cannot migrate the folio at the > + moment, driver can return -EAGAIN. On -EAGAIN, VM will retry folio > + migration in a short time because VM interprets -EAGAIN as "temporary > + migration failure". On returning any error except -EAGAIN, VM will > + give up the folio migration without retrying. > + > + Driver shouldn't touch the folio.lru field while in the migrate_folio() > + function. > > 3. ``void (*putback_page)(struct page *);`` Hmm, here it's a bit more complicated now, because we essentially have two paths: LRU+migrate_folio or !LRU+movable_ops (isolate/migrate/putback page)
On Thu, Jun 09, 2022 at 02:50:20PM +0200, David Hildenbrand wrote: > On 08.06.22 17:02, Matthew Wilcox (Oracle) wrote: > > diff --git a/Documentation/filesystems/locking.rst b/Documentation/filesystems/locking.rst > > index c0fe711f14d3..3d28b23676bd 100644 > > --- a/Documentation/filesystems/locking.rst > > +++ b/Documentation/filesystems/locking.rst > > @@ -253,7 +253,8 @@ prototypes:: > > void (*free_folio)(struct folio *); > > int (*direct_IO)(struct kiocb *, struct iov_iter *iter); > > bool (*isolate_page) (struct page *, isolate_mode_t); > > - int (*migratepage)(struct address_space *, struct page *, struct page *); > > + int (*migrate_folio)(struct address_space *, struct folio *dst, > > + struct folio *src, enum migrate_mode); > > void (*putback_page) (struct page *); > > isolate_page/putback_page are leftovers from the previous patch, no? Argh, right, I completely forgot I needed to update the documentation in that patch. > > +++ b/Documentation/vm/page_migration.rst > > @@ -181,22 +181,23 @@ which are function pointers of struct address_space_operations. > > Once page is successfully isolated, VM uses page.lru fields so driver > > shouldn't expect to preserve values in those fields. > > > > -2. ``int (*migratepage) (struct address_space *mapping,`` > > -| ``struct page *newpage, struct page *oldpage, enum migrate_mode);`` > > - > > - After isolation, VM calls migratepage() of driver with the isolated page. > > - The function of migratepage() is to move the contents of the old page to the > > - new page > > - and set up fields of struct page newpage. Keep in mind that you should > > - indicate to the VM the oldpage is no longer movable via __ClearPageMovable() > > - under page_lock if you migrated the oldpage successfully and returned > > - MIGRATEPAGE_SUCCESS. If driver cannot migrate the page at the moment, driver > > - can return -EAGAIN. On -EAGAIN, VM will retry page migration in a short time > > - because VM interprets -EAGAIN as "temporary migration failure". On returning > > - any error except -EAGAIN, VM will give up the page migration without > > - retrying. > > - > > - Driver shouldn't touch the page.lru field while in the migratepage() function. > > +2. ``int (*migrate_folio) (struct address_space *mapping,`` > > +| ``struct folio *dst, struct folio *src, enum migrate_mode);`` > > + > > + After isolation, VM calls the driver's migrate_folio() with the > > + isolated folio. The purpose of migrate_folio() is to move the contents > > + of the source folio to the destination folio and set up the fields > > + of destination folio. Keep in mind that you should indicate to the > > + VM the source folio is no longer movable via __ClearPageMovable() > > + under folio if you migrated the source successfully and returned > > + MIGRATEPAGE_SUCCESS. If driver cannot migrate the folio at the > > + moment, driver can return -EAGAIN. On -EAGAIN, VM will retry folio > > + migration in a short time because VM interprets -EAGAIN as "temporary > > + migration failure". On returning any error except -EAGAIN, VM will > > + give up the folio migration without retrying. > > + > > + Driver shouldn't touch the folio.lru field while in the migrate_folio() > > + function. > > > > 3. ``void (*putback_page)(struct page *);`` > > Hmm, here it's a bit more complicated now, because we essentially have > two paths: LRU+migrate_folio or !LRU+movable_ops > (isolate/migrate/putback page) Oh ... actually, this is just documenting the driver side of things. I don't really like how it's written. Here, have some rewritten documentation (which is now part of the previous patch): +++ b/Documentation/vm/page_migration.rst @@ -152,110 +152,15 @@ Steps: Non-LRU page migration ====================== -Although migration originally aimed for reducing the latency of memory accesses -for NUMA, compaction also uses migration to create high-order pages. +Although migration originally aimed for reducing the latency of memory +accesses for NUMA, compaction also uses migration to create high-order +pages. For compaction purposes, it is also useful to be able to move +non-LRU pages, such as zsmalloc and virtio-balloon pages. -Current problem of the implementation is that it is designed to migrate only -*LRU* pages. However, there are potential non-LRU pages which can be migrated -in drivers, for example, zsmalloc, virtio-balloon pages. - -For virtio-balloon pages, some parts of migration code path have been hooked -up and added virtio-balloon specific functions to intercept migration logics. -It's too specific to a driver so other drivers who want to make their pages -movable would have to add their own specific hooks in the migration path. - -To overcome the problem, VM supports non-LRU page migration which provides -generic functions for non-LRU movable pages without driver specific hooks -in the migration path. - -If a driver wants to make its pages movable, it should define three functions -which are function pointers of struct address_space_operations. - -1. ``bool (*isolate_page) (struct page *page, isolate_mode_t mode);`` - - What VM expects from isolate_page() function of driver is to return *true* - if driver isolates the page successfully. On returning true, VM marks the page - as PG_isolated so concurrent isolation in several CPUs skip the page - for isolation. If a driver cannot isolate the page, it should return *false*. - - Once page is successfully isolated, VM uses page.lru fields so driver - shouldn't expect to preserve values in those fields. - -2. ``int (*migratepage) (struct address_space *mapping,`` -| ``struct page *newpage, struct page *oldpage, enum migrate_mode);`` - - After isolation, VM calls migratepage() of driver with the isolated page. - The function of migratepage() is to move the contents of the old page to the - new page - and set up fields of struct page newpage. Keep in mind that you should - indicate to the VM the oldpage is no longer movable via __ClearPageMovable() - under page_lock if you migrated the oldpage successfully and returned - MIGRATEPAGE_SUCCESS. If driver cannot migrate the page at the moment, driver - can return -EAGAIN. On -EAGAIN, VM will retry page migration in a short time - because VM interprets -EAGAIN as "temporary migration failure". On returning - any error except -EAGAIN, VM will give up the page migration without - retrying. - - Driver shouldn't touch the page.lru field while in the migratepage() function. - -3. ``void (*putback_page)(struct page *);`` - - If migration fails on the isolated page, VM should return the isolated page - to the driver so VM calls the driver's putback_page() with the isolated page. - In this function, the driver should put the isolated page back into its own data - structure. - -Non-LRU movable page flags - - There are two page flags for supporting non-LRU movable page. - - * PG_movable - - Driver should use the function below to make page movable under page_lock:: - - void __SetPageMovable(struct page *page, struct address_space *mapping) - - It needs argument of address_space for registering migration - family functions which will be called by VM. Exactly speaking, - PG_movable is not a real flag of struct page. Rather, VM - reuses the page->mapping's lower bits to represent it:: - - #define PAGE_MAPPING_MOVABLE 0x2 - page->mapping = page->mapping | PAGE_MAPPING_MOVABLE; - - so driver shouldn't access page->mapping directly. Instead, driver should - use page_mapping() which masks off the low two bits of page->mapping under - page lock so it can get the right struct address_space. - - For testing of non-LRU movable pages, VM supports __PageMovable() function. - However, it doesn't guarantee to identify non-LRU movable pages because - the page->mapping field is unified with other variables in struct page. - If the driver releases the page after isolation by VM, page->mapping - doesn't have a stable value although it has PAGE_MAPPING_MOVABLE set - (look at __ClearPageMovable). But __PageMovable() is cheap to call whether - page is LRU or non-LRU movable once the page has been isolated because LRU - pages can never have PAGE_MAPPING_MOVABLE set in page->mapping. It is also - good for just peeking to test non-LRU movable pages before more expensive - checking with lock_page() in pfn scanning to select a victim. - - For guaranteeing non-LRU movable page, VM provides PageMovable() function. - Unlike __PageMovable(), PageMovable() validates page->mapping and - mapping->a_ops->isolate_page under lock_page(). The lock_page() prevents - sudden destroying of page->mapping. - - Drivers using __SetPageMovable() should clear the flag via - __ClearMovablePage() under page_lock() before the releasing the page. - - * PG_isolated - - To prevent concurrent isolation among several CPUs, VM marks isolated page - as PG_isolated under lock_page(). So if a CPU encounters PG_isolated - non-LRU movable page, it can skip it. Driver doesn't need to manipulate the - flag because VM will set/clear it automatically. Keep in mind that if the - driver sees a PG_isolated page, it means the page has been isolated by the - VM so it shouldn't touch the page.lru field. - The PG_isolated flag is aliased with the PG_reclaim flag so drivers - shouldn't use PG_isolated for its own purposes. +If a driver wants to make its pages movable, it should define a struct +movable_operations. It then needs to call __SetPageMovable() on each +page that it may be able to move. This uses the ``page->mapping`` field, +so this field is not available for the driver to use for other purposes. Monitoring Migration ===================== @@ -286,3 +191,5 @@ THP_MIGRATION_FAIL and PGMIGRATE_FAIL to increase. Christoph Lameter, May 8, 2006. Minchan Kim, Mar 28, 2016. + +.. kernel-doc:: include/linux/migrate.h +++ b/include/linux/migrate.h @@ -19,6 +19,43 @@ struct migration_target_control; */ #define MIGRATEPAGE_SUCCESS 0 +/** + * struct movable_operations - Driver page migration + * @isolate_page: + * The VM calls this function to prepare the page to be moved. The page + * is locked and the driver should not unlock it. The driver should + * return ``true`` if the page is movable and ``false`` if it is not + * currently movable. After this function returns, the VM uses the + * page->lru field, so the driver must preserve any information which + * is usually stored here. + * + * @migrate_page: + * After isolation, the VM calls this function with the isolated + * @src page. The driver should copy the contents of the + * @src page to the @dst page and set up the fields of @dst page. + * Both pages are locked. + * If page migration is successful, the driver should call + * __ClearPageMovable(@src) and return MIGRATEPAGE_SUCCESS. + * If the driver cannot migrate the page at the moment, it can return + * -EAGAIN. The VM interprets this as a temporary migration failure and + * will retry it later. Any other error value is a permanent migration + * failure and migration will not be retried. + * The driver shouldn't touch the @src->lru field while in the + * migrate_page() function. It may write to @dst->lru. + * + * @putback_page: + * If migration fails on the isolated page, the VM informs the driver + * that the page is no longer a candidate for migration by calling + * this function. The driver should put the isolated page back into + * its own data structure. + */ +struct movable_operations { + bool (*isolate_page)(struct page *, isolate_mode_t); + int (*migrate_page)(struct page *dst, struct page *src, + enum migrate_mode); + void (*putback_page)(struct page *); +}; + /* Defined in mm/debug.c: */ extern const char *migrate_reason_names[MR_TYPES];
On 09.06.22 16:35, Matthew Wilcox wrote: > On Thu, Jun 09, 2022 at 02:50:20PM +0200, David Hildenbrand wrote: >> On 08.06.22 17:02, Matthew Wilcox (Oracle) wrote: >>> diff --git a/Documentation/filesystems/locking.rst b/Documentation/filesystems/locking.rst >>> index c0fe711f14d3..3d28b23676bd 100644 >>> --- a/Documentation/filesystems/locking.rst >>> +++ b/Documentation/filesystems/locking.rst >>> @@ -253,7 +253,8 @@ prototypes:: >>> void (*free_folio)(struct folio *); >>> int (*direct_IO)(struct kiocb *, struct iov_iter *iter); >>> bool (*isolate_page) (struct page *, isolate_mode_t); >>> - int (*migratepage)(struct address_space *, struct page *, struct page *); >>> + int (*migrate_folio)(struct address_space *, struct folio *dst, >>> + struct folio *src, enum migrate_mode); >>> void (*putback_page) (struct page *); >> >> isolate_page/putback_page are leftovers from the previous patch, no? > > Argh, right, I completely forgot I needed to update the documentation in > that patch. > >>> +++ b/Documentation/vm/page_migration.rst >>> @@ -181,22 +181,23 @@ which are function pointers of struct address_space_operations. >>> Once page is successfully isolated, VM uses page.lru fields so driver >>> shouldn't expect to preserve values in those fields. >>> >>> -2. ``int (*migratepage) (struct address_space *mapping,`` >>> -| ``struct page *newpage, struct page *oldpage, enum migrate_mode);`` >>> - >>> - After isolation, VM calls migratepage() of driver with the isolated page. >>> - The function of migratepage() is to move the contents of the old page to the >>> - new page >>> - and set up fields of struct page newpage. Keep in mind that you should >>> - indicate to the VM the oldpage is no longer movable via __ClearPageMovable() >>> - under page_lock if you migrated the oldpage successfully and returned >>> - MIGRATEPAGE_SUCCESS. If driver cannot migrate the page at the moment, driver >>> - can return -EAGAIN. On -EAGAIN, VM will retry page migration in a short time >>> - because VM interprets -EAGAIN as "temporary migration failure". On returning >>> - any error except -EAGAIN, VM will give up the page migration without >>> - retrying. >>> - >>> - Driver shouldn't touch the page.lru field while in the migratepage() function. >>> +2. ``int (*migrate_folio) (struct address_space *mapping,`` >>> +| ``struct folio *dst, struct folio *src, enum migrate_mode);`` >>> + >>> + After isolation, VM calls the driver's migrate_folio() with the >>> + isolated folio. The purpose of migrate_folio() is to move the contents >>> + of the source folio to the destination folio and set up the fields >>> + of destination folio. Keep in mind that you should indicate to the >>> + VM the source folio is no longer movable via __ClearPageMovable() >>> + under folio if you migrated the source successfully and returned >>> + MIGRATEPAGE_SUCCESS. If driver cannot migrate the folio at the >>> + moment, driver can return -EAGAIN. On -EAGAIN, VM will retry folio >>> + migration in a short time because VM interprets -EAGAIN as "temporary >>> + migration failure". On returning any error except -EAGAIN, VM will >>> + give up the folio migration without retrying. >>> + >>> + Driver shouldn't touch the folio.lru field while in the migrate_folio() >>> + function. >>> >>> 3. ``void (*putback_page)(struct page *);`` >> >> Hmm, here it's a bit more complicated now, because we essentially have >> two paths: LRU+migrate_folio or !LRU+movable_ops >> (isolate/migrate/putback page) > > Oh ... actually, this is just documenting the driver side of things. > I don't really like how it's written. Here, have some rewritten > documentation (which is now part of the previous patch): > LGTM, thanks.
diff --git a/Documentation/filesystems/locking.rst b/Documentation/filesystems/locking.rst index c0fe711f14d3..3d28b23676bd 100644 --- a/Documentation/filesystems/locking.rst +++ b/Documentation/filesystems/locking.rst @@ -253,7 +253,8 @@ prototypes:: void (*free_folio)(struct folio *); int (*direct_IO)(struct kiocb *, struct iov_iter *iter); bool (*isolate_page) (struct page *, isolate_mode_t); - int (*migratepage)(struct address_space *, struct page *, struct page *); + int (*migrate_folio)(struct address_space *, struct folio *dst, + struct folio *src, enum migrate_mode); void (*putback_page) (struct page *); int (*launder_folio)(struct folio *); bool (*is_partially_uptodate)(struct folio *, size_t from, size_t count); @@ -281,7 +282,7 @@ release_folio: yes free_folio: yes direct_IO: isolate_page: yes -migratepage: yes (both) +migrate_folio: yes (both) putback_page: yes launder_folio: yes is_partially_uptodate: yes diff --git a/Documentation/filesystems/vfs.rst b/Documentation/filesystems/vfs.rst index a08c652467d7..3ae1b039b03f 100644 --- a/Documentation/filesystems/vfs.rst +++ b/Documentation/filesystems/vfs.rst @@ -740,7 +740,8 @@ cache in your filesystem. The following members are defined: /* isolate a page for migration */ bool (*isolate_page) (struct page *, isolate_mode_t); /* migrate the contents of a page to the specified target */ - int (*migratepage) (struct page *, struct page *); + int (*migrate_folio)(struct mapping *, struct folio *dst, + struct folio *src, enum migrate_mode); /* put migration-failed page back to right list */ void (*putback_page) (struct page *); int (*launder_folio) (struct folio *); @@ -935,12 +936,12 @@ cache in your filesystem. The following members are defined: is successfully isolated, VM marks the page as PG_isolated via __SetPageIsolated. -``migrate_page`` +``migrate_folio`` This is used to compact the physical memory usage. If the VM - wants to relocate a page (maybe off a memory card that is - signalling imminent failure) it will pass a new page and an old - page to this function. migrate_page should transfer any private - data across and update any references that it has to the page. + wants to relocate a folio (maybe from a memory device that is + signalling imminent failure) it will pass a new folio and an old + folio to this function. migrate_folio should transfer any private + data across and update any references that it has to the folio. ``putback_page`` Called by the VM when isolated page's migration fails. diff --git a/Documentation/vm/page_migration.rst b/Documentation/vm/page_migration.rst index 8c5cb8147e55..e0f73ddfabb1 100644 --- a/Documentation/vm/page_migration.rst +++ b/Documentation/vm/page_migration.rst @@ -181,22 +181,23 @@ which are function pointers of struct address_space_operations. Once page is successfully isolated, VM uses page.lru fields so driver shouldn't expect to preserve values in those fields. -2. ``int (*migratepage) (struct address_space *mapping,`` -| ``struct page *newpage, struct page *oldpage, enum migrate_mode);`` - - After isolation, VM calls migratepage() of driver with the isolated page. - The function of migratepage() is to move the contents of the old page to the - new page - and set up fields of struct page newpage. Keep in mind that you should - indicate to the VM the oldpage is no longer movable via __ClearPageMovable() - under page_lock if you migrated the oldpage successfully and returned - MIGRATEPAGE_SUCCESS. If driver cannot migrate the page at the moment, driver - can return -EAGAIN. On -EAGAIN, VM will retry page migration in a short time - because VM interprets -EAGAIN as "temporary migration failure". On returning - any error except -EAGAIN, VM will give up the page migration without - retrying. - - Driver shouldn't touch the page.lru field while in the migratepage() function. +2. ``int (*migrate_folio) (struct address_space *mapping,`` +| ``struct folio *dst, struct folio *src, enum migrate_mode);`` + + After isolation, VM calls the driver's migrate_folio() with the + isolated folio. The purpose of migrate_folio() is to move the contents + of the source folio to the destination folio and set up the fields + of destination folio. Keep in mind that you should indicate to the + VM the source folio is no longer movable via __ClearPageMovable() + under folio if you migrated the source successfully and returned + MIGRATEPAGE_SUCCESS. If driver cannot migrate the folio at the + moment, driver can return -EAGAIN. On -EAGAIN, VM will retry folio + migration in a short time because VM interprets -EAGAIN as "temporary + migration failure". On returning any error except -EAGAIN, VM will + give up the folio migration without retrying. + + Driver shouldn't touch the folio.lru field while in the migrate_folio() + function. 3. ``void (*putback_page)(struct page *);`` diff --git a/include/linux/fs.h b/include/linux/fs.h index 5d8ee3155ca2..47431cf8fbb3 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -362,9 +362,11 @@ struct address_space_operations { void (*free_folio)(struct folio *folio); ssize_t (*direct_IO)(struct kiocb *, struct iov_iter *iter); /* - * migrate the contents of a page to the specified target. If + * migrate the contents of a folio to the specified target. If * migrate_mode is MIGRATE_ASYNC, it must not block. */ + int (*migrate_folio)(struct address_space *, struct folio *dst, + struct folio *src, enum migrate_mode); int (*migratepage) (struct address_space *, struct page *, struct page *, enum migrate_mode); int (*launder_folio)(struct folio *); diff --git a/mm/compaction.c b/mm/compaction.c index f23efba1d118..458f49f9ab09 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -1042,7 +1042,9 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, goto isolate_fail_put; mapping = page_mapping(page); - migrate_dirty = !mapping || mapping->a_ops->migratepage; + migrate_dirty = !mapping || + mapping->a_ops->migrate_folio || + mapping->a_ops->migratepage; unlock_page(page); if (!migrate_dirty) goto isolate_fail_put; diff --git a/mm/migrate.c b/mm/migrate.c index 3ce6fee87efa..e064b998ead0 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -856,14 +856,17 @@ static int move_to_new_folio(struct folio *dst, struct folio *src, if (!mapping) rc = migrate_page(mapping, &dst->page, &src->page, mode); - else if (mapping->a_ops->migratepage) + else if (mapping->a_ops->migrate_folio) /* - * Most pages have a mapping and most filesystems - * provide a migratepage callback. Anonymous pages + * Most folios have a mapping and most filesystems + * provide a migrate_folio callback. Anonymous folios * are part of swap space which also has its own - * migratepage callback. This is the most common path + * migrate_folio callback. This is the most common path * for page migration. */ + rc = mapping->a_ops->migrate_folio(mapping, dst, src, + mode); + else if (mapping->a_ops->migratepage) rc = mapping->a_ops->migratepage(mapping, &dst->page, &src->page, mode); else