diff options
Diffstat (limited to 'Documentation/vm/page_migration')
| -rw-r--r-- | Documentation/vm/page_migration | 129 |
1 files changed, 129 insertions, 0 deletions
diff --git a/Documentation/vm/page_migration b/Documentation/vm/page_migration new file mode 100644 index 00000000000..c52820fcf50 --- /dev/null +++ b/Documentation/vm/page_migration @@ -0,0 +1,129 @@ +Page migration +-------------- + +Page migration allows the moving of the physical location of pages between +nodes in a numa system while the process is running. This means that the +virtual addresses that the process sees do not change. However, the +system rearranges the physical location of those pages. + +The main intend of page migration is to reduce the latency of memory access +by moving pages near to the processor where the process accessing that memory +is running. + +Page migration allows a process to manually relocate the node on which its +pages are located through the MF_MOVE and MF_MOVE_ALL options while setting +a new memory policy. The pages of process can also be relocated +from another process using the sys_migrate_pages() function call. The +migrate_pages function call takes two sets of nodes and moves pages of a +process that are located on the from nodes to the destination nodes. + +Manual migration is very useful if for example the scheduler has relocated +a process to a processor on a distant node. A batch scheduler or an +administrator may detect the situation and move the pages of the process +nearer to the new processor. At some point in the future we may have +some mechanism in the scheduler that will automatically move the pages. + +Larger installations usually partition the system using cpusets into +sections of nodes. Paul Jackson has equipped cpusets with the ability to +move pages when a task is moved to another cpuset. This allows automatic +control over locality of a process. If a task is moved to a new cpuset +then also all its pages are moved with it so that the performance of the +process does not sink dramatically (as is the case today). + +Page migration allows the preservation of the relative location of pages +within a group of nodes for all migration techniques which will preserve a +particular memory allocation pattern generated even after migrating a +process. This is necessary in order to preserve the memory latencies. +Processes will run with similar performance after migration. + +Page migration occurs in several steps. First a high level +description for those trying to use migrate_pages() and then +a low level description of how the low level details work. + +A. Use of migrate_pages() +------------------------- + +1. Remove pages from the LRU. + + Lists of pages to be migrated are generated by scanning over + pages and moving them into lists. This is done by + calling isolate_lru_page() or __isolate_lru_page(). + Calling isolate_lru_page increases the references to the page + so that it cannot vanish under us. + +2. Generate a list of newly allocates page to move the contents + of the first list to. + +3. The migrate_pages() function is called which attempts + to do the migration. It returns the moved pages in the + list specified as the third parameter and the failed + migrations in the fourth parameter. The first parameter + will contain the pages that could still be retried. + +4. The leftover pages of various types are returned + to the LRU using putback_to_lru_pages() or otherwise + disposed of. The pages will still have the refcount as + increased by isolate_lru_pages()! + +B. Operation of migrate_pages() +-------------------------------- + +migrate_pages does several passes over its list of pages. A page is moved +if all references to a page are removable at the time. + +Steps: + +1. Lock the page to be migrated + +2. Insure that writeback is complete. + +3. Make sure that the page has assigned swap cache entry if + it is an anonyous page. The swap cache reference is necessary + to preserve the information contain in the page table maps. + +4. Prep the new page that we want to move to. It is locked + and set to not being uptodate so that all accesses to the new + page immediately lock while we are moving references. + +5. All the page table references to the page are either dropped (file backed) + or converted to swap references (anonymous pages). This should decrease the + reference count. + +6. The radix tree lock is taken + +7. The refcount of the page is examined and we back out if references remain + otherwise we know that we are the only one referencing this page. + +8. The radix tree is checked and if it does not contain the pointer to this + page then we back out. + +9. The mapping is checked. If the mapping is gone then a truncate action may + be in progress and we back out. + +10. The new page is prepped with some settings from the old page so that accesses + to the new page will be discovered to have the correct settings. + +11. The radix tree is changed to point to the new page. + +12. The reference count of the old page is dropped because the reference has now + been removed. + +13. The radix tree lock is dropped. + +14. The page contents are copied to the new page. + +15. The remaining page flags are copied to the new page. + +16. The old page flags are cleared to indicate that the page does + not use any information anymore. + +17. Queued up writeback on the new page is triggered. + +18. If swap pte's were generated for the page then remove them again. + +19. The locks are dropped from the old and new page. + +20. The new page is moved to the LRU. + +Christoph Lameter, December 19, 2005. + |