summaryrefslogtreecommitdiffstats
path: root/mm/page_alloc.c
diff options
context:
space:
mode:
authorDavid Rientjes <rientjes@google.com>2014-06-04 16:08:30 -0700
committerLinus Torvalds <torvalds@linux-foundation.org>2014-06-04 16:54:06 -0700
commit75f30861a12a6b09b759dfeeb9290b681af89057 (patch)
tree2c4fa4e5d25f32364bdfc52c548dd5c36d9fea59 /mm/page_alloc.c
parente0b9daeb453e602a95ea43853dc12d385558ce1f (diff)
mm, thp: avoid excessive compaction latency during fault
Synchronous memory compaction can be very expensive: it can iterate an enormous amount of memory without aborting, constantly rescheduling, waiting on page locks and lru_lock, etc, if a pageblock cannot be defragmented. Unfortunately, it's too expensive for transparent hugepage page faults and it's much better to simply fallback to pages. On 128GB machines, we find that synchronous memory compaction can take O(seconds) for a single thp fault. Now that async compaction remembers where it left off without strictly relying on sync compaction, this makes thp allocations best-effort without causing egregious latency during fault. We still need to retry async compaction after reclaim, but this won't stall for seconds. Signed-off-by: David Rientjes <rientjes@google.com> Acked-by: Mel Gorman <mgorman@suse.de> Cc: Greg Thelen <gthelen@google.com> Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> Cc: Vlastimil Babka <vbabka@suse.cz> Cc: Hugh Dickins <hughd@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Diffstat (limited to 'mm/page_alloc.c')
-rw-r--r--mm/page_alloc.c9
1 files changed, 8 insertions, 1 deletions
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index afb29da0576..d88d6758476 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -2575,7 +2575,14 @@ rebalance:
&did_some_progress);
if (page)
goto got_pg;
- migration_mode = MIGRATE_SYNC_LIGHT;
+
+ /*
+ * It can become very expensive to allocate transparent hugepages at
+ * fault, so use asynchronous memory compaction for THP unless it is
+ * khugepaged trying to collapse.
+ */
+ if (!(gfp_mask & __GFP_NO_KSWAPD) || (current->flags & PF_KTHREAD))
+ migration_mode = MIGRATE_SYNC_LIGHT;
/*
* If compaction is deferred for high-order allocations, it is because