summaryrefslogtreecommitdiffstats
path: root/arch/x86
AgeCommit message (Collapse)Author
2012-02-21x32: Fix coding style violations in the x32 VDSO codeH. Peter Anvin
Move the prototype for x32_setup_additional_pages() to a header file, and adjust the coding style to match standard. Signed-off-by: H. Peter Anvin <hpa@linux.intel.com> Cc: H. J. Lu <hjl.tools@gmail.com>
2012-02-21i387: Split up <asm/i387.h> into exported and internal interfacesLinus Torvalds
While various modules include <asm/i387.h> to get access to things we actually *intend* for them to use, most of that header file was really pretty low-level internal stuff that we really don't want to expose to others. So split the header file into two: the small exported interfaces remain in <asm/i387.h>, while the internal definitions that are only used by core architecture code are now in <asm/fpu-internal.h>. The guiding principle for this was to expose functions that we export to modules, and leave them in <asm/i387.h>, while stuff that is used by task switching or was marked GPL-only is in <asm/fpu-internal.h>. The fpu-internal.h file could be further split up too, especially since arch/x86/kvm/ uses some of the remaining stuff for its module. But that kvm usage should probably be abstracted out a bit, and at least now the internal FPU accessor functions are much more contained. Even if it isn't perhaps as contained as it _could_ be. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Link: http://lkml.kernel.org/r/alpine.LFD.2.02.1202211340330.5354@i5.linux-foundation.org Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
2012-02-21i387: Uninline the generic FP helpers that we expose to kernel modulesLinus Torvalds
Instead of exporting the very low-level internals of the FPU state save/restore code (ie things like 'fpu_owner_task'), we should export the higher-level interfaces. Inlining these things is pointless anyway: sure, sometimes the end result is small, but while 'stts()' can result in just three x86 instructions, those are not cheap instructions (writing %cr0 is a serializing instruction and a very slow one at that). So the overhead of a function call is not noticeable, and we really don't want random modules mucking about with our internal state save logic anyway. So this unexports 'fpu_owner_task', and instead uninlines and exports the actual functions that modules can use: fpu_kernel_begin/end() and unlazy_fpu(). Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Link: http://lkml.kernel.org/r/alpine.LFD.2.02.1202211339590.5354@i5.linux-foundation.org Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
2012-02-20i387: export 'fpu_owner_task' per-cpu variableLinus Torvalds
(And define it properly for x86-32, which had its 'current_task' declaration in separate from x86-64) Bitten by my dislike for modules on the machines I use, and the fact that apparently nobody else actually wanted to test the patches I sent out. Snif. Nobody else cares. Anyway, we probably should uninline the 'kernel_fpu_begin()' function that is what modules actually use and that references this, but this is the minimal fix for now. Reported-by: Josh Boyer <jwboyer@gmail.com> Reported-and-tested-by: Jongman Heo <jongman.heo@samsung.com> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-02-20x86: Specify a size for the cmp in the NMI handlerSteven Rostedt
Linus noticed that the cmp used to check if the code segment is __KERNEL_CS or not did not specify a size. Perhaps it does not matter as H. Peter Anvin noted that user space can not set the bottom two bits of the %cs register. But it's best not to let the assembly choose and change things between different versions of gas, but instead just pick the size. Four bytes are used to compare the saved code segment against __KERNEL_CS. Perhaps this might mess up Xen, but we can fix that when the time comes. Also I noticed that there was another non-specified cmp that checks the special stack variable if it is 1 or 0. This too probably doesn't matter what cmp is used, but this patch uses cmpl just to make it non ambiguous. Link: http://lkml.kernel.org/r/CA+55aFxfAn9MWRgS3O5k2tqN5ys1XrhSFVO5_9ZAoZKDVgNfGA@mail.gmail.com Suggested-by: Linus Torvalds <torvalds@linux-foundation.org> Cc: H. Peter Anvin <hpa@zytor.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-02-20x32: Add x32 VDSO supportH. J. Lu
Add support for the x32 VDSO. The x32 VDSO takes advantage of the similarity between the x86-64 and the x32 ABIs to contain the same content, only the container is different, as the x32 VDSO obviously is an x32 shared object. Signed-off-by: H. Peter Anvin <hpa@zytor.com>
2012-02-20x32: Allow x32 to be configuredH. J. Lu
At this point, one should be able to build an x32 kernel. Note that for now we depend on CONFIG_IA32_EMULATION. Long term, x32 and IA32 should be detangled. Signed-off-by: H. Peter Anvin <hpa@zytor.com>
2012-02-20x32: If configured, add x32 system calls to system call tablesH. Peter Anvin
If CONFIG_X86_X32_ABI is defined, add the x32 system calls to the system call tables. Signed-off-by: H. Peter Anvin <hpa@zytor.com>
2012-02-20x32: Handle process creationH. Peter Anvin
Allow an x32 process to be started. Originally-by: H. J. Lu <hjl.tools@gmail.com> Signed-off-by: H. Peter Anvin <hpa@zytor.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
2012-02-20x32: Signal-related system callsH. Peter Anvin
x32 uses the 64-bit signal frame format, obviously, but there are some structures which mixes that with pointers or sizeof(long) types, as such we have to create a handful of system calls specific to x32. By and large these are a mixture of the 64-bit and the compat system calls. Originally-by: H. J. Lu <hjl.tools@gmail.com> Signed-off-by: H. Peter Anvin <hpa@zytor.com>
2012-02-20x86: Add #ifdef CONFIG_COMPAT to <asm/sys_ia32.h>H. Peter Anvin
Unfortunately a lot of the compat types are guarded with CONFIG_COMPAT or the equivalent, so add a similar guard to <asm/sys_ia32.h> to avoid compilation failures when CONFIG_COMPAT=n. Signed-off-by: H. Peter Anvin <hpa@zytor.com>
2012-02-20x32: Handle the x32 system call flagH. Peter Anvin
x32 shares most system calls with x86-64, but unfortunately some subsystem (the input subsystem is the chief offender) which require is_compat() when operating with a 32-bit userspace. The input system actually has text files in sysfs whose meaning is dependent on sizeof(long) in userspace! We could solve this by having two completely disjoint system call tables; requiring that each system call be duplicated. This patch takes a different approach: we add a flag to the system call number; this flag doesn't affect the system call dispatch but requests compat treatment from affected subsystems for the duration of the system call. The change of cmpq to cmpl is safe since it immediately follows the and. Signed-off-by: H. Peter Anvin <hpa@zytor.com>
2012-02-20x32: Add rt_sigframe_x32H. Peter Anvin
Add rt_sigframe_x32 to <asm/sigframe.h>. Unfortunately we can't just define all the data structures unconditionally, due to the #ifdef CONFIG_COMPAT in <linux/compat.h> and its trickle-down effects, hence the #ifdef mess. Signed-off-by: H. Peter Anvin <hpa@zytor.com>
2012-02-20x32: Add struct ucontext_x32H. J. Lu
Add a definition for struct ucontext_x32; this is inherently a mix of the 32- and 64-bit versions. Signed-off-by: H. Peter Anvin <hpa@zytor.com>
2012-02-20x32: Export setup/restore_sigcontext from signal.cH. Peter Anvin
Export setup_sigcontext() and restore_sigcontext() from signal.c, so we can use the 64-bit versions verbatim for x32. Signed-off-by: H. Peter Anvin <hpa@zytor.com>
2012-02-20x86: Move some signal-handling definitions to a common headerH. Peter Anvin
There are some definitions which are duplicated between kernel/signal.c and ia32/ia32_signal.c; move them to a common header file. Rather than adding stuff to existing header files which contain data structures, create a new header file; hence the slightly odd name ("all the good ones were taken.") Note: nothing relied on signal_fault() being defined in <asm/ptrace.h>. Signed-off-by: H. Peter Anvin <hpa@zytor.com>
2012-02-20x32: Generate <asm/unistd_64_x32.h>H. Peter Anvin
Generate macros for the *kernel* code to use to refer to x32 system calls. These have an __NR_x32_ prefix and do not include __X32_SYSCALL_BIT. Signed-off-by: H. Peter Anvin <hpa@zytor.com>
2012-02-20x32: Generate <asm/unistd_x32.h>H. Peter Anvin
Generate <asm/unistd_x32.h>; this exports x32 system call numbers to user space. [ v2: Enclose all arguments to syshdr in '' so empty arguments aren't dropped on the floor. ] Signed-off-by: H. Peter Anvin <hpa@zytor.com>
2012-02-20x32: Add x32 system calls to syscall/syscall_64.tblH. Peter Anvin
Split the 64-bit system calls into "64" (64-bit only) and "common" (64-bit or x32) and add the x32 system call numbers. Signed-off-by: H. Peter Anvin <hpa@zytor.com>
2012-02-20x86-64, ia32: Drop sys32_rt_sigprocmaskH. Peter Anvin
On x86, the only difference between sys_rt_sigprocmask and sys32_rt_sigprocmask is the alignment of the data structures. However, x86 allows data accesses with arbitrary alignment, and therefore there is no reason for this code to be different. Reported-by: Gregory M. Lueck <gregory.m.lueck@intel.com> Signed-off-by: H. Peter Anvin <hpa@zytor.com>
2012-02-20x32: Add a thread flag for x32 processesH. Peter Anvin
An x32 process is *almost* the same thing as a 64-bit process with a 32-bit address limit, but there are a few minor differences -- in particular core dumps are 32 bits and signal handling is different. Signed-off-by: H. Peter Anvin <hpa@zytor.com>
2012-02-20x86-64: Add prototype for old_rsp to a header fileH. J. Lu
So far this has only been used in process_64.c, but the x32 code will need it in additional code. Signed-off-by: H. Peter Anvin <hpa@zytor.com>
2012-02-20x32: Create posix_types_x32.hH. Peter Anvin
This is the same as the 64-bit posix_types.h, except that __kernel_[u]long_t is defined to be [unsigned] long long and therefore 64 bits. Signed-off-by: H. Peter Anvin <hpa@zytor.com>
2012-02-20x86-64: Use explicit sizes in sigcontext.h, prepare for x32H. Peter Anvin
Use explicit sizes (__u64) instead of implicit sizes (unsigned long) in the definition for sigcontext.h; this will allow this structure to be shared between the x86-64 native ABI and the x32 ABI. Originally-by: H. J. Lu <hjl.tools@gmail.com> Signed-off-by: H. Peter Anvin <hpa@zytor.com> Link: http://lkml.kernel.org/n/tip-4pr1xnnksprt7t0h3w5fw4rv@git.kernel.org
2012-02-20x86: Factor out TIF_IA32 from 32-bit address spaceH. Peter Anvin
Factor out IA32 (compatibility instruction set) from 32-bit address space in the thread_info flags; this is a precondition patch for x32 support. Originally-by: H. J. Lu <hjl.tools@gmail.com> Signed-off-by: H. Peter Anvin <hpa@zytor.com> Link: http://lkml.kernel.org/n/tip-4pr1xnnksprt7t0h3w5fw4rv@git.kernel.org
2012-02-20i387: support lazy restore of FPU stateLinus Torvalds
This makes us recognize when we try to restore FPU state that matches what we already have in the FPU on this CPU, and avoids the restore entirely if so. To do this, we add two new data fields: - a percpu 'fpu_owner_task' variable that gets written any time we update the "has_fpu" field, and thus acts as a kind of back-pointer to the task that owns the CPU. The exception is when we save the FPU state as part of a context switch - if the save can keep the FPU state around, we leave the 'fpu_owner_task' variable pointing at the task whose FP state still remains on the CPU. - a per-thread 'last_cpu' field, that indicates which CPU that thread used its FPU on last. We update this on every context switch (writing an invalid CPU number if the last context switch didn't leave the FPU in a lazily usable state), so we know that *that* thread has done nothing else with the FPU since. These two fields together can be used when next switching back to the task to see if the CPU still matches: if 'fpu_owner_task' matches the task we are switching to, we know that no other task (or kernel FPU usage) touched the FPU on this CPU in the meantime, and if the current CPU number matches the 'last_cpu' field, we know that this thread did no other FP work on any other CPU, so the FPU state on the CPU must match what was saved on last context switch. In that case, we can avoid the 'f[x]rstor' entirely, and just clear the CR0.TS bit. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-02-20i387: use 'restore_fpu_checking()' directly in task switching codeLinus Torvalds
This inlines what is usually just a couple of instructions, but more importantly it also fixes the theoretical error case (can that FPU restore really ever fail? Maybe we should remove the checking). We can't start sending signals from within the scheduler, we're much too deep in the kernel and are holding the runqueue lock etc. So don't bother even trying. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-02-20i387: fix up some fpu_counter confusionLinus Torvalds
This makes sure we clear the FPU usage counter for newly created tasks, just so that we start off in a known state (for example, don't try to preload the FPU state on the first task switch etc). It also fixes a thinko in when we increment the fpu_counter at task switch time, introduced by commit 34ddc81a230b ("i387: re-introduce FPU state preloading at context switch time"). We should increment the *new* task fpu_counter, not the old task, and only if we decide to use that state (whether lazily or preloaded). Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-02-20xen/pat: Disable PAT support for now.Konrad Rzeszutek Wilk
[Pls also look at https://lkml.org/lkml/2012/2/10/228] Using of PAT to change pages from WB to WC works quite nicely. Changing it back to WB - not so much. The crux of the matter is that the code that does this (__page_change_att_set_clr) has only limited information so when it tries to the change it gets the "raw" unfiltered information instead of the properly filtered one - and the "raw" one tell it that PSE bit is on (while infact it is not). As a result when the PTE is set to be WB from WC, we get tons of: :WARNING: at arch/x86/xen/mmu.c:475 xen_make_pte+0x67/0xa0() :Hardware name: HP xw4400 Workstation .. snip.. :Pid: 27, comm: kswapd0 Tainted: G W 3.2.2-1.fc16.x86_64 #1 :Call Trace: : [<ffffffff8106dd1f>] warn_slowpath_common+0x7f/0xc0 : [<ffffffff8106dd7a>] warn_slowpath_null+0x1a/0x20 : [<ffffffff81005a17>] xen_make_pte+0x67/0xa0 : [<ffffffff810051bd>] __raw_callee_save_xen_make_pte+0x11/0x1e : [<ffffffff81040e15>] ? __change_page_attr_set_clr+0x9d5/0xc00 : [<ffffffff8114c2e8>] ? __purge_vmap_area_lazy+0x158/0x1d0 : [<ffffffff8114cca5>] ? vm_unmap_aliases+0x175/0x190 : [<ffffffff81041168>] change_page_attr_set_clr+0x128/0x4c0 : [<ffffffff81041542>] set_pages_array_wb+0x42/0xa0 : [<ffffffff8100a9b2>] ? check_events+0x12/0x20 : [<ffffffffa0074d4c>] ttm_pages_put+0x1c/0x70 [ttm] : [<ffffffffa0074e98>] ttm_page_pool_free+0xf8/0x180 [ttm] : [<ffffffffa0074f78>] ttm_pool_mm_shrink+0x58/0x90 [ttm] : [<ffffffff8112ba04>] shrink_slab+0x154/0x310 : [<ffffffff8112f17a>] balance_pgdat+0x4fa/0x6c0 : [<ffffffff8112f4b8>] kswapd+0x178/0x3d0 : [<ffffffff815df134>] ? __schedule+0x3d4/0x8c0 : [<ffffffff81090410>] ? remove_wait_queue+0x50/0x50 : [<ffffffff8112f340>] ? balance_pgdat+0x6c0/0x6c0 : [<ffffffff8108fb6c>] kthread+0x8c/0xa0 for every page. The proper fix for this is has been posted and is https://lkml.org/lkml/2012/2/10/228 "x86/cpa: Use pte_attrs instead of pte_flags on CPA/set_p.._wb/wc operations." along with a detailed description of the problem and solution. But since that posting has gone nowhere I am proposing this band-aid solution so that at least users don't get the page corruption (the pages that are WC don't get changed to WB and end up being recycled for filesystem or other things causing mysterious crashes). The negative impact of this patch is that users of WC flag (which are InfiniBand, radeon, nouveau drivers) won't be able to set that flag - so they are going to see performance degradation. But stability is more important here. Fixes RH BZ# 742032, 787403, and 745574 Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
2012-02-20xen/setup: Remove redundant filtering of PTE masks.Konrad Rzeszutek Wilk
commit 7347b4082e55ac4a673f06a0a0ce25c37273c9ec "xen: Allow unprivileged Xen domains to create iomap pages" added a redundant line in the early bootup code to filter out the PTE. That filtering is already done a bit earlier so this extra processing is not required. Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
2012-02-20x86-32/irq: Don't switch to irq stack for a user-mode irqLinus Torvalds
If the irq happens in user mode, our kernel stack is empty (apart from the pt_regs themselves, of course), so there's no need or advantage to switch. And it really doesn't save any stack space, quite the reverse: it means that a nested interrupt cannot switch irq stacks. So instead of saving kernel stack space, it actually causes the potential for *more* stack usage. Also simplify the preemption count copy when we do switch stacks: just copy the whole preemption count, rather than just the softirq parts of it. There is no advantage to the partial copy: it is more effort to get a less correct result. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Link: http://lkml.kernel.org/r/alpine.LFD.2.02.1202191139260.10000@i5.linux-foundation.org Signed-off-by: Ingo Molnar <mingo@elte.hu>
2012-02-20x86/nmi: Test saved %cs in NMI to determine nested NMI caseSteven Rostedt
Currently, the NMI handler tests if it is nested by checking the special variable saved on the stack (set during NMI handling) and whether the saved stack is the NMI stack as well (to prevent the race when the variable is set to zero). But userspace may set their %rsp to any value as long as they do not derefence it, and it may make it point to the NMI stack, which will prevent NMIs from triggering while the userspace app is running. (I tested this, and it is indeed the case) Add another check to determine nested NMIs by looking at the saved %cs (code segment register) and making sure that it is the kernel code segment. Signed-off-by: Steven Rostedt <rostedt@goodmis.org> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: <stable@kernel.org> Link: http://lkml.kernel.org/r/1329687817.1561.27.camel@acer.local.home Signed-off-by: Ingo Molnar <mingo@elte.hu>
2012-02-20x86/UV: Lower UV rtc clocksource ratingDimitri Sivanich
Lower the rating of the UV rtc clocksource to just below that of the tsc, to improve performance. Reading the tsc clocksource has lower latency than reading the rtc, so favor it in situations where it is synchronized and stable. When the tsc is unsynchronized, the rtc needs to be the chosen clocksource. Signed-off-by: Dimitri Sivanich <sivanich@sgi.com> Cc: John Stultz <johnstul@us.ibm.com> Cc: Jack Steiner <steiner@sgi.com> Link: http://lkml.kernel.org/r/20120217141641.GA28063@sgi.com Signed-off-by: Ingo Molnar <mingo@elte.hu>
2012-02-18i387: re-introduce FPU state preloading at context switch timeLinus Torvalds
After all the FPU state cleanups and finally finding the problem that caused all our FPU save/restore problems, this re-introduces the preloading of FPU state that was removed in commit b3b0870ef3ff ("i387: do not preload FPU state at task switch time"). However, instead of simply reverting the removal, this reimplements preloading with several fixes, most notably - properly abstracted as a true FPU state switch, rather than as open-coded save and restore with various hacks. In particular, implementing it as a proper FPU state switch allows us to optimize the CR0.TS flag accesses: there is no reason to set the TS bit only to then almost immediately clear it again. CR0 accesses are quite slow and expensive, don't flip the bit back and forth for no good reason. - Make sure that the same model works for both x86-32 and x86-64, so that there are no gratuitous differences between the two due to the way they save and restore segment state differently due to architectural differences that really don't matter to the FPU state. - Avoid exposing the "preload" state to the context switch routines, and in particular allow the concept of lazy state restore: if nothing else has used the FPU in the meantime, and the process is still on the same CPU, we can avoid restoring state from memory entirely, just re-expose the state that is still in the FPU unit. That optimized lazy restore isn't actually implemented here, but the infrastructure is set up for it. Of course, older CPU's that use 'fnsave' to save the state cannot take advantage of this, since the state saving also trashes the state. In other words, there is now an actual _design_ to the FPU state saving, rather than just random historical baggage. Hopefully it's easier to follow as a result. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-02-18i387: move TS_USEDFPU flag from thread_info to task_structLinus Torvalds
This moves the bit that indicates whether a thread has ownership of the FPU from the TS_USEDFPU bit in thread_info->status to a word of its own (called 'has_fpu') in task_struct->thread.has_fpu. This fixes two independent bugs at the same time: - changing 'thread_info->status' from the scheduler causes nasty problems for the other users of that variable, since it is defined to be thread-synchronous (that's what the "TS_" part of the naming was supposed to indicate). So perfectly valid code could (and did) do ti->status |= TS_RESTORE_SIGMASK; and the compiler was free to do that as separate load, or and store instructions. Which can cause problems with preemption, since a task switch could happen in between, and change the TS_USEDFPU bit. The change to TS_USEDFPU would be overwritten by the final store. In practice, this seldom happened, though, because the 'status' field was seldom used more than once, so gcc would generally tend to generate code that used a read-modify-write instruction and thus happened to avoid this problem - RMW instructions are naturally low fat and preemption-safe. - On x86-32, the current_thread_info() pointer would, during interrupts and softirqs, point to a *copy* of the real thread_info, because x86-32 uses %esp to calculate the thread_info address, and thus the separate irq (and softirq) stacks would cause these kinds of odd thread_info copy aliases. This is normally not a problem, since interrupts aren't supposed to look at thread information anyway (what thread is running at interrupt time really isn't very well-defined), but it confused the heck out of irq_fpu_usable() and the code that tried to squirrel away the FPU state. (It also caused untold confusion for us poor kernel developers). It also turns out that using 'task_struct' is actually much more natural for most of the call sites that care about the FPU state, since they tend to work with the task struct for other reasons anyway (ie scheduling). And the FPU data that we are going to save/restore is found there too. Thanks to Arjan Van De Ven <arjan@linux.intel.com> for pointing us to the %esp issue. Cc: Arjan van de Ven <arjan@linux.intel.com> Reported-and-tested-by: Raphael Prevost <raphael@buro.asia> Acked-and-tested-by: Suresh Siddha <suresh.b.siddha@intel.com> Tested-by: Peter Anvin <hpa@zytor.com> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-02-17Merge branch 'tip/perf/core' of ↵Ingo Molnar
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace into perf/core
2012-02-16i387: move AMD K7/K8 fpu fxsave/fxrstor workaround from save to restoreLinus Torvalds
The AMD K7/K8 CPUs don't save/restore FDP/FIP/FOP unless an exception is pending. In order to not leak FIP state from one process to another, we need to do a floating point load after the fxsave of the old process, and before the fxrstor of the new FPU state. That resets the state to the (uninteresting) kernel load, rather than some potentially sensitive user information. We used to do this directly after the FPU state save, but that is actually very inconvenient, since it (a) corrupts what is potentially perfectly good FPU state that we might want to lazy avoid restoring later and (b) on x86-64 it resulted in a very annoying ordering constraint, where "__unlazy_fpu()" in the task switch needs to be delayed until after the DS segment has been reloaded just to get the new DS value. Coupling it to the fxrstor instead of the fxsave automatically avoids both of these issues, and also ensures that we only do it when actually necessary (the FP state after a save may never actually get used). It's simply a much more natural place for the leaked state cleanup. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-02-16i387: do not preload FPU state at task switch timeLinus Torvalds
Yes, taking the trap to re-load the FPU/MMX state is expensive, but so is spending several days looking for a bug in the state save/restore code. And the preload code has some rather subtle interactions with both paravirtualization support and segment state restore, so it's not nearly as simple as it should be. Also, now that we no longer necessarily depend on a single bit (ie TS_USEDFPU) for keeping track of the state of the FPU, we migth be able to do better. If we are really switching between two processes that keep touching the FP state, save/restore is inevitable, but in the case of having one process that does most of the FPU usage, we may actually be able to do much better than the preloading. In particular, we may be able to keep track of which CPU the process ran on last, and also per CPU keep track of which process' FP state that CPU has. For modern CPU's that don't destroy the FPU contents on save time, that would allow us to do a lazy restore by just re-enabling the existing FPU state - with no restore cost at all! Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-02-16i387: don't ever touch TS_USEDFPU directly, use helper functionsLinus Torvalds
This creates three helper functions that do the TS_USEDFPU accesses, and makes everybody that used to do it by hand use those helpers instead. In addition, there's a couple of helper functions for the "change both CR0.TS and TS_USEDFPU at the same time" case, and the places that do that together have been changed to use those. That means that we have fewer random places that open-code this situation. The intent is partly to clarify the code without actually changing any semantics yet (since we clearly still have some hard to reproduce bug in this area), but also to make it much easier to use another approach entirely to caching the CR0.TS bit for software accesses. Right now we use a bit in the thread-info 'status' variable (this patch does not change that), but we might want to make it a full field of its own or even make it a per-cpu variable. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-02-16i387: move TS_USEDFPU clearing out of __save_init_fpu and into callersLinus Torvalds
Touching TS_USEDFPU without touching CR0.TS is confusing, so don't do it. By moving it into the callers, we always do the TS_USEDFPU next to the CR0.TS accesses in the source code, and it's much easier to see how the two go hand in hand. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-02-16i387: fix x86-64 preemption-unsafe user stack save/restoreLinus Torvalds
Commit 5b1cbac37798 ("i387: make irq_fpu_usable() tests more robust") added a sanity check to the #NM handler to verify that we never cause the "Device Not Available" exception in kernel mode. However, that check actually pinpointed a (fundamental) race where we do cause that exception as part of the signal stack FPU state save/restore code. Because we use the floating point instructions themselves to save and restore state directly from user mode, we cannot do that atomically with testing the TS_USEDFPU bit: the user mode access itself may cause a page fault, which causes a task switch, which saves and restores the FP/MMX state from the kernel buffers. This kind of "recursive" FP state save is fine per se, but it means that when the signal stack save/restore gets restarted, it will now take the '#NM' exception we originally tried to avoid. With preemption this can happen even without the page fault - but because of the user access, we cannot just disable preemption around the save/restore instruction. There are various ways to solve this, including using the "enable/disable_page_fault()" helpers to not allow page faults at all during the sequence, and fall back to copying things by hand without the use of the native FP state save/restore instructions. However, the simplest thing to do is to just allow the #NM from kernel space, but fix the race in setting and clearing CR0.TS that this all exposed: the TS bit changes and the TS_USEDFPU bit absolutely have to be atomic wrt scheduling, so while the actual state save/restore can be interrupted and restarted, the act of actually clearing/setting CR0.TS and the TS_USEDFPU bit together must not. Instead of just adding random "preempt_disable/enable()" calls to what is already excessively ugly code, this introduces some helper functions that mostly mirror the "kernel_fpu_begin/end()" functionality, just for the user state instead. Those helper functions should probably eventually replace the other ad-hoc CR0.TS and TS_USEDFPU tests too, but I'll need to think about it some more: the task switching functionality in particular needs to expose the difference between the 'prev' and 'next' threads, while the new helper functions intentionally were written to only work with 'current'. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-02-15i387: fix sense of sanity checkLinus Torvalds
The check for save_init_fpu() (introduced in commit 5b1cbac37798: "i387: make irq_fpu_usable() tests more robust") was the wrong way around, but I hadn't noticed, because my "tests" were bogus: the FPU exceptions are disabled by default, so even doing a divide by zero never actually triggers this code at all unless you do extra work to enable them. So if anybody did enable them, they'd get one spurious warning. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-02-14Merge tag 'stable/for-linus-fixes-3.3-rc3' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen Two fixes for VCPU offlining; One to fix the string format exposed by the xen-pci[front|back] to conform to the one used in majority of PCI drivers; Two fixes to make the code more resilient to invalid configurations. Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> * tag 'stable/for-linus-fixes-3.3-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen: xenbus_dev: add missing error check to watch handling xen/pci[front|back]: Use %d instead of %1x for displaying PCI devfn. xen pvhvm: do not remap pirqs onto evtchns if !xen_have_vector_callback xen/smp: Fix CPU online/offline bug triggering a BUG: scheduling while atomic. xen/bootup: During bootup suppress XENBUS: Unable to read cpu state
2012-02-14x86: Use generic posix_types.hH. Peter Anvin
Change the x86 architecture to use <asm-generic/posix_types.h>. Signed-off-by: H. Peter Anvin <hpa@zytor.com> Link: http://lkml.kernel.org/r/1328677745-20121-20-git-send-email-hpa@zytor.com Cc: Ingo Molnar <mingo@elte.hu> Cc: Thomas Gleixner <tglx@linutronix.de>
2012-02-14x86/PCI: don't fall back to defaults if _CRS has no aperturesBjorn Helgaas
Host bridges that lead to things like the Uncore need not have any I/O port or MMIO apertures. For example, in this case: ACPI: PCI Root Bridge [UNC1] (domain 0000 [bus ff]) PCI: root bus ff: using default resources PCI host bridge to bus 0000:ff pci_bus 0000:ff: root bus resource [io 0x0000-0xffff] pci_bus 0000:ff: root bus resource [mem 0x00000000-0x3fffffffffff] we should not pretend those default resources are available on bus ff. CC: Yinghai Lu <yinghai@kernel.org> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2012-02-14x86/PCI: Convert maintaining FW-assigned BIOS BAR values to use a listMyron Stowe
This patch converts the underlying maintenance aspects of FW-assigned BIOS BAR values from a statically allocated array within struct pci_dev to a list of temporary, stand alone, entries. Signed-off-by: Myron Stowe <myron.stowe@redhat.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2012-02-14x86/PCI: Infrastructure to maintain a list of FW-assigned BIOS BAR valuesMyron Stowe
Commit 58c84eda075 introduced functionality to try and reinstate the original BIOS BAR addresses of a PCI device when normal resource assignment attempts fail. To keep track of the BIOS BAR addresses, struct pci_dev was augmented with an array to hold the BAR addresses of the PCI device: 'resource_size_t fw_addr[DEVICE_COUNT_RESOURCE]'. The reinstatement of BAR addresses is an uncommon event leaving the 'fw_addr' array unused under normal circumstances. This functionality is also currently architecture specific with an implementation limited to x86. As the use of struct pci_dev is so prevalent, having the 'fw_addr' array residing within such seems somewhat wasteful. This patch introduces a stand alone data structure and interfacing routines for maintaining a list of FW-assigned BIOS BAR value entries. Signed-off-by: Myron Stowe <myron.stowe@redhat.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2012-02-14crypto: serpent-sse2 - remove dead code from ↵Jesper Juhl
serpent_sse2_glue.c::serpent_sse2_init() We cannot reach the line after 'return err'. Remove it. Signed-off-by: Jesper Juhl <jj@chaosbits.net> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2012-02-14crypto: twofish-x86 - Remove dead code from twofish_glue_3way.c::init()Jesper Juhl
We can never reach the line just after the 'return 0' statement. Remove it. Signed-off-by: Jesper Juhl <jj@chaosbits.net> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2012-02-13x86/cpu: Clean up modalias feature matchingBen Hutchings
We currently include commas on both sides of the feature ID in a modalias, but this prevents the lowest numbered feature of a CPU from being matched. Since all feature IDs have the same length, we do not need to worry about substring matches, so omit commas from the modalias entirely. Avoid generating multiple adjacent wildcards when there is no feature ID to match. Signed-off-by: Ben Hutchings <ben@decadent.org.uk> Acked-by: Thomas Renninger <trenn@suse.de> Acked-by: H. Peter Anvin <hpa@zytor.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>