linux/kernel/sched/clock.c, branch v4.5

Merge branch 'for-4.5' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq

2016-01-12T02:53:13Z

Pull workqueue update from Tejun Heo: "Workqueue changes for v4.5. One cleanup patch and three to improve the debuggability. Workqueue now has a stall detector which dumps workqueue state if any worker pool hasn't made forward progress over a certain amount of time (30s by default) and also triggers a warning if a workqueue which can be used in memory reclaim path tries to wait on something which can't be. These should make workqueue hangs a lot easier to debug." * 'for-4.5' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq: workqueue: simplify the apply_workqueue_attrs_locked() workqueue: implement lockup detector watchdog: introduce touch_softlockup_watchdog_sched() workqueue: warn if memory reclaim tries to flush !WQ_MEM_RECLAIM workqueue

watchdog: introduce touch_softlockup_watchdog_sched()

2015-12-08T16:29:42Z

touch_softlockup_watchdog() is used to tell watchdog that scheduler stall is expected. One group of usage is from paths where the task may not be able to yield for a long time such as performing slow PIO to finicky device and coming out of suspend. The other is to account for scheduler and timer going idle. For scheduler softlockup detection, there's no reason to distinguish the two cases; however, workqueue lockup detector is planned and it can use the same signals from the former group while the latter would spuriously prevent detection. This patch introduces a new function touch_softlockup_watchdog_sched() and convert the latter group to call it instead. For now, it just calls touch_softlockup_watchdog() and there's no functional difference. Signed-off-by: Tejun Heo Cc: Ulrich Obergfell Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Andrew Morton

treewide: Remove old email address

2015-11-23T08:44:58Z

There were still a number of references to my old Red Hat email address in the kernel source. Remove these while keeping the Red Hat copyright notices intact. Signed-off-by: Peter Zijlstra (Intel) Cc: Arnaldo Carvalho de Melo Cc: Jiri Olsa Cc: Linus Torvalds Cc: Mike Galbraith Cc: Peter Zijlstra Cc: Stephane Eranian Cc: Thomas Gleixner Cc: Vince Weaver Signed-off-by: Ingo Molnar

kernel/sched/clock.c: add another clock for use with the soft lockup watchdog

2015-02-13T02:54:13Z

When the hypervisor pauses a virtualised kernel the kernel will observe a jump in timebase, this can cause spurious messages from the softlockup detector. Whilst these messages are harmless, they are accompanied with a stack trace which causes undue concern and more problematically the stack trace in the guest has nothing to do with the observed problem and can only be misleading. Futhermore, on POWER8 this is completely avoidable with the introduction of the Virtual Time Base (VTB) register. This patch (of 2): This permits the use of arch specific clocks for which virtualised kernels can use their notion of 'running' time, not the elpased wall time which will include host execution time. Signed-off-by: Cyril Bur Cc: Michael Ellerman Cc: Andrew Jones Acked-by: Don Zickus Cc: Ingo Molnar Cc: Ulrich Obergfell Cc: chai wen Cc: Fabian Frederick Cc: Aaron Tomlin Cc: Ben Zhang Cc: Martin Schwidefsky Cc: John Stultz Cc: Thomas Gleixner Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

time: Replace __get_cpu_var uses

2014-08-26T17:45:44Z

Convert uses of __get_cpu_var for creating a address from a percpu offset to this_cpu_ptr. The two cases where get_cpu_var is used to actually access a percpu variable are changed to use this_cpu_read/raw_cpu_read. Reviewed-by: Thomas Gleixner Signed-off-by: Christoph Lameter Signed-off-by: Tejun Heo

kernel: use macros from compiler.h instead of attribute((...))

2014-04-07T23:36:11Z

To increase compiler portability there is which provides convenience macros for various gcc constructs. Eg: __weak for __attribute__((weak)). I've replaced all instances of gcc attributes with the right macro in the kernel subsystem. Signed-off-by: Gideon Israel Dsouza Cc: "Rafael J. Wysocki" Cc: Ingo Molnar Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

sched/clock: Prevent tracing recursion in sched_clock_cpu()

2014-03-11T10:33:48Z

Prevent tracing of preempt_disable/enable() in sched_clock_cpu(). When CONFIG_DEBUG_PREEMPT is enabled, preempt_disable/enable() are traced and this causes trace_clock() users (and probably others) to go into an infinite recursion. Systems with a stable sched_clock() are not affected. This problem is similar to that fixed by upstream commit 95ef1e52922 ("KVM guest: prevent tracing recursion with kvmclock"). Signed-off-by: Fernando Luis Vazquez Cao Signed-off-by: Peter Zijlstra Acked-by: Steven Rostedt Cc: Andrew Morton Cc: Linus Torvalds Link: http://lkml.kernel.org/r/1394083528.4524.3.camel@nexus Signed-off-by: Ingo Molnar

sched/clock: Fixup early initialization

2014-01-23T13:48:36Z

The code would assume sched_clock_stable() and switch to !stable later, this switch brings a discontinuity in time. The discontinuity on switching from stable to unstable was always present, but previously we would set stable/unstable before initializing TSC and usually stick to the one we start out with. So the static_key bits brought an extra switch where there previously wasn't one. Things are further complicated by the fact that we cannot use static_key as early as we usually call set_sched_clock_stable(). Fix things by tracking the stable state in a regular variable and only set the static_key to the right state on sched_clock_init(), which is ran right after late_time_init->tsc_init(). Before this we would not be using the TSC anyway. Reported-and-Tested-by: Sasha Levin Reported-by: dyoung@redhat.com Fixes: 35af99e646c7 ("sched/clock, x86: Use a static_key for sched_clock_stable") Cc: jacob.jun.pan@linux.intel.com Cc: Mike Galbraith Cc: hpa@zytor.com Cc: paulmck@linux.vnet.ibm.com Cc: John Stultz Cc: Andy Lutomirski Cc: Arjan van de Ven Cc: lenb@kernel.org Cc: rjw@rjwysocki.net Cc: Eliezer Tamir Cc: rui.zhang@intel.com Signed-off-by: Peter Zijlstra Link: http://lkml.kernel.org/r/20140122115918.GG3694@twins.programming.kicks-ass.net Signed-off-by: Ingo Molnar

sched/clock: Fix up clear_sched_clock_stable()

2014-01-13T14:13:15Z

The below tells us the static_key conversion has a problem; since the exact point of clearing that flag isn't too important, delay the flip and use a workqueue to process it. [ ] TSC synchronization [CPU#0 -> CPU#22]: [ ] Measured 8 cycles TSC warp between CPUs, turning off TSC clock. [ ] [ ] ====================================================== [ ] [ INFO: possible circular locking dependency detected ] [ ] 3.13.0-rc3-01745-g848b0d0322cb-dirty #637 Not tainted [ ] ------------------------------------------------------- [ ] swapper/0/1 is trying to acquire lock: [ ] (jump_label_mutex){+.+...}, at: [] jump_label_lock+0x17/0x20 [ ] [ ] but task is already holding lock: [ ] (cpu_hotplug.lock){+.+.+.}, at: [] cpu_hotplug_begin+0x2b/0x60 [ ] [ ] which lock already depends on the new lock. [ ] [ ] [ ] the existing dependency chain (in reverse order) is: [ ] [ ] -> #1 (cpu_hotplug.lock){+.+.+.}: [ ] [] lock_acquire+0x90/0x130 [ ] [] mutex_lock_nested+0x63/0x3e0 [ ] [] get_online_cpus+0x3c/0x60 [ ] [] arch_jump_label_transform+0x37/0x130 [ ] [] __jump_label_update+0x5f/0x80 [ ] [] jump_label_update+0x9d/0xb0 [ ] [] static_key_slow_inc+0x9d/0xb0 [ ] [] sched_feat_set+0xf5/0x100 [ ] [] set_numabalancing_state+0x2c/0x30 [ ] [] numa_policy_init+0x1af/0x1b7 [ ] [] start_kernel+0x35d/0x41f [ ] [] x86_64_start_reservations+0x2a/0x2c [ ] [] x86_64_start_kernel+0xfb/0xfe [ ] [ ] -> #0 (jump_label_mutex){+.+...}: [ ] [] __lock_acquire+0x1701/0x1eb0 [ ] [] lock_acquire+0x90/0x130 [ ] [] mutex_lock_nested+0x63/0x3e0 [ ] [] jump_label_lock+0x17/0x20 [ ] [] static_key_slow_inc+0x6b/0xb0 [ ] [] clear_sched_clock_stable+0x15/0x20 [ ] [] mark_tsc_unstable+0x23/0x70 [ ] [] check_tsc_sync_source+0x14b/0x150 [ ] [] native_cpu_up+0x3a2/0x890 [ ] [] _cpu_up+0xdb/0x160 [ ] [] cpu_up+0x79/0x90 [ ] [] smp_init+0x60/0x8c [ ] [] kernel_init_freeable+0x8c/0x197 [ ] [] kernel_init+0xe/0x130 [ ] [] ret_from_fork+0x7c/0xb0 [ ] [ ] other info that might help us debug this: [ ] [ ] Possible unsafe locking scenario: [ ] [ ] CPU0 CPU1 [ ] ---- ---- [ ] lock(cpu_hotplug.lock); [ ] lock(jump_label_mutex); [ ] lock(cpu_hotplug.lock); [ ] lock(jump_label_mutex); [ ] [ ] *** DEADLOCK *** [ ] [ ] 2 locks held by swapper/0/1: [ ] #0: (cpu_add_remove_lock){+.+.+.}, at: [] cpu_maps_update_begin+0x17/0x20 [ ] #1: (cpu_hotplug.lock){+.+.+.}, at: [] cpu_hotplug_begin+0x2b/0x60 [ ] [ ] stack backtrace: [ ] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.13.0-rc3-01745-g848b0d0322cb-dirty #637 [ ] Hardware name: Supermicro X8DTN/X8DTN, BIOS 4.6.3 01/08/2010 [ ] ffffffff82c9c270 ffff880236843bb8 ffffffff8165c5f5 ffffffff82c9c270 [ ] ffff880236843bf8 ffffffff81658c02 ffff880236843c80 ffff8802368586a0 [ ] ffff880236858678 0000000000000001 0000000000000002 ffff880236858000 [ ] Call Trace: [ ] [] dump_stack+0x4e/0x7a [ ] [] print_circular_bug+0x1f9/0x207 [ ] [] __lock_acquire+0x1701/0x1eb0 [ ] [] ? __atomic_notifier_call_chain+0x8f/0xb0 [ ] [] lock_acquire+0x90/0x130 [ ] [] ? jump_label_lock+0x17/0x20 [ ] [] ? jump_label_lock+0x17/0x20 [ ] [] mutex_lock_nested+0x63/0x3e0 [ ] [] ? jump_label_lock+0x17/0x20 [ ] [] jump_label_lock+0x17/0x20 [ ] [] static_key_slow_inc+0x6b/0xb0 [ ] [] clear_sched_clock_stable+0x15/0x20 [ ] [] mark_tsc_unstable+0x23/0x70 [ ] [] check_tsc_sync_source+0x14b/0x150 [ ] [] native_cpu_up+0x3a2/0x890 [ ] [] _cpu_up+0xdb/0x160 [ ] [] cpu_up+0x79/0x90 [ ] [] smp_init+0x60/0x8c [ ] [] kernel_init_freeable+0x8c/0x197 [ ] [] ? rest_init+0xd0/0xd0 [ ] [] kernel_init+0xe/0x130 [ ] [] ret_from_fork+0x7c/0xb0 [ ] [] ? rest_init+0xd0/0xd0 [ ] ------------[ cut here ]------------ [ ] WARNING: CPU: 0 PID: 1 at /usr/src/linux-2.6/kernel/smp.c:374 smp_call_function_many+0xad/0x300() [ ] Modules linked in: [ ] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.13.0-rc3-01745-g848b0d0322cb-dirty #637 [ ] Hardware name: Supermicro X8DTN/X8DTN, BIOS 4.6.3 01/08/2010 [ ] 0000000000000009 ffff880236843be0 ffffffff8165c5f5 0000000000000000 [ ] ffff880236843c18 ffffffff81093d8c 0000000000000000 0000000000000000 [ ] ffffffff81ccd1a0 ffffffff810ca951 0000000000000000 ffff880236843c28 [ ] Call Trace: [ ] [] dump_stack+0x4e/0x7a [ ] [] warn_slowpath_common+0x8c/0xc0 [ ] [] ? sched_clock_tick+0x1/0xa0 [ ] [] warn_slowpath_null+0x1a/0x20 [ ] [] smp_call_function_many+0xad/0x300 [ ] [] ? arch_unregister_cpu+0x30/0x30 [ ] [] ? arch_unregister_cpu+0x30/0x30 [ ] [] ? sched_clock_tick+0x1/0xa0 [ ] [] smp_call_function+0x46/0x80 [ ] [] ? arch_unregister_cpu+0x30/0x30 [ ] [] on_each_cpu+0x3c/0xa0 [ ] [] ? sched_clock_idle_sleep_event+0x20/0x20 [ ] [] ? sched_clock_tick+0x1/0xa0 [ ] [] text_poke_bp+0x64/0xd0 [ ] [] ? sched_clock_idle_sleep_event+0x20/0x20 [ ] [] arch_jump_label_transform+0xae/0x130 [ ] [] __jump_label_update+0x5f/0x80 [ ] [] jump_label_update+0x9d/0xb0 [ ] [] static_key_slow_inc+0x9d/0xb0 [ ] [] clear_sched_clock_stable+0x15/0x20 [ ] [] mark_tsc_unstable+0x23/0x70 [ ] [] check_tsc_sync_source+0x14b/0x150 [ ] [] native_cpu_up+0x3a2/0x890 [ ] [] _cpu_up+0xdb/0x160 [ ] [] cpu_up+0x79/0x90 [ ] [] smp_init+0x60/0x8c [ ] [] kernel_init_freeable+0x8c/0x197 [ ] [] ? rest_init+0xd0/0xd0 [ ] [] kernel_init+0xe/0x130 [ ] [] ret_from_fork+0x7c/0xb0 [ ] [] ? rest_init+0xd0/0xd0 [ ] ---[ end trace 6ff1df5620c49d26 ]--- [ ] tsc: Marking TSC unstable due to check_tsc_sync_source failed Signed-off-by: Peter Zijlstra Link: http://lkml.kernel.org/n/tip-v55fgqj3nnyqnngmvuu8ep6h@git.kernel.org Signed-off-by: Ingo Molnar

sched/clock, x86: Use a static_key for sched_clock_stable

2014-01-13T14:13:13Z

In order to avoid the runtime condition and variable load turn sched_clock_stable into a static_key. Also provide a shorter implementation of local_clock() and cpu_clock(int) when sched_clock_stable==1. MAINLINE PRE POST sched_clock_stable: 1 1 1 (cold) sched_clock: 329841 221876 215295 (cold) local_clock: 301773 234692 220773 (warm) sched_clock: 38375 25602 25659 (warm) local_clock: 100371 33265 27242 (warm) rdtsc: 27340 24214 24208 sched_clock_stable: 0 0 0 (cold) sched_clock: 382634 235941 237019 (cold) local_clock: 396890 297017 294819 (warm) sched_clock: 38194 25233 25609 (warm) local_clock: 143452 71234 71232 (warm) rdtsc: 27345 24245 24243 Signed-off-by: Peter Zijlstra Cc: Linus Torvalds Cc: Andrew Morton Link: http://lkml.kernel.org/n/tip-eummbdechzz37mwmpags1gjr@git.kernel.org Signed-off-by: Ingo Molnar