linux/kernel/sched.c, branch v2.6.25

NOHZ: reevaluate idle sleep length after add_timer_on()

2008-03-26T07:28:55Z

add_timer_on() can add a timer on a CPU which is currently in a long idle sleep, but the timer wheel is not reevaluated by the nohz code on that CPU. So a timer can be delayed for quite a long time. This triggered a false positive in the clocksource watchdog code. To avoid this we need to wake up the idle CPU and enforce the reevaluation of the timer wheel for the next timer event. Add a function, which checks a given CPU for idle state, marks the idle task with NEED_RESCHED and sends a reschedule IPI to notify the other CPU of the change in the timer wheel. Call this function from add_timer_on(). Signed-off-by: Thomas Gleixner Acked-by: Peter Zijlstra Acked-by: Ingo Molnar Cc: stable@kernel.org -- include/linux/sched.h | 6 ++++++ kernel/sched.c | 43 +++++++++++++++++++++++++++++++++++++++++++ kernel/timer.c | 10 +++++++++- 3 files changed, 58 insertions(+), 1 deletion(-)

sched: add arch_update_cpu_topology hook.

2008-03-21T15:43:48Z

Will be called each time the scheduling domains are rebuild. Needed for architectures that don't have a static cpu topology. Signed-off-by: Heiko Carstens Signed-off-by: Martin Schwidefsky Signed-off-by: Ingo Molnar

sched: add exported arch_reinit_sched_domains() to header file.

2008-03-21T15:43:47Z

Needed so it can be called from outside of sched.c. Signed-off-by: Heiko Carstens Signed-off-by: Martin Schwidefsky Signed-off-by: Ingo Molnar

sched: remove double unlikely from schedule()

2008-03-21T15:43:47Z

Combine two unlikely's Signed-off-by: Roel Kluin <12o3l@tiscali.nl> Signed-off-by: Ingo Molnar

sched: cleanup old and rarely used 'debug' features.

2008-03-21T15:43:47Z

TREE_AVG and APPROX_AVG are initial task placement policies that have been disabled for a long while.. time to remove them. Signed-off-by: Peter Zijlstra CC: Srivatsa Vaddagiri Signed-off-by: Ingo Molnar

sched: wakeup-buddy tasks are cache-hot

2008-03-19T03:27:53Z

Wakeup-buddy tasks are cache-hot - this makes it a bit harder for the load-balancer to tear them apart. (but it's still possible, if the load is sufficiently assymetric) Signed-off-by: Ingo Molnar

sched: improve affine wakeups

2008-03-19T03:27:53Z

improve affine wakeups. Maintain the 'overlap' metric based on CFS's sum_exec_runtime - which means the amount of time a task executes after it wakes up some other task. Use the 'overlap' for the wakeup decisions: if the 'overlap' is short, it means there's strong workload coupling between this task and the woken up task. If the 'overlap' is large then the workload is decoupled and the scheduler will move them to separate CPUs more easily. ( Also slightly move the preempt_check within try_to_wake_up() - this has no effect on functionality but allows 'early wakeups' (for still-on-rq tasks) to be correctly accounted as well.) Signed-off-by: Ingo Molnar

sched: fix overload performance: buddy wakeups

2008-03-15T02:02:50Z

Currently we schedule to the leftmost task in the runqueue. When the runtimes are very short because of some server/client ping-pong, especially in over-saturated workloads, this will cycle through all tasks trashing the cache. Reduce cache trashing by keeping dependent tasks together by running newly woken tasks first. However, by not running the leftmost task first we could starve tasks because the wakee can gain unlimited runtime. Therefore we only run the wakee if its within a small (wakeup_granularity) window of the leftmost task. This preserves fairness, but does alternate server/client task groups. Signed-off-by: Peter Zijlstra Signed-off-by: Ingo Molnar

sched: fix calc_delta_mine()

2008-03-15T02:02:50Z

lw->weight can be 0 for a short time during bootup. Signed-off-by: Ingo Molnar Signed-off-by: Peter Zijlstra

sched: fix update_load_add()/sub()

2008-03-15T02:02:49Z

Clear the cached inverse value when updating load. This is needed for calc_delta_mine() to work correctly when using the rq load. Signed-off-by: Ingo Molnar Signed-off-by: Peter Zijlstra