linux/block, branch v4.4-rc3

Revert "blk-flush: Queue through IO scheduler when flush not required"

2015-11-25T17:12:54Z

This reverts commit 1b2ff19e6a957b1ef0f365ad331b608af80e932e. Jan writes: -- Thanks for report! After some investigation I found out we allocate elevator specific data in __get_request() only for non-flush requests. And this is actually required since the flush machinery uses the space in struct request for something else. Doh. So my patch is just wrong and not easy to fix since at the time __get_request() is called we are not sure whether the flush machinery will be used in the end. Jens, please revert 1b2ff19e6a957b1ef0f365ad331b608af80e932e. Thanks! I'm somewhat surprised that you can reliably hit the race where flushing gets disabled for the device just while the request is in flight. But I guess during boot it makes some sense. -- So let's just revert it, we can fix the queue run manually after the fact. This race is rare enough that it didn't trigger in testing, it requires the specific disable-while-in-flight scenario to trigger.

block: fix blk_abort_request for blk-mq drivers

2015-11-24T22:24:10Z

We only added the request to the request list for the !blk-mq case, so we should only delete it in that case as well. Signed-off-by: Christoph Hellwig Signed-off-by: Jens Axboe

Merge branch 'for-linus' of git://git.kernel.dk/linux-block

2015-11-24T18:26:30Z

Pull block layer fixes from Jens Axboe: "A round of fixes/updates for the current series. This looks a little bigger than it is, but that's mainly because we pushed the lightnvm enabled null_blk change out of the merge window so it could be updated a bit. The rest of the volume is also mostly lightnvm. In particular: - Lightnvm. Various fixes, additions, updates from Matias and Javier, as well as from Wenwei Tao. - NVMe: - Fix for potential arithmetic overflow from Keith. - Also from Keith, ensure that we reap pending completions from a completion queue before deleting it. Fixes kernel crashes when resetting a device with IO pending. - Various little lightnvm related tweaks from Matias. - Fixup flushes to go through the IO scheduler, for the cases where a flush is not required. Fixes a case in CFQ where we would be idling and not see this request, hence not break the idling. From Jan Kara. - Use list_{first,prev,next} in elevator.c for cleaner code. From Gelian Tang. - Fix for a warning trigger on btrfs and raid on single queue blk-mq devices, where we would flush plug callbacks with preemption disabled. From me. - A mac partition validation fix from Kees Cook. - Two merge fixes from Ming, marked stable. A third part is adding a new warning so we'll notice this quicker in the future, if we screw up the accounting. - Cleanup of thread name/creation in mtip32xx from Rasmus Villemoes" * 'for-linus' of git://git.kernel.dk/linux-block: (32 commits) blk-merge: warn if figured out segment number is bigger than nr_phys_segments blk-merge: fix blk_bio_segment_split block: fix segment split blk-mq: fix calling unplug callbacks with preempt disabled mac: validate mac_partition is within sector mtip32xx: use formatting capability of kthread_create_on_node NVMe: reap completion entries when deleting queue lightnvm: add free and bad lun info to show luns lightnvm: keep track of block counts nvme: lightnvm: use admin queues for admin cmds lightnvm: missing free on init error lightnvm: wrong return value and redundant free null_blk: do not del gendisk with lightnvm null_blk: use device addressing mode null_blk: use ppa_cache pool NVMe: Fix possible arithmetic overflow for max segments blk-flush: Queue through IO scheduler when flush not required null_blk: register as a LightNVM device elevator: use list_{first,prev,next}_entry lightnvm: cleanup queue before target removal ...

blk-merge: warn if figured out segment number is bigger than nr_phys_segments

2015-11-24T03:16:55Z

We had seen lots of reports of this kind issue, so add one warnning in blk-merge, then it can be triggered easily and avoid to depend on warning/bug from drivers. Signed-off-by: Ming Lei Signed-off-by: Jens Axboe

blk-merge: fix blk_bio_segment_split

2015-11-24T03:16:53Z

Commit bdced438acd83a(block: setup bi_phys_segments after splitting) introduces function of computing bio->bi_phys_segments during bio splitting. Unfortunately both bio->bi_seg_front_size and bio->bi_seg_back_size arn't computed, so too many physical segments may be obtained for one request since both the two are used to check if one segment across two bios can be possible. This patch fixes the issue by computing the two variables in blk_bio_segment_split(). Fixes: bdced438acd83a(block: setup bi_phys_segments after splitting) Reported-by: Michael Ellerman Reported-by: Mark Salter Tested-by: Laurent Dufour Tested-by: Mark Salter Signed-off-by: Ming Lei Signed-off-by: Jens Axboe

block: fix segment split

2015-11-24T03:16:51Z

Inside blk_bio_segment_split(), previous bvec pointer(bvprvp) always points to the iterator local variable, which is obviously wrong, so fix it by pointing to the local variable of 'bvprv'. Fixes: 5014c311baa2b(block: fix bogus compiler warnings in blk-merge.c) Cc: stable@kernel.org #4.3 Reported-by: Michael Ellerman Reported-by: Mark Salter Tested-by: Laurent Dufour Tested-by: Mark Salter Signed-off-by: Ming Lei Signed-off-by: Jens Axboe

blk-mq: fix calling unplug callbacks with preempt disabled

2015-11-21T03:29:45Z

Liu reported that running certain parts of xfstests threw the following error: BUG: sleeping function called from invalid context at mm/page_alloc.c:3190 in_atomic(): 1, irqs_disabled(): 0, pid: 6, name: kworker/u16:0 3 locks held by kworker/u16:0/6: #0: ("writeback"){++++.+}, at: [] process_one_work+0x173/0x730 #1: ((&(&wb->dwork)->work)){+.+.+.}, at: [] process_one_work+0x173/0x730 #2: (&type->s_umount_key#44){+++++.}, at: [] trylock_super+0x25/0x60 CPU: 5 PID: 6 Comm: kworker/u16:0 Tainted: G OE 4.3.0+ #3 Hardware name: Red Hat KVM, BIOS Bochs 01/01/2011 Workqueue: writeback wb_workfn (flush-btrfs-108) ffffffff81a3abab ffff88042e282ba8 ffffffff8130191b ffffffff81a3abab 0000000000000c76 ffff88042e282ba8 ffff88042e27c180 ffff88042e282bd8 ffffffff8108ed95 ffff880400000004 0000000000000000 0000000000000c76 Call Trace: [] dump_stack+0x4f/0x74 [] ___might_sleep+0x185/0x240 [] __might_sleep+0x52/0x90 [] __alloc_pages_nodemask+0x268/0x410 [] ? sched_clock_local+0x1c/0x90 [] ? local_clock+0x21/0x40 [] ? __lock_release+0x420/0x510 [] ? __lock_acquired+0x16c/0x3c0 [] alloc_pages_current+0xc5/0x210 [] ? rbio_is_full+0x55/0x70 [btrfs] [] ? mark_held_locks+0x78/0xa0 [] ? _raw_spin_unlock_irqrestore+0x40/0x60 [] full_stripe_write+0x5a/0xc0 [btrfs] [] __raid56_parity_write+0x39/0x60 [btrfs] [] run_plug+0x11b/0x140 [btrfs] [] btrfs_raid_unplug+0x23/0x70 [btrfs] [] blk_flush_plug_list+0x82/0x1f0 [] blk_sq_make_request+0x1f9/0x740 [] ? generic_make_request_checks+0x222/0x7c0 [] ? blk_queue_enter+0x124/0x310 [] ? blk_queue_enter+0x92/0x310 [] generic_make_request+0x172/0x2c0 [] ? generic_make_request+0x164/0x2c0 [] submit_bio+0x70/0x140 [] ? rbio_add_io_page+0x99/0x150 [btrfs] [] finish_rmw+0x4d9/0x600 [btrfs] [] full_stripe_write+0x9c/0xc0 [btrfs] [] raid56_parity_write+0xef/0x160 [btrfs] [] btrfs_map_bio+0xe3/0x2d0 [btrfs] [] btrfs_submit_bio_hook+0x8d/0x1d0 [btrfs] [] submit_one_bio+0x74/0xb0 [btrfs] [] submit_extent_page+0xe5/0x1c0 [btrfs] [] __extent_writepage_io+0x408/0x4c0 [btrfs] [] ? alloc_dummy_extent_buffer+0x140/0x140 [btrfs] [] __extent_writepage+0x218/0x3a0 [btrfs] [] ? mark_held_locks+0x78/0xa0 [] extent_write_cache_pages.clone.0+0x2f9/0x400 [btrfs] [] extent_writepages+0x52/0x70 [btrfs] [] ? btrfs_set_inode_index+0x70/0x70 [btrfs] [] btrfs_writepages+0x27/0x30 [btrfs] [] do_writepages+0x23/0x40 [] __writeback_single_inode+0x89/0x4d0 [] ? writeback_sb_inodes+0x260/0x480 [] ? writeback_sb_inodes+0x260/0x480 [] ? writeback_sb_inodes+0x15f/0x480 [] writeback_sb_inodes+0x2d2/0x480 [] ? down_read_trylock+0x57/0x60 [] ? trylock_super+0x25/0x60 [] ? rcu_read_lock_sched_held+0x4f/0x90 [] __writeback_inodes_wb+0x8c/0xc0 [] wb_writeback+0x2b5/0x500 [] ? mark_held_locks+0x78/0xa0 [] ? __local_bh_enable_ip+0x68/0xc0 [] ? wb_do_writeback+0x62/0x310 [] wb_do_writeback+0xc1/0x310 [] ? set_worker_desc+0x79/0x90 [] wb_workfn+0x92/0x330 [] process_one_work+0x223/0x730 [] ? process_one_work+0x173/0x730 [] ? worker_thread+0x18f/0x430 [] worker_thread+0x11d/0x430 [] ? maybe_create_worker+0xf0/0xf0 [] ? maybe_create_worker+0xf0/0xf0 [] kthread+0xef/0x110 [] ? schedule_tail+0x1e/0xd0 [] ? __init_kthread_worker+0x70/0x70 [] ret_from_fork+0x3f/0x70 [] ? __init_kthread_worker+0x70/0x70 The issue is that we've got the software context pinned while calling blk_flush_plug_list(), which flushes callbacks that are allowed to sleep. btrfs and raid has such callbacks. Flip the checks around a bit, so we can enable preempt a bit earlier and flush plugs without having preempt disabled. This only affects blk-mq driven devices, and only those that register a single queue. Reported-by: Liu Bo Tested-by: Liu Bo Cc: stable@kernel.org Signed-off-by: Jens Axboe

mac: validate mac_partition is within sector

2015-11-20T15:49:28Z

If md->signature == MAC_DRIVER_MAGIC and md->block_size == 1023, a single 512 byte sector would be read (secsize / 512). However the partition structure would be located past the end of the buffer (secsize % 512). Signed-off-by: Kees Cook Cc: stable@vger.kernel.org Signed-off-by: Jens Axboe

block: protect rw_page against device teardown

2015-11-19T21:47:10Z

Fix use after free crashes like the following: general protection fault: 0000 [#1] SMP Call Trace: [] ? pmem_do_bvec.isra.12+0xa6/0xf0 [nd_pmem] [] pmem_rw_page+0x42/0x80 [nd_pmem] [] bdev_read_page+0x50/0x60 [] do_mpage_readpage+0x510/0x770 [] ? I_BDEV+0x20/0x20 [] ? lru_cache_add+0x1c/0x50 [] mpage_readpages+0x107/0x170 [] ? I_BDEV+0x20/0x20 [] ? I_BDEV+0x20/0x20 [] blkdev_readpages+0x1d/0x20 [] __do_page_cache_readahead+0x28f/0x310 [] ? __do_page_cache_readahead+0x169/0x310 [] ? pagecache_get_page+0x2d/0x1d0 [] filemap_fault+0x396/0x530 [] __do_fault+0x4e/0xf0 [] handle_mm_fault+0x11bd/0x1b50 Cc: Cc: Jens Axboe Cc: Alexander Viro Reported-by: kbuild test robot Acked-by: Matthew Wilcox [willy: symmetry fixups] Signed-off-by: Dan Williams

blk-flush: Queue through IO scheduler when flush not required

2015-11-16T22:23:51Z

Currently blk_insert_flush() just adds flush request to q->queue_head when flush is not required. That completely bypasses IO scheduler so e.g. CFQ can be idling waiting for new request to arrive and will idle through the whole window unnecessarily. Luckily this only happens in rare cases as usually checks in generic_make_request_checks() clear FLUSH and FUA flags early if they are not needed. When no flushing is actually required, we can easily fix the problem by properly queueing the request through the IO scheduler. Ideally IO scheduler should be also made aware of requests queued via blk_flush_queue_rq(). However inserting flush request through IO scheduler can have unwanted side-effects since due to flush batching delaying the flush request in IO scheduler will delay all flush requests possibly coming from other processes. So we keep adding the request directly to q->queue_head. Signed-off-by: Jan Kara Reviewed-by: Jeff Moyer Signed-off-by: Jens Axboe