linux/drivers/block/loop.c, branch v4.5

Merge branch 'for-4.4/core' into for-4.4/drivers

2015-10-09T16:40:29Z

Signed-off-by: Jens Axboe

blk-mq: fix racy updates of rq->errors

2015-10-01T08:10:55Z

blk_mq_complete_request may be a no-op if the request has already been completed by others means (e.g. a timeout or cancellation), but currently drivers have to set rq->errors before calling blk_mq_complete_request, which might leave us with the wrong error value. Add an error parameter to blk_mq_complete_request so that we can defer setting rq->errors until we known we won the race to complete the request. Signed-off-by: Christoph Hellwig Reviewed-by: Sagi Grimberg Signed-off-by: Jens Axboe

block: loop: support DIO & AIO

2015-09-23T17:01:16Z

There are at least 3 advantages to use direct I/O and AIO on read/write loop's backing file: 1) double cache can be avoided, then memory usage gets decreased a lot 2) not like user space direct I/O, there isn't cost of pinning pages 3) avoid context switch for obtaining good throughput - in buffered file read, random I/O top throughput is often obtained only if they are submitted concurrently from lots of tasks; but for sequential I/O, most of times they can be hit from page cache, so concurrent submissions often introduce unnecessary context switch and can't improve throughput much. There was such discussion[1] to use non-blocking I/O to improve the problem for application. - with direct I/O and AIO, concurrent submissions can be avoided and random read throughput can't be affected meantime xfstests(-g auto, ext4) is basically passed when running with direct I/O(aio), one exception is generic/232, but it failed in loop buffered I/O(4.2-rc6-next-20150814) too. Follows the fio test result for performance purpose: 4 jobs fio test inside ext4 file system over loop block 1) How to run - KVM: 4 VCPUs, 2G RAM - linux kernel: 4.2-rc6-next-20150814(base) with the patchset - the loop block is over one image on SSD. - linux psync, 4 jobs, size 1500M, ext4 over loop block - test result: IOPS from fio output 2) Throughput(IOPS) becomes a bit better with direct I/O(aio) ------------------------------------------------------------- test cases |randread |read |randwrite |write | ------------------------------------------------------------- base |8015 |113811 |67442 |106978 ------------------------------------------------------------- base+loop aio |8136 |125040 |67811 |111376 ------------------------------------------------------------- - somehow, it should be caused by more page cache avaiable for application or one extra page copy is avoided in case of direct I/O 3) context switch - context switch decreased by ~50% with loop direct I/O(aio) compared with loop buffered I/O(4.2-rc6-next-20150814) 4) memory usage from /proc/meminfo ------------------------------------------------------------- | Buffers | Cached ------------------------------------------------------------- base | > 760MB | ~950MB ------------------------------------------------------------- base+loop direct I/O(aio) | < 5MB | ~1.6GB ------------------------------------------------------------- - so there are much more page caches available for application with direct I/O [1] https://lwn.net/Articles/612483/ Signed-off-by: Ming Lei Reviewed-by: Christoph Hellwig Signed-off-by: Jens Axboe

block: loop: introduce ioctl command of LOOP_SET_DIRECT_IO

2015-09-23T17:01:16Z

If loop block is mounted via 'mount -o loop', it isn't easy to pass file descriptor opened as O_DIRECT, so this patch introduces a new command to support direct IO for this case. Cc: linux-api@vger.kernel.org Signed-off-by: Ming Lei Reviewed-by: Christoph Hellwig Signed-off-by: Jens Axboe

block: loop: prepare for supporing direct IO

2015-09-23T17:01:16Z

This patches provides one interface for enabling direct IO from user space: - userspace(such as losetup) can pass 'file' which is opened/fcntl as O_DIRECT Also __loop_update_dio() is introduced to check if direct I/O can be used on current loop setting. The last big change is to introduce LO_FLAGS_DIRECT_IO flag for userspace to know if direct IO is used to access backing file. Cc: linux-api@vger.kernel.org Signed-off-by: Ming Lei Reviewed-by: Christoph Hellwig Signed-off-by: Jens Axboe

block: loop: use kthread_work

2015-09-23T17:01:16Z

The following patch will use dio/aio to submit IO to backing file, then it needn't to schedule IO concurrently from work, so use kthread_work for decreasing context switch cost a lot. For non-AIO case, single thread has been used for long long time, and it was just converted to work in v4.0, which has caused performance regression for fedora live booting already. In discussion[1], even though submitting I/O via work concurrently can improve random read IO throughput, meantime it might hurt sequential read IO performance, so better to restore to single thread behaviour. For the following AIO support, it is better to use multi hw-queue with per-hwq kthread than current work approach suppose there is so high performance requirement for loop. [1] http://marc.info/?t=143082678400002&r=1&w=2 Signed-off-by: Ming Lei Reviewed-by: Christoph Hellwig Signed-off-by: Jens Axboe

block: loop: set QUEUE_FLAG_NOMERGES for request queue of loop

2015-09-23T17:01:16Z

It doesn't make sense to enable merge because the I/O submitted to backing file is handled page by page. Signed-off-by: Ming Lei Reviewed-by: Christoph Hellwig Signed-off-by: Jens Axboe

block: have drivers use blk_queue_max_discard_sectors()

2015-07-17T14:41:53Z

Some drivers use it now, others just set the limits field manually. But in preparation for splitting this into a hard and soft limit, ensure that they all call the proper function for setting the hw limit for discards. Reviewed-by: Jeff Moyer Signed-off-by: Jens Axboe

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2015-07-05T02:36:06Z

Pull more vfs updates from Al Viro: "Assorted VFS fixes and related cleanups (IMO the most interesting in that part are f_path-related things and Eric's descriptor-related stuff). UFS regression fixes (it got broken last cycle). 9P fixes. fs-cache series, DAX patches, Jan's file_remove_suid() work" [ I'd say this is much more than "fixes and related cleanups". The file_table locking rule change by Eric Dumazet is a rather big and fundamental update even if the patch isn't huge. - Linus ] * 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs: (49 commits) 9p: cope with bogus responses from server in p9_client_{read,write} p9_client_write(): avoid double p9_free_req() 9p: forgetting to cancel request on interrupted zero-copy RPC dax: bdev_direct_access() may sleep block: Add support for DAX reads/writes to block devices dax: Use copy_from_iter_nocache dax: Add block size note to documentation fs/file.c: __fget() and dup2() atomicity rules fs/file.c: don't acquire files->file_lock in fd_install() fs:super:get_anon_bdev: fix race condition could cause dev exceed its upper limitation vfs: avoid creation of inode number 0 in get_next_ino namei: make set_root_rcu() return void make simple_positive() public ufs: use dir_pages instead of ufs_dir_pages() pagemap.h: move dir_pages() over there remove the pointless include of lglock.h fs: cleanup slight list_entry abuse xfs: Correctly lock inode when removing suid and file capabilities fs: Call security_ops->inode_killpriv on truncate fs: Provide function telling whether file_remove_privs() will do anything ...

Merge branch 'for-4.2/drivers' of git://git.kernel.dk/linux-block

2015-06-25T22:12:50Z

Pull block driver updates from Jens Axboe: "This contains: - a few race fixes for null_blk, from Akinobu Mita. - a series of fixes for mtip32xx, from Asai Thambi and Selvan Mani at Micron. - NVMe: * Fix for missing error return on allocation failure, from Axel Lin. * Code consolidation and cleanups from Christoph. * Memory barrier addition, syncing queue count and queue pointers. From Jon Derrick. * Various fixes from Keith, an addition to support user issue reset from sysfs or ioctl, and automatic namespace rescan. * Fix from Matias, avoiding losing some request flags when marking the request failfast. - small cleanups and sparse fixups for ps3vram. From Geert Uytterhoeven and Geoff Lavand. - s390/dasd dead code removal, from Jarod Wilson. - a set of fixes and optimizations for loop, from Ming Lei. - conversion to blkdev_reread_part() of loop, dasd, ndb. From Ming Lei. - updates to cciss. From Tomas Henzl" * 'for-4.2/drivers' of git://git.kernel.dk/linux-block: (44 commits) mtip32xx: Fix accessing freed memory block: nvme-scsi: Catch kcalloc failure NVMe: Fix IO for extended metadata formats nvme: don't overwrite req->cmd_flags on sync cmd mtip32xx: increase wait time for hba reset mtip32xx: fix minor number mtip32xx: remove unnecessary sleep in mtip_ftl_rebuild_poll() mtip32xx: fix crash on surprise removal of the drive mtip32xx: Abort I/O during secure erase operation mtip32xx: fix incorrectly setting MTIP_DDF_SEC_LOCK_BIT mtip32xx: remove unused variable 'port->allocated' mtip32xx: fix rmmod issue MAINTAINERS: Update ps3vram block driver block/ps3vram: Remove obsolete reference to MTD block/ps3vram: Fix sparse warnings NVMe: Automatic namespace rescan NVMe: Memory barrier before queue_count is incremented NVMe: add sysfs and ioctl controller reset null_blk: restart request processing on completion handler null_blk: prevent timer handler running on a different CPU where started ...