aboutsummaryrefslogtreecommitdiffstats
path: root/src (follow)
AgeCommit message (Collapse)AuthorFilesLines
2011-01-29copy: make extent_copy use sparse_copy, rather than its own codeJim Meyering1-60/+49
* src/copy.c (extent_copy): Before this change, extent_copy would fail to create holes, thus breaking --sparse=auto and --sparse=always. I.e., copying a large enough file of all zeros, cp --sparse=always should introduce a hole, but with extent_copy, it would not.
2011-01-29copy: remove obsolete commentJim Meyering1-12/+9
* src/copy.c (sparse_copy): Remove now-obsolete comment about how we used to work around lack of ftruncate. Combine nested if conditions into one.
2011-01-29copy: factor sparse-copying code into its own function, becauseJim Meyering1-98/+114
we're going to have to use it from within extent_copy, too. * src/copy.c (sparse_copy): New function, factored out of... (copy_reg): ...here. Remove now-unused locals.
2011-01-29fiemap copy: avoid leak-on-errorJim Meyering1-4/+6
* src/copy.c (extent_copy): Don't leak an extent_scan buffer on failed lseek, read, or write.
2011-01-29fiemap copy: avoid a performance hit due to very small bufferJim Meyering1-3/+2
* src/copy.c (extent_copy): Don't let what should have been a temporary reduction of buf_size (to handle a short ext_len) become permanent and thus impact the performance of all further iterations.
2011-01-29fiemap copy: simplify post-loop logic; improve commentsJim Meyering1-30/+16
* src/copy.c (extent_copy): Avoid duplication in post-loop extend-to-desired-length code.
2011-01-29fiemap copy: rename some localsJim Meyering1-11/+11
(extent_copy): Rename locals: s/*ext_logical/*ext_start/
2011-01-22copy: don't allocate a separate buffer just for extent-based copyJim Meyering1-19/+17
* src/copy.c (copy_reg): Move use of extent_scan to just *after* we allocate the main copying buffer, so we can... (extent_scan): Take a new parameter, BUF, and use that rather than allocating a private buffer. Update caller.
2011-01-22copy: tweak variable name; improve a commentJim Meyering1-6/+6
* src/copy.c (copy_reg): Rename a variable to make more sense from caller's perspective: s/require_normal_copy/normal_copy_required/. This is an output-only variable, and the original name could make it look like an input (or i&o) variable.
2011-01-22copy: call extent_copy also when make_holes is false, ...Jim Meyering1-16/+13
so that we benefit from using extents also when reading a sparse input file with --sparse=never. * src/copy.c (copy_reg): Remove erroneous test of "make_holes" so that we call extent_copy also when make_holes is false. Otherwise, what's the point of that parameter?
2011-01-22* src/copy.c (copy_reg): Remove useless else-after-goto.Jim Meyering1-6/+4
2011-01-22copy.c: shorten a comment to fit in 80 columnsJim Meyering1-1/+1
2011-01-22extent-scan.c: don't include error.h or quote.hJim Meyering1-2/+0
* src/extent-scan.c: Don't include error.h or quote.h. Neither is used.
2011-01-22formattingJim Meyering2-5/+8
2011-01-22distribute extent-scan.h, tooJim Meyering1-1/+1
* src/Makefile.am (copy_sources): Also distribute extent-scan.h.
2011-01-22rename extent-scan functions to start with extent_scan_Jim Meyering3-22/+22
2011-01-22rename extent_scan memberJim Meyering3-6/+6
* extent-scan.h [struct extent_scan]: Rename member: s/hit_last_extent/hit_final_extent/. "final" is clearer, since "last" can be interpreted as "preceding".
2011-01-22fiemap copy: don't let write failure go unreported; adjust style, etc.Jim Meyering1-9/+19
* src/copy.c (write_zeros): Add comments. (extent_copy): Move decls of "ok" and "i" down to scope where used. Adjust comments. Rename local: s/holes_len/hole_size/ Print a diagnostic upon failure to write zeros.
2011-01-22bug#6131: [PATCH]: fiemap support for efficient sparse file copyjeff.liu4-80/+296
Jim Meyering wrote: > jeff.liu wrote: >> Sorry for the delay. >> >> This is the new patch to isolate the stuff regarding to extents reading to a new module. and teach >> cp(1) to make use of it. > > Jeff, > > I applied your patch to my rebased fiemap-copy branch. > My first step was to run the usual > > ./bootstrap && ./configure && make && make check > > "make check" failed on due to a double free in your new code: > (x86_64, Fedora 13, ext4 working directory) > > To get details, I made this temporary modification: Hi Jim, I am sorry for the fault, it fixed at the patch below. Would you please revie at your convenience? Changes: ======== 1. fix write_zeros() as Jim's comments, thanks for pointing this out. 2. remove char const *fname from struct extent_scan. 3. change the signature of open_extent_scan() from "void open_extent_scan(struct extent_scan **scan)" to "void open_extent_scan(struct extent_scan *scan)"; the reason is I'd like to reduce once memory allocation for the extent_scan variable, instead, using stack to save it. 4. remove close_extent_scan() from a function defined at extent-scan.c to extent-scan.h as a Macro definination, but it does nothing for now, since initial extent scan defined at stack. 5. add a macro "free_extents_info()" defined at extent-scan.h to release the memory allocated to extent info which should be called combine with get_extents_info(), it just one line, so IMHO, define it as macro should be ok. I have done the memory check via `valgrind`, no issue found. make test against cp/sparse-fiemap failed at the extent compare stage, but the file content is identical to each other by comparing those two files "j1/j2" manually. Is it make sense if we verify them through diff(1) since the testing file is in small size? or we have to merge the contig extents from the output of `filefrag', I admit I have not dig into the filefrag-extent-compare at the moment, I need to recall the perl language syntax. :-P. >From 50a3338db06442fa2d789fd65175172d140cc96e Mon Sep 17 00:00:00 2001 From: Jie Liu <jeff.liu@oracle.com> Date: Wed, 29 Sep 2010 15:35:43 +0800 Subject: [PATCH 1/1] cp: add a new module for scanning extents * src/extent-scan.c: Source code for scanning extents. Call open_extent_scan() to initialize extent scan. Call get_extents_info() to get a number of extents for each iteration. * src/extent-scan.h: Header file of extent-scan.c. Wrap free_extent_info() as macro define to release the space allocated extent_info per extent scan. Wrap close_extent_scan() as macro define but do nothing at the moment. * src/Makefile.am: Reference it and link it to copy_source. * src/copy.c: Make use of the new module, replace fiemap_copy() with extent_copy(). Signed-off-by: Jie Liu <jeff.liu@oracle.com>
2011-01-22build: distribute new file, fiemap.hJim Meyering1-0/+1
* src/Makefile.am (noinst_HEADERS): Add fiemap.h.
2011-01-22copy.c: add FIEMAP_FLAG_SYNC to fiemap ioctlJie Liu1-0/+1
* src/copy.c (fiemap_copy): Force kernel to sync the source file before mapping.
2011-01-22fiemap.h: include <stdint.h>, not <linux/types.h>Jim Meyering1-1/+1
* src/fiemap.h: Include stdint.h, not linux/types.h, now that this file uses only portable type names.
2011-01-22copy.c: ensure proper alignment of fiemap bufferPaul Eggert1-7/+8
* src/copy.c (fiemap_copy): Ensure that our fiemap buffer is large enough and well-aligned. Replace "0LL" with equivalent "0" as 3rd argument to lseek.
2011-01-22copy.c: adjust comments, tweak semanticsJim Meyering1-26/+36
* src/copy.c (fiemap_copy): Rename from fiemap_copy_ok. Add/improve comments. Remove local, "fail". (fiemap_copy): Do not require caller to set "normal_copy_required" before calling fiemap_copy. Report ioctl failure if it's the 2nd or subsequent call.
2011-01-22cp: Add FIEMAP support for efficient sparse file copyJie Liu2-0/+261
* src/fiemap.h: Add fiemap.h for fiemap ioctl(2) support. Copied from linux's include/linux/fiemap.h, with minor formatting changes. * src/copy.c (copy_reg): Now, when `cp' invoked with --sparse=[WHEN] option, we will try to do FIEMAP-copy if the underlaying file system support it, fall back to a normal copy if it fails.
2011-01-17uniq: replace a wasteful loop with simple calculationJim Meyering1-2/+1
* src/uniq.c (find_field): Remove the byte-skipping loop altogether. Instead, perform the simple calculation. This results in a 10% performance improvement for large byte offsets.
2011-01-17uniq: don't continue field processing after end of lineSami Kerola1-1/+1
* NEWS (Bug fixes): Mention it. * src/uniq.c (find_field): Stop processing loop when end of line is reached. Before this fix, 'uniq -f 10000000000 /etc/passwd' would run for a very long time.
2011-01-14maint: refactor to use read-file from gnulibPádraig Brady2-119/+11
* bootstrap.conf: Add the read-file module * src/ptx.c: Replace the original code which would needlessly read SIZE_MAX bytes of files larger than this. * src/shuf.c: Replace the original code.
2011-01-13maint: trivial system header file cleanupsPádraig Brady4-7/+9
* src/system.h: Note where it should be included, and make ordering check portable to GLIBC > 2 * src/copy.c: Move <sys/ioctl.h> along with other system headers as is done elsewhere. * src/install.c: Move <sys/wait.h> along with other system headers as is done elsewhere. * src/ptx.c: Include <regex.h> rather than "regex.h" as is done elsewhere. Note <regex.h> is kept after "system.h" as per commit dba300a0.
2011-01-11rm: ignore errno related to invalid file namesNadav Har'El1-1/+9
* src/remove.c (nonexistent_file_errno): Also skip EINVAL and EILSEQ, for at least smbfs rejection of '*' in file names. * NEWS: Mention the fix.
2011-01-10du: don't abort when a subdir is renamed during traversalJim Meyering1-4/+11
* NEWS (Bug fixes): Mention it. * src/du.c (prev_level): Move declaration "up" to file-scope global. (du_files): Reset prev_level to 0 upon abnormal fts_read termination. Reported by Johathan Nieder in http://bugs.debian.org/609049 Also, improve a diagnostic. * tests/du/move-dir-while-traversing: Test for the above. * tests/Makefile.am (TESTS): Add it.
2011-01-07maint: suppress some clang scan-build warningsPádraig Brady3-6/+8
* src/pr.c (char_to_clump): Remove a dead store. * src/remove.c (fts_skip_tree): Likewise. * src/sort.c (key_warnings): Likewise. (sort): Suppress an uninitialized pointer warning.
2011-01-07maint: replace uses of ignore_ptr with ignore_valuePádraig Brady3-3/+3
* gnulib: Update for enhanced ignore_value() * src/chcon.c (process_file): Don't use the deprecated ignore_ptr. * src/chmod.c (process_file): Likewise. * src/chown-core.c (change_file_owner): Likewise.
2011-01-01maint: update all copyright year number rangesJim Meyering120-120/+120
Run "make update-copyright".
2010-12-30split: fix the suffix length calculationPádraig Brady1-12/+21
* src/split.c (set_suffix_length): Only auto-calculate the suffix length when the number of files is specified. * tests/misc/split-a: Add a case to trigger the bug, and exercise the suffix length auto-calculation. * NEWS: Mention the fix. Reported by Dmitry V. Levin and Sergey Vlasov at https://bugzilla.altlinux.org/show_bug.cgi?id=24841
2010-12-28coreutils: keep lines within 80-column limitsPaul Eggert22-68/+112
* cfg.mk (LINE_LEN_MAX, FILTER_LONG_LINES): New macros. (sc_long_lines): New rule. * HACKING: Use shorter URLs to the same material. * doc/Makefile.am, doc/coreutils.texi, m4/boottime.m4: * man/help2man, man/stdbuf.x, src/Makefile.am, src/cat.c, src/copy.c: * src/cp.c, src/dd.c, src/df.c, src/du.c, src/groups.c, src/install.c: * src/ls.c, src/md5sum.c, src/mv.c, src/od.c, src/pinky.c, src/ptx.c: * src/readlink.c, src/remove.c, src/rmdir.c, src/setuidgid.c: * src/sort.c, src/tail.c, src/touch.c, tests/Coreutils.pm: * tests/cp/existing-perm-race, tests/cp/perm, tests/cp/preserve-gid: * tests/du/2g, tests/du/long-from-unreadable, tests/init.sh: * tests/install/basic-1, tests/ls/nameless-uid: * tests/ls/readdir-mountpoint-inode, tests/misc/chroot-credentials: * tests/misc/cut, tests/misc/date, tests/misc/join, tests/misc/md5sum: * tests/misc/sha1sum, tests/misc/sha224sum, tests/misc/sort: * tests/misc/sort-continue, tests/misc/sort-files0-from: * tests/misc/sort-rand, tests/misc/stdbuf, tests/misc/tr: * tests/misc/uniq, tests/mv/atomic, tests/mv/part-fail: * tests/mv/part-symlink, tests/mv/sticky-to-xpart, tests/pr/pr-tests: * tests/rm/fail-2eperm, tests/rm/interactive-always: Reformat to fit within 80 columns. * doc/Makefile.am (BAD_POSIX_PERL): New macro. * doc/coreutils.texi: Reword slightly, to make menus and index lines shorter. * src/md5sum.c: Redo --help output so that it fits within 79 columns, since that's a bit more portable and all the other --help strings fit in 79 columns.
2010-12-24maint: avoid syntax-check failure due to unused #includeJim Meyering1-1/+0
* src/getlimits.c: Don't include "c-ctype.h"; no longer used.
2010-12-23csplit: diagnose file counter wraparoundPaul Eggert1-8/+16
* src/csplit.c (create_output_file): Detect overflow when the file counter wraps around, and exit with a diagnostic. Formerly the code silently wrapped around and wrote to the wrong file, losing output data.
2010-12-22sort: minor performance tweak with num_processorsPaul Eggert1-2/+2
* src/sort.c (main): Don't invoke num_processors twice.
2010-12-22getlimits: port to hosts with very wide int, or non-ASCIIPaul Eggert1-50/+24
* src/getlimits.c (decimal_ascii_add): Remove, replacing with ... (decimal_absval_add_one): New function, with different signature, which does not assume ASCII. All callers changed. (print_int): Remove assumptions that integers fit in 206 bits, and that characters are ASCII. These assumptions are portable in practice but are easy to remove here.
2010-12-20who: omit useless definitions of MAXHOSTNAMELENPaul Eggert2-8/+0
This prevents a compilation failure on Solaris 8, GCC 4.4.2, with "configure --enable-gcc-warnings". * src/who.c (MAXHOSTNAMELEN): Remove; no longer needed. * src/pinky.c: Likewise.
2010-12-20maint: fix a typo in sort --parallel help messagePádraig Brady1-1/+1
Also fix up Chen Guo's contacts * src/sort.c (usage): Add a missing "of" * THANKS: Add Chen Guo * .mailmap: Add Chen Guo's UCLA address
2010-12-19wc: fix a possible hang with --files0-fromPádraig Brady1-2/+2
* src/wc.c (main): exit when we get a read error on the --files0-from file, rather than retrying and spinning the CPU
2010-12-19cp: ensure backups are created when -T specifiedPádraig Brady1-0/+2
* src/cp.c (do_copy): When -T is specified, initialize the NEW_DST and SB variables, which are checked when running: cp -T --force --backup file file * tests/cp/backup-1: Add the -T case
2010-12-19sort: use at most 8 threads by defaultPádraig Brady1-6/+11
* src/sort.c (main): If --parallel isn't specified, restrict the number of threads to 8 by default. If the --parallel option is specified, then allow any number of threads to be set, independent of the number of processors on the system. * doc/coreutils.texi (sort invocation): Document the changes to determining the number of threads to use. Mention the memory overhead when using multiple threads. * tests/misc/sort-spinlock-abuse: Allow single core systems that support pthreads. * tests/misc/sort-stale-thread-mem: Likewise. * tests/misc/sort-unique-segv: Likewise. * NEWS: Mention the change in behaviour.
2010-12-16sort: do not generate thousands of subprocesses for 16-way mergePaul Eggert1-13/+21
Without this change, tests/misc/sort-compress-hang would consume more than 10,000 process slots on my RHEL 5.5 x86-64 server, making it likely for other applications to fail due to lack of process slots. With this change, the same benchmark causes 'sort' to consume at most 19 process slots. The change also improved wall-clock time by 2% and user+system time by 14% on that benchmark. * NEWS: Document this. * src/sort.c (MAX_PROCS_BEFORE_REAP): Remove. (reap_exited): Renamed from reap_some; this is a more accurate name, since "some" incorrectly implies that it reaps at least one process. All uses changed. (reap_some): New function: it *does* reap at least one process. (pipe_fork): Do not allow more than NMERGE + 2 subprocesses. (mergefps, sort): Omit check for exited processes: no longer needed, and anyway the code consumed too much CPU per line when 2 < nprocs.
2010-12-16sort: fix hang with sort --compressPaul Eggert1-79/+80
* NEWS: Document this. * src/sort.c (UNCOMPRESSED, UNREAPED, REAPED): New constants. (struct tempnode): New member 'state', to hold these constants. The pid member is now undefined if state == UNCOMPRESSED. (struct sortfile): Replace member 'pid' with member 'temp'. (uintptr): Remove. (proctab_hasher, proctab_comparator, register_proc, delete_proc): Proctab entries are now struct tempnode *, not pid_t, to handle the case where multiple tempnode objects correspond to the same pid. This avoids a race condition that can cause a hang. (register_proc): Arg is now struct tempnode *, not pid_t. All callers changed. (delete_proc): Set tempnode state to REAPED. (create_temp_file): No need to set pid member here; it's now done when the pid is known. (maybe_create_temp, create_temp): Remove PPID arg. Return struct tempnode *, not char *. All callers changed. (maybe_create_temp): Set node state to UNCOMPRESSED or UNREAPED. No need to set node->pid to 0. (open_temp): Replace NAME and PID args with a single TEMP arg. All callers changed. Wait only for unreaped children. (zaptemp): Wait for decompressor to finish before removing its temporary-file input. This avoids .nfsXXXX hassles with NFS and fixes a race (leading to a hang) regardless of NFS. (open_input_files): Adjust to new way of dealing with temp files and their subprocesses. * tests/Makefile.am (TESTS): Add misc/sort-compress-hang. * tests/misc/sort-compress-hang: New file.
2010-12-16sort: don't dump core when merging from input twicePaul Eggert1-25/+17
* NEWS: Document this. * src/sort.c (avoid_trashing_input): The previous fix to this function didn't fix all the problems with this code. Replace it with something simpler: just copy the input file. This doesn't change the number of files, so return void instead of the updated file count. Caller changed. * tests/misc/sort-merge-fdlimit: Test for the bug.
2010-12-14doc: tail: semi-deprecate --sleep-interval and --max-unchanged-statsJim Meyering1-2/+4
Those options are useful only on systems that lack inotify support and in the unusual event that a system with inotify support must resort to polling. * src/tail.c (usage): Note that the --max-unchanged-stats=N and --sleep-interval=N options are rarely useful on systems with inotify support. * doc/coreutils.texi (tail invocation): Likewise.
2010-12-14sort: fix very-unlikely buffer overrun when merging to input filePaul Eggert1-3/+2
* src/sort.c (avoid_trashing_input): Fix a typo that could cause a buffer overrun in theory. In practice this is extremely unlikely, as it requires running out of file descriptors in a small merge, presumably because some other process is hogging all the OS's file descriptors.
generated by cgit v1.2.3 (git 2.39.1) at 2025-12-13 01:27:36 +0000