Introduction
add2dir is a command‑line utility designed to facilitate the movement and copying of files into destination directories under a variety of conditions. It is commonly used in scripting environments, build automation, and data management workflows where large numbers of files must be organized efficiently. The tool offers a combination of straightforward copy/move operations, directory creation, pattern matching, and checksum verification, enabling users to maintain file integrity and directory structure with minimal manual intervention.
History and Background
Origins
The concept of a utility that automatically populates directories with selected files traces back to early Unix file management practices. The need for robust automation arose in the 1990s, when software distribution increasingly relied on package managers and build tools. add2dir emerged in the early 2000s as an open‑source project written in C++ with a focus on portability and performance. Its initial release targeted Linux systems but was later extended to support Windows and macOS via cross‑compilation tools.
Evolution
Over the past two decades, add2dir has undergone numerous revisions. Version 1.x introduced basic copy and move functionality, while version 2.x added support for recursive directory traversal and glob patterns. Version 3.0 expanded checksum verification using MD5 and SHA‑256, and added a verbosity mode for detailed logging. The most recent 4.x series focuses on security enhancements, such as safe path resolution and avoidance of directory traversal attacks, and provides an API for embedding the utility within other applications.
Core Functionality
File Transfer Modes
add2dir supports three primary modes of file transfer:
- Copy – duplicates the source file into the destination directory while preserving the original file.
- Move – transfers the file, removing the original copy from the source location.
- Link – creates a symbolic or hard link in the destination directory that points to the original file.
Users can select the desired mode using the -c, -m, or -l options, respectively. The utility automatically determines whether the operating system supports symbolic links and falls back to hard links where necessary.
Pattern Matching and Filters
add2dir accepts Unix‑style glob patterns to specify which files should be processed. The patterns may be provided directly as arguments or read from a file via the -p option. Additionally, the --include and --exclude options allow fine‑grained control over file selection, supporting multiple patterns per option.
Directory Management
When the destination directory does not exist, add2dir can create it automatically if the -d flag is set. Users may also specify nested directories within the destination path; add2dir will create intermediate directories as required. A dry‑run mode is available via the -n flag, which reports the actions that would be taken without performing any file operations.
Checksum Verification
Integrity checks are an integral part of add2dir’s workflow. After copying or moving a file, the utility compares the MD5 or SHA‑256 hash of the source and destination files. Users can choose the algorithm with the -s option. If a mismatch is detected, the file is considered corrupted, and the utility attempts a retry or aborts based on the -r option. The hash results are logged to a specified file using the -h option.
Installation and Compatibility
Supported Platforms
add2dir is designed to run on a wide range of operating systems:
- Linux (kernel 2.6 and newer)
- macOS (10.12 and newer)
- Windows 7 and newer (64‑bit)
- FreeBSD, OpenBSD, and other BSD derivatives
All platforms require a C++11 compatible compiler for source builds. Pre‑compiled binaries are distributed for 64‑bit Linux, macOS, and Windows.
Installation Methods
Users may install add2dir via one of several methods:
- Package Manager – On Linux distributions that support the
aptoryumecosystems, add2dir can be installed withapt install add2diroryum install add2dir. For macOS, Homebrew providesbrew install add2dir. Windows users can download the MSI installer from the project’s release page. - Source Build – Cloning the Git repository followed by running
cmakeandmakewill produce a binary in thebuild/bindirectory. Users must ensure thatopenssldevelopment headers are available for checksum support. - Container Images – Docker images are available on the project’s container registry. A typical run command is
docker run --rm -v /src:/src -v /dst:/dst add2dir add2dir -c /src/*.txt /dst/.
Dependencies
add2dir requires the following libraries at runtime:
- OpenSSL for checksum calculation
- Boost.Filesystem for portable path handling (optional on Linux; mandatory on Windows)
- libuv for asynchronous I/O (used only when the
--asyncflag is enabled)
All dependencies are automatically detected during configuration, and the build process fails if required libraries are missing.
Command Syntax and Options
Basic Invocation
The general syntax is:
add2dir [options] [source-pattern]... [destination-directory]
At least one source pattern and a destination directory must be specified.
Common Options
-c– copy files (default mode)-m– move files-l– link files (symbolic if supported)-d– create destination directory if it does not exist-n– dry run; show intended actions without executing-s [md5|sha256]– checksum algorithm; default ismd5-r [retry-count]– number of checksum verification retries; default 0 (no retry)-h [file]– write hash results tofile; default isstdout-v– verbose output; can be repeated for increasing detail--async– enable asynchronous file operations (experimental)-p [file]– read patterns fromfile; each line is treated as a pattern--include [pattern]– include files matchingpattern; multiple instances allowed--exclude [pattern]– exclude files matchingpattern; multiple instances allowed
Example Commands
- Copy all JPEG images to a backup directory:
add2dir -c -d /home/user/photos/*.jpg /backup/photos/ - Move log files older than one month to an archive folder, creating the archive if necessary:
add2dir -m -d --include ".log" --exclude ".gz" /var/log/ /archive/logs/ - Create symbolic links for configuration files without copying:
add2dir -l -d /etc/app/config/ ~/app-config/ - Perform a dry run to preview actions:
add2dir -n -c *.txt /tmp/
File Handling Modes
Copy Mode
In copy mode, add2dir opens the source file in read mode and creates a new file at the destination path in write mode. The file’s metadata (permissions, timestamps) is preserved by default. When the -v flag is used, add2dir prints the file name and transfer status after each operation.
Move Mode
Move mode performs a standard rename operation when the source and destination reside on the same filesystem. If the paths are on different filesystems, add2dir falls back to copy-and-delete to ensure data consistency. The operation preserves file metadata unless overridden by the --no-preserve flag (not described earlier but available for advanced use).
Link Mode
Link mode attempts to create a symbolic link if the underlying operating system supports it. On Windows, symbolic links require elevated privileges; if unavailable, add2dir will create a hard link instead. The utility reports the type of link created in verbose mode. Link mode is useful for reducing storage overhead when multiple directories need access to the same file set.
Error Handling
Common Error Scenarios
add2dir detects several error conditions:
- File Not Found – When a source pattern matches no files, the utility logs a warning and continues if other patterns are present.
- Permission Denied – Insufficient permissions to read a source file or write to the destination directory result in an error message. The utility exits with a non‑zero status code.
- Checksum Mismatch – After transferring a file, if the source and destination checksums differ, add2dir reports a corruption error. Depending on the
-rsetting, it may retry the transfer. - Disk Full – If the destination filesystem is full, the operation aborts. A partial transfer may remain; the user should verify file integrity manually.
- Unsupported Link Type – On systems that do not support symbolic links or hard links, add2dir falls back to copy mode and issues a notice.
Exit Status Codes
The utility uses the following exit codes:
0– Success; all files processed without error.1– General error; one or more files could not be processed.2– Invalid arguments or missing required options.3– Permission issues preventing file operations.4– Checksum verification failure after all retries.
Performance Considerations
Sequential vs. Parallel Processing
By default, add2dir processes files sequentially to simplify error handling. The --async flag enables parallel I/O, which can dramatically increase throughput on systems with multiple cores and high‑speed disks. However, asynchronous mode may lead to higher memory usage and is marked as experimental; users should test in staging environments before deploying in production.
Buffer Size Tuning
The internal copy buffer size defaults to 8 MiB. Users can adjust it via the --buffer-size option (value in bytes). Larger buffers reduce system calls for very large files but may increase memory consumption. On SSDs, the optimal buffer size is typically smaller than on spinning disks due to differing I/O characteristics.
Filesystem Caching
When operating on network filesystems such as NFS or SMB, add2dir’s performance is impacted by the network latency and caching policies of the server. Using the --no-cache option forces immediate flushing of data to the remote store, which may be necessary for real‑time backup scenarios.
Use Cases
Backup and Archival
add2dir can be integrated into backup scripts to selectively copy new or modified files into an archive directory while preserving checksums. The dry‑run feature allows administrators to preview changes before committing them, minimizing accidental data loss.
Build Automation
In continuous integration pipelines, build artifacts produced by compilers or assemblers can be automatically routed to artifact repositories. By configuring add2dir to move or copy only files matching certain patterns, build systems can enforce consistent directory layouts across environments.
Configuration Management
System administrators often need to deploy configuration files across multiple servers. Using link mode, add2dir can create symlinks from a central configuration directory to each server’s local directory, ensuring that updates propagate automatically without duplicating files.
Media Organization
Photographers and video editors frequently sort large collections of media files into dated directories. add2dir can automate the process by moving files based on metadata extracted from file names or embedded tags, aiding in long‑term archival and retrieval.
Data Migration
When migrating data between storage tiers or cloud providers, add2dir can transfer large batches of files while verifying integrity. The tool’s ability to resume after failures makes it suitable for long‑running migrations that span multiple network hops.
Integration with Build Systems
Makefile Example
Using add2dir within a Makefile allows build rules to depend on the presence of artifacts in target directories. An example rule:
$(TARGET_DIR): $(SOURCE_FILES) add2dir -m -d -s sha256 $(SOURCE_FILES) $(TARGET_DIR)
This rule moves all source files into the target directory and verifies their integrity using SHA‑256.
Gradle Plugin
A Gradle plugin wrapper exposes add2dir functionality as a task. The wrapper accepts task configuration blocks where users can specify source patterns, destination directories, and options such as checksumAlgorithm or dryRun. This integration allows Java and Kotlin projects to incorporate file distribution logic seamlessly.
Ant Task
The Ant task definition for add2dir accepts properties for source, destination, and a nested options element. This approach is useful in legacy Java build environments that rely on Ant for task orchestration.
Cross‑Platform Support
Linux
Linux distributions typically provide the necessary C++ compiler toolchain and OpenSSL library. The binary is built with the GNU toolchain and supports both 32‑bit and 64‑bit architectures.
macOS
macOS builds use Xcode’s clang compiler. The OpenSSL library is provided via Homebrew or MacPorts. add2dir respects macOS sandboxing rules when invoked from applications.
Windows
On Windows, add2dir is compiled with the MSVC compiler. The OpenSSL library is obtained from the vcpkg package manager. The tool requires administrative privileges for creating symbolic links; otherwise, it uses hard links or copies. Permissions on Windows are mapped to the NTFS ACL system.
BSD Variants
FreeBSD, OpenBSD, and NetBSD builds are available from the ports collection. add2dir leverages the FreeBSD libressl library instead of OpenSSL due to licensing differences.
Security Features
Checksum Verification
Integrity checks prevent silent corruption during data transfer. By default, MD5 is used for speed; SHA‑256 can be selected for environments where security is paramount.
File Metadata Preservation
Preserving timestamps and permissions is essential for security audits. add2dir copies metadata unless overridden, ensuring that file ownership and confidentiality settings are maintained.
Path Traversal Prevention
When patterns include relative components such as ../, add2dir normalizes the path to prevent accidental writes outside the destination directory. In verbose mode, the utility prints the resolved absolute path for each file.
Licensing and Distribution
add2dir is released under the MIT License, allowing free use in commercial and open‑source projects. The source code is hosted on GitHub, with continuous integration pipelines that build and test the tool across all supported platforms. Users may modify and redistribute the software under the same license terms, provided attribution is preserved.
Source Repository
- GitHub:
https://github.com/utility-org/add2dir - Docker image:
docker pull utilityorg/add2dir:latest - Package managers:
apt-get install add2dir,brew install add2dir,vcpkg install add2dir
Contributing
Contributors are encouraged to submit pull requests that address bug fixes, performance improvements, or new feature proposals. The repository includes a CONTRIBUTING.md file outlining guidelines for code style, documentation updates, and testing procedures.
Conclusion
add2dir provides a lightweight, script‑friendly interface for common file distribution tasks. Its support for copying, moving, linking, and integrity verification makes it suitable for a wide range of administrative, development, and data‑management scenarios. By combining a rich set of options with robust error handling and cross‑platform compatibility, add2dir offers a practical alternative to manual file‑management utilities.
For further information, reference the official documentation available on the project website or consult the built‑in help system via add2dir --help.
No comments yet. Be the first to comment!