mirror of https://github.com/oliverbooth/fdup.git
feat: initial commit
This commit is contained in:
commit
f8018096c5
GPG Key ID:
E60B570D1B7557B5
|
|
@ -0,0 +1,133 @@
|
|||
# Contributor Covenant Code of Conduct
|
||||
|
||||
## Our Pledge
|
||||
|
||||
We as members, contributors, and leaders pledge to make participation in our
|
||||
community a harassment-free experience for everyone, regardless of age, body
|
||||
size, visible or invisible disability, ethnicity, sex characteristics, gender
|
||||
identity and expression, level of experience, education, socio-economic status,
|
||||
nationality, personal appearance, race, caste, color, religion, or sexual
|
||||
identity and orientation.
|
||||
|
||||
We pledge to act and interact in ways that contribute to an open, welcoming,
|
||||
diverse, inclusive, and healthy community.
|
||||
|
||||
## Our Standards
|
||||
|
||||
Examples of behavior that contributes to a positive environment for our
|
||||
community include:
|
||||
|
||||
* Demonstrating empathy and kindness toward other people
|
||||
* Being respectful of differing opinions, viewpoints, and experiences
|
||||
* Giving and gracefully accepting constructive feedback
|
||||
* Accepting responsibility and apologizing to those affected by our mistakes,
|
||||
and learning from the experience
|
||||
* Focusing on what is best not just for us as individuals, but for the overall
|
||||
community
|
||||
|
||||
Examples of unacceptable behavior include:
|
||||
|
||||
* The use of sexualized language or imagery, and sexual attention or advances of
|
||||
any kind
|
||||
* Trolling, insulting or derogatory comments, and personal or political attacks
|
||||
* Public or private harassment
|
||||
* Publishing others' private information, such as a physical or email address,
|
||||
without their explicit permission
|
||||
* Other conduct which could reasonably be considered inappropriate in a
|
||||
professional setting
|
||||
|
||||
## Enforcement Responsibilities
|
||||
|
||||
Community leaders are responsible for clarifying and enforcing our standards of
|
||||
acceptable behavior and will take appropriate and fair corrective action in
|
||||
response to any behavior that they deem inappropriate, threatening, offensive,
|
||||
or harmful.
|
||||
|
||||
Community leaders have the right and responsibility to remove, edit, or reject
|
||||
comments, commits, code, wiki edits, issues, and other contributions that are
|
||||
not aligned to this Code of Conduct, and will communicate reasons for moderation
|
||||
decisions when appropriate.
|
||||
|
||||
## Scope
|
||||
|
||||
This Code of Conduct applies within all community spaces, and also applies when
|
||||
an individual is officially representing the community in public spaces.
|
||||
Examples of representing our community include using an official e-mail address,
|
||||
posting via an official social media account, or acting as an appointed
|
||||
representative at an online or offline event.
|
||||
|
||||
## Enforcement
|
||||
|
||||
Instances of abusive, harassing, or otherwise unacceptable behavior may be
|
||||
reported to the community leaders responsible for enforcement at
|
||||
yiliansource@gmail.com or me@olivr.me.
|
||||
All complaints will be reviewed and investigated promptly and fairly.
|
||||
|
||||
All community leaders are obligated to respect the privacy and security of the
|
||||
reporter of any incident.
|
||||
|
||||
## Enforcement Guidelines
|
||||
|
||||
Community leaders will follow these Community Impact Guidelines in determining
|
||||
the consequences for any action they deem in violation of this Code of Conduct:
|
||||
|
||||
### 1. Correction
|
||||
|
||||
**Community Impact**: Use of inappropriate language or other behavior deemed
|
||||
unprofessional or unwelcome in the community.
|
||||
|
||||
**Consequence**: A private, written warning from community leaders, providing
|
||||
clarity around the nature of the violation and an explanation of why the
|
||||
behavior was inappropriate. A public apology may be requested.
|
||||
|
||||
### 2. Warning
|
||||
|
||||
**Community Impact**: A violation through a single incident or series of
|
||||
actions.
|
||||
|
||||
**Consequence**: A warning with consequences for continued behavior. No
|
||||
interaction with the people involved, including unsolicited interaction with
|
||||
those enforcing the Code of Conduct, for a specified period of time. This
|
||||
includes avoiding interactions in community spaces as well as external channels
|
||||
like social media. Violating these terms may lead to a temporary or permanent
|
||||
ban.
|
||||
|
||||
### 3. Temporary Ban
|
||||
|
||||
**Community Impact**: A serious violation of community standards, including
|
||||
sustained inappropriate behavior.
|
||||
|
||||
**Consequence**: A temporary ban from any sort of interaction or public
|
||||
communication with the community for a specified period of time. No public or
|
||||
private interaction with the people involved, including unsolicited interaction
|
||||
with those enforcing the Code of Conduct, is allowed during this period.
|
||||
Violating these terms may lead to a permanent ban.
|
||||
|
||||
### 4. Permanent Ban
|
||||
|
||||
**Community Impact**: Demonstrating a pattern of violation of community
|
||||
standards, including sustained inappropriate behavior, harassment of an
|
||||
individual, or aggression toward or disparagement of classes of individuals.
|
||||
|
||||
**Consequence**: A permanent ban from any sort of public interaction within the
|
||||
community.
|
||||
|
||||
## Attribution
|
||||
|
||||
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
|
||||
version 2.1, available at
|
||||
[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
|
||||
|
||||
Community Impact Guidelines were inspired by
|
||||
[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
|
||||
|
||||
For answers to common questions about this code of conduct, see the FAQ at
|
||||
[https://www.contributor-covenant.org/faq][FAQ]. Translations are available at
|
||||
[https://www.contributor-covenant.org/translations][translations].
|
||||
|
||||
[homepage]: https://www.contributor-covenant.org
|
||||
[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
|
||||
[Mozilla CoC]: https://github.com/mozilla/diversity
|
||||
[FAQ]: https://www.contributor-covenant.org/faq
|
||||
[translations]: https://www.contributor-covenant.org/translations
|
||||
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
ko_fi: oliverbooth
|
||||
custom: ['https://buymeacoffee.com/oliverbooth']
|
||||
|
|
@ -0,0 +1,53 @@
|
|||
name: .NET
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- '*'
|
||||
- '*/*'
|
||||
pull_request:
|
||||
branches:
|
||||
- '*'
|
||||
- '*/*'
|
||||
|
||||
jobs:
|
||||
build:
|
||||
name: "Build & Test"
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v3
|
||||
|
||||
- name: Setup .NET
|
||||
uses: actions/setup-dotnet@v3
|
||||
with:
|
||||
dotnet-version: |
|
||||
6.0.x
|
||||
7.0.x
|
||||
8.0.x
|
||||
|
||||
- name: Add NuGet source
|
||||
run: dotnet nuget add source --username oliverbooth --password ${{ secrets.GITHUB_TOKEN }} --store-password-in-clear-text --name github "https://nuget.pkg.github.com/oliverbooth/index.json"
|
||||
|
||||
- name: Restore dependencies
|
||||
run: dotnet restore
|
||||
|
||||
- name: Build
|
||||
run: dotnet build --no-restore --configuration Release
|
||||
|
||||
- name: Test .NET 6
|
||||
run: dotnet test --no-build --verbosity normal --configuration Release --framework net6.0 --collect:"XPlat Code Coverage" --results-directory test-results/net6.0
|
||||
|
||||
- name: Test .NET 7
|
||||
run: dotnet test --no-build --verbosity normal --configuration Release --framework net7.0 --collect:"XPlat Code Coverage" --results-directory test-results/net7.0
|
||||
|
||||
- name: Test .NET 8
|
||||
run: dotnet test --no-build --verbosity normal --configuration Release --framework net8.0 --collect:"XPlat Code Coverage" --results-directory test-results/net8.0
|
||||
|
||||
- name: Upload coverage reports to Codecov
|
||||
uses: codecov/codecov-action@v4.0.0
|
||||
with:
|
||||
directory: test-results
|
||||
token: ${{ secrets.CODECOV_TOKEN }}
|
||||
slug: oliverbooth/X10D
|
||||
|
|
@ -0,0 +1,37 @@
|
|||
*.swp
|
||||
*.*~
|
||||
project.lock.json
|
||||
.DS_Store
|
||||
*.pyc
|
||||
nupkg/
|
||||
|
||||
# Visual Studio Code
|
||||
.vscode
|
||||
|
||||
# Rider
|
||||
.idea
|
||||
|
||||
# User-specific files
|
||||
*.suo
|
||||
*.user
|
||||
*.userosscache
|
||||
*.sln.docstates
|
||||
|
||||
# Build results
|
||||
[Dd]ebug/
|
||||
[Dd]ebugPublic/
|
||||
[Rr]elease/
|
||||
[Rr]eleases/
|
||||
x64/
|
||||
x86/
|
||||
build/
|
||||
bld/
|
||||
[Bb]in/
|
||||
[Oo]bj/
|
||||
[Oo]ut/
|
||||
msbuild.log
|
||||
msbuild.err
|
||||
msbuild.wrn
|
||||
|
||||
# Visual Studio 2015
|
||||
.vs/
|
||||
|
|
@ -0,0 +1,16 @@
|
|||
|
||||
Microsoft Visual Studio Solution File, Format Version 12.00
|
||||
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "FindDuplicates", "FindDuplicates\FindDuplicates.csproj", "{543F0776-01CD-420B-973A-BCE11F47C758}"
|
||||
EndProject
|
||||
Global
|
||||
GlobalSection(SolutionConfigurationPlatforms) = preSolution
|
||||
Debug|Any CPU = Debug|Any CPU
|
||||
Release|Any CPU = Release|Any CPU
|
||||
EndGlobalSection
|
||||
GlobalSection(ProjectConfigurationPlatforms) = postSolution
|
||||
{543F0776-01CD-420B-973A-BCE11F47C758}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
|
||||
{543F0776-01CD-420B-973A-BCE11F47C758}.Debug|Any CPU.Build.0 = Debug|Any CPU
|
||||
{543F0776-01CD-420B-973A-BCE11F47C758}.Release|Any CPU.ActiveCfg = Release|Any CPU
|
||||
{543F0776-01CD-420B-973A-BCE11F47C758}.Release|Any CPU.Build.0 = Release|Any CPU
|
||||
EndGlobalSection
|
||||
EndGlobal
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<PropertyGroup>
|
||||
<OutputType>Exe</OutputType>
|
||||
<TargetFramework>net8.0</TargetFramework>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<Nullable>enable</Nullable>
|
||||
<AssemblyName>fdup</AssemblyName>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Spectre.Console.Cli" Version="0.48.0"/>
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
|
|
@ -0,0 +1,108 @@
|
|||
using System.Diagnostics;
|
||||
using System.Security.Cryptography;
|
||||
using System.Text;
|
||||
using Spectre.Console;
|
||||
using Spectre.Console.Cli;
|
||||
|
||||
namespace FindDuplicates;
|
||||
|
||||
internal sealed class ListCommand : AsyncCommand<ListSettings>
|
||||
{
|
||||
private readonly Dictionary<string, List<FileInfo>> _fileHashMap = new();
|
||||
|
||||
public override async Task<int> ExecuteAsync(CommandContext context, ListSettings settings)
|
||||
{
|
||||
var inputDirectory = new DirectoryInfo(settings.InputPath);
|
||||
if (!inputDirectory.Exists)
|
||||
{
|
||||
AnsiConsole.MarkupLine($"[red]{inputDirectory} does not exist![/]");
|
||||
return -1;
|
||||
}
|
||||
|
||||
AnsiConsole.MarkupLineInterpolated($"Searching [cyan]{inputDirectory.FullName}[/]");
|
||||
AnsiConsole.MarkupLine($"Recursive mode is {(settings.Recursive ? "[green]ON" : "[red]OFF")}[/]");
|
||||
|
||||
await SearchAsync(inputDirectory, settings);
|
||||
|
||||
AnsiConsole.WriteLine();
|
||||
|
||||
int duplicates = 0;
|
||||
foreach ((string hash, List<FileInfo> files) in _fileHashMap)
|
||||
{
|
||||
int fileCount = files.Count;
|
||||
|
||||
if (fileCount > 1)
|
||||
{
|
||||
duplicates += fileCount;
|
||||
AnsiConsole.MarkupLineInterpolated($"Found [cyan]{fileCount}[/] identical files");
|
||||
AnsiConsole.MarkupLineInterpolated($"SHA512 [green]{hash}[/]:");
|
||||
|
||||
foreach (FileInfo file in files)
|
||||
AnsiConsole.MarkupLineInterpolated($"- {file.FullName}");
|
||||
|
||||
AnsiConsole.WriteLine();
|
||||
}
|
||||
}
|
||||
|
||||
if (duplicates == 0)
|
||||
AnsiConsole.MarkupLine("[green]No duplicates found![/]");
|
||||
else
|
||||
AnsiConsole.MarkupLineInterpolated($"[yellow]Found [cyan]{duplicates}[/] duplicates![/]");
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
private async Task SearchAsync(DirectoryInfo inputDirectory, ListSettings settings)
|
||||
{
|
||||
var tasks = new List<Task>();
|
||||
var directoryStack = new Stack<DirectoryInfo>([inputDirectory]);
|
||||
while (directoryStack.Count > 0)
|
||||
{
|
||||
DirectoryInfo currentDirectory = directoryStack.Pop();
|
||||
string relativePath = Path.GetRelativePath(inputDirectory.FullName, currentDirectory.FullName);
|
||||
if (relativePath != ".")
|
||||
AnsiConsole.MarkupLineInterpolated($"Searching [cyan]{relativePath}[/]");
|
||||
|
||||
if (settings.Recursive)
|
||||
{
|
||||
foreach (DirectoryInfo childDirectory in currentDirectory.EnumerateDirectories())
|
||||
directoryStack.Push(childDirectory);
|
||||
}
|
||||
|
||||
foreach (FileInfo file in currentDirectory.EnumerateFiles())
|
||||
{
|
||||
string relativeFilePath = Path.GetRelativePath(inputDirectory.FullName, file.FullName);
|
||||
AnsiConsole.MarkupLineInterpolated($"Checking hash for [cyan]{relativeFilePath}[/]");
|
||||
tasks.Add(Task.Run(() => ProcessFile(file)));
|
||||
}
|
||||
}
|
||||
|
||||
await Task.WhenAll(tasks);
|
||||
}
|
||||
|
||||
private void ProcessFile(FileInfo file)
|
||||
{
|
||||
Span<byte> buffer = stackalloc byte[64];
|
||||
using FileStream stream = file.OpenRead();
|
||||
using BufferedStream bufferedStream = new BufferedStream(stream, 1048576 /* 1MB */);
|
||||
SHA512.HashData(bufferedStream, buffer);
|
||||
string hash = ByteSpanToString(buffer);
|
||||
Trace.WriteLine($"{file.FullName}: {hash}");
|
||||
|
||||
if (!_fileHashMap.TryGetValue(hash, out List<FileInfo>? cache))
|
||||
_fileHashMap[hash] = cache = new List<FileInfo>();
|
||||
|
||||
lock (cache)
|
||||
cache.Add(file);
|
||||
}
|
||||
|
||||
private static string ByteSpanToString(ReadOnlySpan<byte> buffer)
|
||||
{
|
||||
var builder = new StringBuilder();
|
||||
|
||||
foreach (byte b in buffer)
|
||||
builder.Append($"{b:X2}");
|
||||
|
||||
return builder.ToString();
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,16 @@
|
|||
using System.ComponentModel;
|
||||
using Spectre.Console.Cli;
|
||||
|
||||
namespace FindDuplicates;
|
||||
|
||||
internal sealed class ListSettings : CommandSettings
|
||||
{
|
||||
[CommandArgument(0, "<path>")]
|
||||
[Description("The path to search.")]
|
||||
public string InputPath { get; set; } = string.Empty;
|
||||
|
||||
[CommandOption("-r|--recursive")]
|
||||
[Description("When this flag is set, the directory will be scanned recursively. This may take longer.")]
|
||||
[DefaultValue(false)]
|
||||
public bool Recursive { get; set; } = false;
|
||||
}
|
||||
|
|
@ -0,0 +1,5 @@
|
|||
using FindDuplicates;
|
||||
using Spectre.Console.Cli;
|
||||
|
||||
var app = new CommandApp<ListCommand>();
|
||||
await app.RunAsync(args).ConfigureAwait(false);
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
MIT License
|
||||
|
||||
Copyright (c) 2024 Oliver Booth
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
|
|
@ -0,0 +1,57 @@
|
|||
# Find Duplicates (fdup)
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## About
|
||||
fdup is a small command-line utility written in C# to quickly and easily find duplicate files. It can also search recursively to find duplicate files in child directories.
|
||||
|
||||
## Usage
|
||||
```bash
|
||||
$ fdup --help
|
||||
USAGE:
|
||||
fdup <path> [OPTIONS]
|
||||
|
||||
ARGUMENTS:
|
||||
<path> The path to search
|
||||
|
||||
OPTIONS:
|
||||
-h, --help Prints help information
|
||||
-v, --version Prints version information
|
||||
-r, --recursive When this flag is set, the directory will be scanned recursively. This may take longer
|
||||
```
|
||||
|
||||
## Example
|
||||
```bash
|
||||
$ echo "Hello World" > file1
|
||||
$ echo "Goodbye World" > file2
|
||||
$ fdup .
|
||||
Searching /home/user/example
|
||||
Recursive mode is OFF
|
||||
Checking hash for file2
|
||||
Checking hash for file1
|
||||
|
||||
No duplicates found!
|
||||
$ echo "Hello World" > file2
|
||||
$ fdup .
|
||||
Searching /home/user/example
|
||||
Recursive mode is OFF
|
||||
Checking hash for file2
|
||||
Checking hash for file1
|
||||
|
||||
Found 2 identical files
|
||||
SHA512 E1C112FF908FEBC3B98B1693A6CD3564EAF8E5E6CA629D084D9F0EBA99247CACDD72E369FF8941397C2807409FF66BE64BE908DA17AD7B8A49A2A26C0E8086AA:
|
||||
- /home/user/example/file1
|
||||
- /home/user/example/file2
|
||||
|
||||
Found 2 duplicates!
|
||||
```
|
||||
|
||||
## Contributing
|
||||
|
||||
Contributions are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md).
|
||||
|
||||
## License
|
||||
|
||||
X10D is released under the MIT License. See [here](https://github.com/oliverbooth/X10D/blob/main/LICENSE.md) for more details.
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
{
|
||||
"sdk": {
|
||||
"version": "8.0.0",
|
||||
"rollForward": "latestMinor",
|
||||
"allowPrerelease": false
|
||||
}
|
||||
}
|
||||
Loading…
Reference in New Issue