Mercurial > dedupe

annotate scripts/duplicates.sh @ 110:bae09ef524cb

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Fixes escaping problem in SQL. Introduces striping functionality for prefixes.
author Tom Fredrik Blenning Klaussen <bfg@bfgconsult.no>
date Tue, 18 Feb 2014 19:42:05 +0100
parents 6b997f4f7e19
children 3951f6d27219
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
97
34f11b2a1178 Script to extract duplicates directly to the commandline.
Tom Fredrik Blenning Klaussen <bfg@bfgconsult.no>
parents:
diff changeset
1 #!/bin/bash
34f11b2a1178 Script to extract duplicates directly to the commandline.
Tom Fredrik Blenning Klaussen <bfg@bfgconsult.no>
parents:
diff changeset
2
34f11b2a1178 Script to extract duplicates directly to the commandline.
Tom Fredrik Blenning Klaussen <bfg@bfgconsult.no>
parents:
diff changeset
3 PREFIX=$1
34f11b2a1178 Script to extract duplicates directly to the commandline.
Tom Fredrik Blenning Klaussen <bfg@bfgconsult.no>
parents:
diff changeset
4
110
bae09ef524cb Fixes escaping problem in SQL.
Tom Fredrik Blenning Klaussen <bfg@bfgconsult.no>
parents: 103
diff changeset
5 STRIP=true
97
34f11b2a1178 Script to extract duplicates directly to the commandline.
Tom Fredrik Blenning Klaussen <bfg@bfgconsult.no>
parents:
diff changeset
6 if [ -n "$PREFIX" ]
34f11b2a1178 Script to extract duplicates directly to the commandline.
Tom Fredrik Blenning Klaussen <bfg@bfgconsult.no>
parents:
diff changeset
7 then
110
bae09ef524cb Fixes escaping problem in SQL.
Tom Fredrik Blenning Klaussen <bfg@bfgconsult.no>
parents: 103
diff changeset
8 SEDPREFIX=$(echo $PREFIX | sed -e 's/\//\\\//g')
bae09ef524cb Fixes escaping problem in SQL.
Tom Fredrik Blenning Klaussen <bfg@bfgconsult.no>
parents: 103
diff changeset
9 SQLPREFIX=$(echo $PREFIX | sed -e "s/'/''/g")
bae09ef524cb Fixes escaping problem in SQL.
Tom Fredrik Blenning Klaussen <bfg@bfgconsult.no>
parents: 103
diff changeset
10
bae09ef524cb Fixes escaping problem in SQL.
Tom Fredrik Blenning Klaussen <bfg@bfgconsult.no>
parents: 103
diff changeset
11 PREFIX="WHERE path LIKE '${SQLPREFIX}%'"
97
34f11b2a1178 Script to extract duplicates directly to the commandline.
Tom Fredrik Blenning Klaussen <bfg@bfgconsult.no>
parents:
diff changeset
12 fi
34f11b2a1178 Script to extract duplicates directly to the commandline.
Tom Fredrik Blenning Klaussen <bfg@bfgconsult.no>
parents:
diff changeset
13
110
bae09ef524cb Fixes escaping problem in SQL.
Tom Fredrik Blenning Klaussen <bfg@bfgconsult.no>
parents: 103
diff changeset
14 if ${STRIP}
bae09ef524cb Fixes escaping problem in SQL.
Tom Fredrik Blenning Klaussen <bfg@bfgconsult.no>
parents: 103
diff changeset
15 then
bae09ef524cb Fixes escaping problem in SQL.
Tom Fredrik Blenning Klaussen <bfg@bfgconsult.no>
parents: 103
diff changeset
16 sqlite3 ~/.DeDupe.sqlite "SELECT hex(checksum),size,path FROM (SELECT * FROM files WHERE checksum IN (SELECT checksum FROM (SELECT checksum, COUNT(*) as num FROM files ${PREFIX} GROUP BY checksum) WHERE num >= 2)) ${PREFIX} ORDER BY size;" | sed "s/|${SEDPREFIX}/|/"
bae09ef524cb Fixes escaping problem in SQL.
Tom Fredrik Blenning Klaussen <bfg@bfgconsult.no>
parents: 103
diff changeset
17 else
bae09ef524cb Fixes escaping problem in SQL.
Tom Fredrik Blenning Klaussen <bfg@bfgconsult.no>
parents: 103
diff changeset
18 sqlite3 ~/.DeDupe.sqlite "SELECT hex(checksum),size,path FROM (SELECT * FROM files WHERE checksum IN (SELECT checksum FROM (SELECT checksum, COUNT(*) as num FROM files ${PREFIX} GROUP BY checksum) WHERE num >= 2)) ${PREFIX} ORDER BY size;"
bae09ef524cb Fixes escaping problem in SQL.
Tom Fredrik Blenning Klaussen <bfg@bfgconsult.no>
parents: 103
diff changeset
19 fi