find duplicates from the cli using checksum

linux: find . -type f -print0 | xargs -0 -n1 md5sum | sort -k 1,32 | uniq -w 32 -d –all-repeated=separate | sed -e ‘s/^[0-9a-f]*\ *//;’ mac osx: find . -type f -print0 | xargs -0 cksum | sort | awk ‘{if($1 == prevsum) {printf(“—-\n %s\n %s\n”, prev, $0);} prev=$0; prevsum=$1;}’

gentoo goodness

new kernel goodness : (0.37)-(~)-(villa-straylight)-(28 files, 480Kb)-> uname -a Linux villa-straylight 3.0.0-gentoo-poa #1 SMP PREEMPT Tue Jul 26 09:30:13 EDT 2011 x86_64 AMD Phenom(tm) II X6 1055T Processor AuthenticAMD GNU/Linux size comparisons from the last couple of kernels : -rw-r–r– 1 root root 3536336 Jun 21 14:40 vmlinuz-2.6.39-gentoo-cdfscode -rw-r–r– 1 root root 3633520 Jun 9 … Read More