File :-(, x, )
pic dupe detection software Anonymous
I don't have a perfect memory of my ecchi collection, and occaisionally I get dupes of pics. I can find some of these dupes when the files are identical (same file-hash) but when people have recompressed files or done stupid shit then it won't find the dupes.

I was thinking of writing a program to detect these kinds of dupes myself, by taking the average colour value of a block of pixels in a couple of spots on the picture and then comparing them to every other picture in the collection. Any better ideas about how to do this? Or else what software can I use to do this and save myself a day or two of programming?
>> Anonymous
     File :-(, x)
++
>> Anonymous
     File :-(, x)
++
>> Anonymous
Visipics
>> Anonymous
     File :-(, x)
++

>>490707
Thanks a lot. Seems like the perfect thing :)
>> Anonymous
     File :-(, x)
>> Anonymous
     File :-(, x)
>> Anonymous
     File :-(, x)
>> Anonymous
     File :-(, x)
>> Anonymous
     File :-(, x)
>> Anonymous
     File :-(, x)
>> Anonymous
     File :-(, x)
>> Anonymous
     File :-(, x)
>> Anonymous
     File :-(, x)
>> Anonymous
     File :-(, x)
>> Anonymous
     File :-(, x)
>> Anonymous
     File :-(, x)
>> Anonymous
     File :-(, x)
>> Anonymous
     File :-(, x)
>> Anonymous
     File :-(, x)
Thanks again!
>> Anonymous
as a side note, for Linux/Mac gqview works well for resize/resaved pics, and imgSeek is slower but can also find cropped pics to some degree.
>> Anonymous
>>490705

>++

Quit incrementing my erection.
>> Anonymous
Divide the image into quarters and average all the pixels in each, so each image is effectively reduced to four pixels. The same images usually reduce to the same four pixels, and different images rarely do (though you wouldn't be able to tell just by looking).
>> Anonymous
Sauce on OP pic?
>> Tiver
I've been working on such a program to do image duplicate checking as i've been annoyed with what i've used out there. most of the ones i try crap out on me.

The method i'm using is resizing the image to some WxH size, adjustable by user but has to be same for a set of images to be compared. Then it equalizes the colors so it better matches images where one is brighter or darker than another but they show the same thing.

It builds a database of files and these resized images, i've been using 16x16 for my testing which results in 768 bytes data plus a bit more for filename and last modified date. Then the comparison operation works by calculating the avg distance between the pixels in the 2 16x16 normalized images. basically acting as if each RGB pixel is a 3D point. Then it determines the % difference as the avg distance over the max distance.

In my tests this works rather well, i have been tweaking it to help the results, but i've got the app running in building and displaying a list of duplicates. going to be another couple days of programming to make the interface nicer for actually selecting duplicates to delete. I also plan on letting you specify that some are not duplicates.

All of this data will be stored in a database as it processes so if it crashes or you kill the app, it can pick up where it left off. This also means scanning for duplicates will only have to create the "fingerprints" for each image once, which is by far the most expensive operation.

On my set of 20,000 images it took it 5 hours to build the fingerprints, and 1 hour to build the list of duplicate groups (which was about 1000 files, i hadn't scanned in quite a while).
>> Tiver
For current free software out there though, i've used "Dup Detector" and "DuMP3". both have had issues as my collection has grown larger. When mine's at a good point i plan on putting it out there, likely going to called ImageDiff unless I can think of a better name.
>> ficken
This would be a very essential software to any chan lurker.
>> Anonymous
>>491007

Where will you be announcing this software? You got a site or something?
>> Anonymous
>>490726
There's also a windows build of GQView at http://gqview-win.sourceforge.net/

XnView http://www.xnview.com/ also has a dupe finder buit the interface is a little odd.
>> Anonymous
>>490949
>Divide the image into quarters

Cow2beef.exe?
>> Anonymous
http://www.al-soft.com/dfd/duplicate-file-finder.shtml

al-soft's good stuff
>> Anonymous
     File :-(, x)
Just used VisiPics to move 29 images. Ill post the ecchi ones here(some are loli and peeing)


ehhhh...only looks like I can post 2 of them. Oh well.
>> Anonymous
     File :-(, x)
>> Anonymous
For Linux, Mac, Cygwin, etc:
fdupes - Finds duplicate files by hash
findimagedupes - Finds similar images

>>491007See findimagedupes, 98% accurate