Image matching software (looking for recomendations)

Looking at cartoons, such as Kim Possible, tends to remind me of a project I've had in mind for a while.  Said project would require matching up 2d images.  Cartoons remind me because traditional cartoons are flat.  A pan in real life actually does some complex things because how something is captured by a camera depends on which part of the lens the light is coming through and such.  In a cartoon to pan the picture moves sideways so it's a much simpler process.

Anyway, what I want is software that can go through a lot of images and automatically sort them into categories of ones with overlapping portions.

Weirdly, if I wanted to do this with the much more complex task of photos of three dimensional objects, I'd already know which software to use even though that is much more complex and difficult task.

As an example of what I'm talking about here's how cartoons remind me of this stuff.  All of these are the same picture, they've been cropped, resized, or in one example had an obstruction added, but there's none of the changes that we'd have if they were actual photos of a real life thing:

Scaled down so you can see full original image

Portion at original scale:

Cartoon Pan:

Cartoon Tilt:

Cartoon Pan-Tilt

Obstruction:

They all contain the center of the original picture, which means that they all overlap and thus any two of them (without looking at the others) can be appropriately concluded to have originated with the same 2d image.

But, there are various different scales, various different portions of the original, and so forth.  I don't know how to automate determining that they're versions of the same image and that is what I'd like to do.  I'd like something that can look at every image in a given directory and sort them by putting ones that contain versions of the same image together.

Somewhat oddly, as noted above, I know software that could do that, in a roundabout way, if I were doing a more complex version.  For example if that were a real house and and each of the images was a different photo of it.  If I wanted to do that I could not only get  the fact they were of the same object, I could also get a rough idea of the three dimensional shape of the object (based on the differences between the photos.)

What I want is a good deal simpler and, hopefully, faster.

I can see sort of how one would do it for things at the same scale (because you'd end up with the exact same pixels on the overlap, with the only differences being a result of artifacts of the image format) but when it comes to matching across scales I'm at a bit of a loss.

Anyone have any advice?