21
May
image

Unless you've been living under a rock, you are probably aware of the recent improvements and updates to the Google+ experience, both on the web and in mobile apps. While Auto Awesome, Auto Enhance, Auto Highlight, Auto Backup, and other widely discussed features are certainly exciting, one subtle nicety managed to fly under our radar until a post by Google's +Tor Norbye pointed out just how awesome it is.

Visual Recognition

The feature I'm talking about is visual recognition in Google+ photo search.

Remember when +Vic Gundotra mentioned during the I/O 2013 keynote that Google+ will now attempt to guess what you're talking about and auto-tag posts based on, among other things, attached pictures? Turns out auto-tagging posts isn't the only way this visual analysis has surfaced, as there's now an option to Search photos under Photos in Google+ on the web and in the Android app:

Screenshot_2013-05-21-14-00-58 Screenshot_2013-05-21-14-01-44 Screenshot_2013-05-21-13-45-57

Left: Photos in the old Google+ v3.5; Middle, right: Photos in the new Google+ v4.0

Examples

Now let's see just how well it does, shall we? I have 642 Google+ photos by my quick calculations (I had to dig into Chrome Dev Tools to find this count - why not make this number a little more obvious, Google?), so that's a decent amount to produce meaningful results. Additionally, you can search for photos shared by people in your circles.

"jewelry"

100% accuracy here. It didn't pick up on all of my jewelry photos, but the ones it did pick up on are exactly right.

Screenshot_2013-05-21-13-46-56 Screenshot_2013-05-21-13-46-42
image

"food"

Yum! Excellent results that show just how versatile and adaptive Google's algorithms are. Pretty amazing.

Screenshot_2013-05-21-13-46-07 image

"barcelona"

Hey, those are photos from my MWC trip to Barcelona. Not all, but definitely spot on.

image 

"snake"

This is +Tor Norbye's example and is most definitely the one to rule them all. The results speak for themselves:

snakes

Tor is blown away just like we are:

I'm blown away by the new photo search in Google+ where it's recognizing subjects in my own photos -- here the various times I've run into snakes while hiking.

It's unbelievable how the state of the art in computer vision has progressed recently. And I can't imagine the amount of number crunching that must have gone into all this picture processing!

Now, photos search is not perfect yet, and for a lot of my queries it produced no results at all, but when it did work, it was pretty great. I'm used to the idea of visual recognition in search in general - it's not a new topic by far, but I certainly never had it apply on such a personal level, to the photos I took - photos that are private to me and no search engine (other than Google) or person has ever seen.

Your Turn

Is anyone else as impressed as I am? Share your own results with us by uploading screenshots of searches to Imgur, the newly re-launched Flickr, or any photo sharing site of your choice and dropping the links in the comments below.

Blown Away thumbnail credit: Greg Johnson

Artem Russakovskii
Artem is a die-hard Android fan, passionate tech blogger, obsessive-compulsive editor, bug hunting programmer, and the founder of Android Police.
Most of the time, you will find Artem either hacking away at code or thinking of the next 15 blog posts.

  • Jose Luis Cob

    Its awesome how it works also with people (the tags are a help I know, but I dont have tags in all my pics).

  • Michael Lee

    I take tons of photos of my cat. It found every last one of them.

    There was also some random stuffed sheep I took a picture of that it found, but hey, it's pretty dang accurate!

  • http://www.facebook.com/leonardob0880 Leonardo Baez

    its strange, it only works in english

    i am spanish speaker, and it dont recognize any of the searchs in spanish (like perro o gato), but it recognizes english entries (dog or cat).

    • http://www.facebook.com/leonardob0880 Leonardo Baez

      screens

      • Florian Buchner

        cat works great.

        • blueseeker

          That's a nice cat!

        • http://www.facebook.com/leonardob0880 Leonardo Baez

          lol. nice chinchilla

        • ABS

          This rodent hopes to be a cat someday. So the search is even better: it takes into account the current state of things AND the possible future.

        • Torben Brams

          Love this one - At least I am not alone. I searched for "dog", which I found, but also horses and even Taekwondo competitors in a breakthrough contest.

          http://brams.dk/blog/2013/08/18/searching-pictures-in-google/

  • Jays2Kings

    It worked for food, I have a lot of family pics, but I typed i family and it got one (which is not bad, but very weird, since I only have 2 family members in circles and neither where in the pic). Also snow works well too.

  • Raoul

    Doesn't seem to work with french keyword, but "dog" showed my dog's pictures ... and a photo a chewbakka :)

  • WinSuk

    http://gyazo.com/988edec9a307436753d7a84d6523af44

    Not all of them I have, but still cool :)

  • Rolf Karner

    Doesn't work for me, finds absolutely nothing. English only?

    • http://www.androidpolice.com/ Artem Russakovskii

      That'd be my guess.

  • Brandon Duckett

    I typed in cake. It found one of my latest birthday, one of my nephew's last year. and one of a car, BUT STILL. neither were tagged. incredible.

    • Brandon Duckett

      ok, wow. tried "cat" and it returned 11 photos. all of my cat. even a small side face view. impressed.

      • Brandon Duckett

        Also, just to note, I have never tagged the contents of the photos that were scanned, nor posted them where comments could help identify them. Either Google be spying on my photos, or they've got some impressive recognition software in the works.

    • Matthew Fry

      lol. Cake... car pretty close.

    • Preveen Rodrigo

      I typed cake and it found a whole load of cake pics. And a picture of a guys bleeding foot after running barefoot. And a guy playing guitar at a metal concert. So not 100%, but pretty darned impressive.

      • squiddy20

        For the guy playing guitar at a metal concert, did he bear any resemblance to any of the members of CAKE (the band), and that's why it popped up? :D

  • Davy De Waele

    Was impressed with my car search ... Even pictures of the interieur were found... very impressive. https://plus.google.com/112215288642007559493/posts/dP9RW2tTKyo

    • http://www.androidpolice.com/ Artem Russakovskii

      That is pretty impressive indeed.

  • http://profiles.google.com/marcusleejh Marcus Lee

    You have some blingtastic taste in jewelry...

  • http://twitter.com/cthonctic Cthonctic

    Why do you have so many pics of jewelry? O.o

    • http://www.androidpolice.com/ Artem Russakovskii

      I'm guessing you're not married.

      Edit: The screenshot on the right shows my circles' pictures, not mine.

      • http://twitter.com/cthonctic Cthonctic

        I am, but I still don't take pics of jewelry. Then again, I also never take pics of my food... ;P

        • http://www.androidpolice.com/ Artem Russakovskii

          I take pics of jewelry when I/we shop for it, to compare later and decide which one we want.

  • Bradley Ruiz

    it works with colors too, tried blue and red and got pics of my kid wearing these colors awesome

  • daas88

    Dog search: http://i.imgur.com/fYFgk4e.png
    Mérida (a province in my country) search: http://i.imgur.com/TLLeEeU.png
    It's ok, although there's a cat in the dog results hehe. Looking for contact names also worked, but I'm not showing the results.

    • Cerberus_tm

      Great results! By the way, I believe automated face recognition on people's pictures without asking permission is illegal in the European Union (Facebook was forced to turn it off)...I wonder whether the part where it recognises your contacts is legal here. At any rate, it's pretty cool, at least the non-people objects.

      • Carlos

        It just recognises and suggests you people to tag. It doesnt auto-tag, so I don't think is that bad...

      • daas88

        For me it didn't automatically tagged people, it showed me pictures of people I had already tagged, or pictures that I had shared to (or received from) those people on the hangouts app, even if there's no face in there.
        So nothing illegal, I think.

        • Cerberus_tm

          Ohh I see, that's much better. So what happened was, you tagged "John" in one picture, and then it automatically tagged "John" in other pictures; but it never tagged "Mary", because you hadn't tagged her, even though her picture was in your contact list?

          • daas88

            It only shows tagged pictures, or pics you have send/received to/from someone (via hangouts app or g+ post) or if their names are in the picture or folder name.
            I guess they could make better predictions but they seem to really respect privacy, so that's a +1 from me.

          • Cerberus_tm

            Ah OK, I understand. And I agree.

    • Dwayne Wilkinson

      Here's my Dog search... three of my sad little powderpuff and one of ... a horse guard in London???

      • http://www.facebook.com/leonardob0880 Leonardo Baez

        how do you insert the pic in the post

        • Dwayne Wilkinson

          You click the little image icon on the left of the comment box :-)

          • http://www.facebook.com/leonardob0880 Leonardo Baez

            thanks

      • porter86

        That's a big dog!

      • Simon

        No doubt the guard in the photo's brother used to call the horse "thunder cat" when it was small... Google AI wins again ;)

  • Joe Pas

    You can also do this in Google Drive, if you've got a bunch of images stored there. It's pretty fantastic.

  • IrishSid

    I don't understand how to do this or what it's meant to be doing?

    • http://www.androidpolice.com/ Artem Russakovskii

      Umm. Really?

  • http://riteshtripathy.wordpress.com/ Ritesh

    I'll admit it - this one left me GOBSMACKED..!!! :O

  • Stefan Hofbauer

    A search for "market" worked very well...it came up with lots of pictures of markets or at least stores I have been to. This feature is just impressive... Google just dominates data processing!

  • Guest

    Crazy, I tried searching for "dog", "cat", and "forest" and each returned incredibly accurate results. I'm blown away.

  • http://blog.k12.com/ Scott Holm

    Crazy, I tried searching for "dog", "mountain", and "forest" and each returned incredibly accurate results. I'm blown away.

  • arturodelac

    Seraching for concert really blew me away

  • Dave Gilbert

    Most impressive piece of AI I've ever seen.

    I can imagine that this is perhaps going to be a real boon for 'Glass' - being able to say something like 'take a picture of the dog'.
    Now, where's Street view search?

  • Raymond Berger

    Damnnn search for 'game' or 'food' or 'phone' or any location (I think it works by the geotaggs)
    http://i.imgur.com/9nFRMCy.jpg

    http://i.imgur.com/z5Xsw0y.png

  • Rehan Ahmed

    That's it, I am finally switching from Dropbox!

  • Cerberus_tm

    I still wonder why they didn't advertise this, because it is way cooler than anything Google has produced in years, like Google Now or whatever (which are mainly about trying to get you to spend money in/on shops, restaurants, flights, sports matches, music, etc.). Could it be because the face recognition part is possibly illegal in the European Union, where Facebook was reprimanded for applying automated face recognition without asking permission? Either way, the recognition of pets and objects is 100 % legal everywhere, yay!

    • porter86

      They wouldn't be stupid enough to do something illegal like that. I don't think this is tagging people so it's not the case. It recognises pictures of people but it's not putting a name to their face (unless a person has tagged them).

  • Guest

    I'm ultimately surprised! Searching for Australia not only show me photos of the famous buildings such as QVB and the Opera House, but also my time with my sis and her kids in the suburbs (without any significant landmarks)!

    http://imgur.com/daD2DMy

    • tumtumtum

      It possibly used your photo's exif gps tags when you searched for Australia

  • http://twitter.com/jusuchin85 Justin Alex

    *double post below as well. My bad!* I'm ultimately surprised! I've searched for "australia" and not only it return results of my recent trip there with the Opera House, QVB and Three Sisters; but it also returns photos of me playing with my sister's kids in suburban Melbourne (with no significant landmarks!)

    http://imgur.com/daD2DMy

    Amazing!

    • selonmoi

      Are you photos geotagged? If I was implementing photo search, I would be certain to consult location information when look for a place.

      • http://twitter.com/jusuchin85 Justin Alex

        They are not geotagged, but maybe one other factor: The album name; since it's named "[2005] Melbourne"

        • Rirath

          I did a search for "[wife's name] in Hawaii" and it returned some pics from our honeymoon, even though they weren't geotagged. The album's name is "maui" - so apparently it not only reads the album name, but intelligently can translate "maui" into "hawaii" and "Melbourne" into "Australia"

  • Sootie

    That is crazy, finds both motor and push bikes straight from my google instant uploads

    http://i.imgur.com/LuvWIR8.jpg

    You can even search for desert, thats crazy

  • ksavai

    This is amazing. Highly impressed. Another reason to autoupload photo to G+

  • Ivaylo Stoyanov

    It even recognizes photography techniques. I'm amazed!

    • theap

      Maybe it also pulls in data from the EXIF tags too.

  • AHMed SAMir

    you can also search for autoawesome actions, like "HDR" "motion" "pano" macro and more ...

  • https://www.facebook.com/iSpyHI SpyHI

    So, this really impressed me. With dogs and cats, I figured maybe there was some metadata reading going on, but as I searched for more abstract things this came up. It looks like the ocean (in fact, even I had to do a double-take), but it's actually old lava from a hike I did on the big island. It wasn't tagged as the ocean or the beach, because it *isn't* the ocean or the beach...but a computer can tell it's visually similar. Simply incredible.

    This article came out on Wired the other day...I can't help but think it has something to do with this. http://www.wired.com/wiredenterprise/2013/05/hinton/

    Here's the imgur link to my screenshot, for closer inspection: http://imgur.com/JXmgPCc

  • René

    Wow - it works in German :-)

  • Ivan Petkovic

    Impressive. I searched "ocean" and found 0 pics. Then I searched "sea" and found a lots - all of them of Black Sea, where I work. Geotagged they were.

  • Dawid Kania

    Simply astonishing. Searched for food, see, car, bridge, Munich, stadium - all correct. Searching for animals also brought only perfect results. My mind has just melted!

  • jurrabi

    It would be more amazing if it got "comida" as well as "food" taking into account that my profile default language is Spanish.

    • vintermann

      It got "mat" for me (Norwegian). All the 50+ images were of food, except one, which was of a literal (English) mat.

      But it seems to acquire this low rate of false positives by being careful: It found no pictures of food in my own albums, even though there are several.

  • http://profiles.google.com/chopster179 Sam Brown

    I searched for dog and although it brought up pictures of my cat too, it brought up videos containing a dog! I was really impressed!

  • BrainOfSweden

    I'm really impressed. It has some flaws, and it doesn't seem to find everything, but overall, works like magic.

  • Edd

    That means one day video analytic can also read lips. Imagine the application!

  • donrhummy

    The photos it did work for - did you post text with those photos that used language similar to what you used to search for? The ones it didn't find, were those posted without that language?

  • http://thezargon.org Fernando Fonseca

    A search for beer: http://imgur.com/VK9Mv9D

  • Alan Kinech

    I uploaded a picture of my guitar on the social network and it got auto-tagged.

  • Altavoz Distribution

    Noticed the picture in the article resembled part of the cover of a book we're distributing in North American called Blown Away.....BTW this was done via eyeball recognition http://altavoz.com/new-release-blownaway.php

  • Mohammad Khatib

    Tried 'Water' and I got amazing results. Not only it recognized photos but also videos that contained water falls, rivers, sea Aaand water bottles! Check it out here: https://plus.google.com/u/0/113296553558440750483/posts/TEQDt85QjPk.

  • someguy

    I searched for beard. It showed me pictures with beards in!

  • Jenny Spaghetti

    I did a search for Seattle and it gave me a bunch of pictures of sushi. Maybe it knows me too well? >_>

  • Lori Carey

    I'm not convinced this is a visual search. My search results yielded many images that had nothing to do with the search term but the text of my post or the image caption contained the word in another context. For example searching for "beach" shows me images taken in Laguna Beach that are not images OF the beach. A search for "orange' brought up black and white images taken in Orange County. I'm sure it is reading text, whether in the post itself, the photo caption, or the EXIF data.

  • http://www.subprint.com joemccann

    Yup Google ahead of the curve. Wrote about how badly Instagram needs this for brands.

    http://subprint.com/blog/instagram-is-visual-twitter

  • Satya

    ¨Share your own results with us by uploading screenshots of searches to Imgur, the newly re-launched Flickr, or any photo sharing site of your choice¨

    This is Google+´s problem right here. People talk about how awesome it is and proceed to share things with other social services.

  • http://www.facebook.com/profile.php?id=100001386596236 Tha Watcher

    Awesome profile photo, Artem!

  • Dennis Elliott

    Wow, I had no idea I could do this. I did search for "Grass" and this is just a portion of what came up.

    http://www.flickr.com/photos/86312040@N05/8802354101/

    I had not labeled or tagged these photos with this term. I also tried "Trees" and got nothing, but did the singular "Tree" and got dozens of photos.

  • Achal Shah

    There's also a startup that is working to solve the same problem; identification of objects in images: http://www.dextro.co

  • http://www.feirox.com/ ankHD

    I gave a search to find me photos of "male" or "female", but google's algos literally failed.

  • Charles Teague

    What if i have a picture of something that i do not know what it is? Is there a search engine that will recognize an image and search for details and info on what is in the image?

  • Bill Tribble

    " ... photos that are private to me and no search engine (other than Google) or person has ever seen." #NSA

  • Sam Howard

    ok, dumb question, but has anyone figured out how to search by filtered FILE NAME of the image??? I get duplicates uploaded by auto-backup and want to remove all of the *.jpg images. Finds NOTHING when I search for *.jpg:

    No photos found for "#AutoBackup jpg"

    Crazy...