//
you're reading...
Technology

Facebook AI can identify objects in your photos

Here is a photo of a man with dogs in his pockets walking a duck that is wearing shoes. I’m putting it here to confuse Lumos.

Initially used to improve the experience for visually impaired members of the Facebook community, the company’s Lumos computer vision platform is now powering image content search for all users. This means you can now search for images on Facebook with key words that describe the contents of a photo, rather than being limited by tags and captions.

To accomplish the task, Facebook trained an ever-fashionable deep neural network on tens of millions of photos. Facebook’s fortunate in this respect because its platform is already host to billions of captioned images. The model essentially matches search descriptors to features pulled from photos with some degree of probability.

After matching terms to images, the model ranks its output using information from both the images and the original search. Facebook also added in weights to prioritize diversity in photo results so you don’t end up with 50 pics of the same thing with small changes in zoom and angle. In practice, all of this should produce more satisfying and relevant results.

Eventually Facebook will apply this technology to its growing video corpus. This could be used both in the personal context of searching a friend’s video to find the exact moment she blew out the candles on her birthday cake, or in a commercial context. The later could help raise the ceiling on Facebook’s potential ad revenue from News Feed.

… Along with today’s new image content search feature, Facebook is updating its original Automatic Alternative Text tool. When Facebook released the tool last April, visually impaired users could leverage existing text-to-speech tools to understand the contents of photos for the first time. The system could tell you that a photo involved a stage and lights, but it wasn’t very good at relating actions to objects.

A Facebook team fixed that problem by painstakingly labeling 130,000 photos pulled from the platform. The company was able to train a computer vision model to identify actions happening in photos. Now you might now hear “people dancing on stage,” a much better, contextualized, description.

The applied computer vision race

Facebook isn’t the only one racing to apply recent computer vision advances to existing products. Pinterest’s visual search feature has been continuously improved to let users search images by the objects within them. This makes photos interactive and more importantly it makes them commercializable.

Google on the other hand open sourced its own image captioning model last fall that can both identify objects and classify actions with accuracy over 90 percent. The open source activity around TensorFlow has helped the framework gain prominence and become very popular with machine learning developers.

Facebook is focused on making machine learning easy for teams across the company to integrate into their projects. This means improving the use of the company’s general purpose FBLearner Flow.

“We’re currently running 1.2 million AI experiments per month on FBLearner Flow, which is six times greater than what we were running a year ago,” said Joaquin Quiñonero Candela, Facebook’s director of applied machine learning.

Lumos was built on top of FBLearner Flow. It has already been used for over 200 visual models. Aside from image content search, engineers have used the tool for fighting spam.

Link

About Xeno

E pluribus unum.

Discussion

No comments yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog Stats

  • 9,297,668 hits

join 82,408 other followers.

Blogs I Like

Suggest Your Site

Hello! Email me to recommend your Wordpress.com site. If I dig it, it may show up above. I add and remove new sites in rotation, so yours may appear and disappear at times.
My Life as an Artist (2)

Smile! You’re at the best WordPress.com site ever

The Wine Raconteur

A great WordPress.com site

Little Fears

Tales of humour, whimsy and courgettes

The Introspective Scribbler

Trivial views on significant issues

soulgifts - Telling Tales

Creating magic with words

Tea and Tales

Let's talk about Tea and tell some Tales!

Teacher as Transformer

Education, Leadership, Life, and Transformation

Zoolon Hub

Guitarist / Songwriter / Blogger

WordPress.com

WordPress.com is the best place for your personal blog or business site.