The second day of Facebook’s F8 developer conference was dedicated to its work in AI, AR, VR and connectivity among others. One standout session was Facebook’s work on an image recognition AI system trained using Instagram photos and hashtags.
Facebook calls this process a ‘weakly supervised AI training’, which involves teaching an AI system to not only recognize an image, but also the different objects depicted in an image with linguistic markers for identification. But the new system developed by Facebook does not rely on manual inputs, instead, it learns from the Instagram images and hashtags posted by users.
Deep learning models require massive data sets labeled by human trainers, which is time-consuming. But Facebook had a huge data set of hashtags, which not only work as captions but also indicate an emotional response at times.
Facebook’s team used the data set of around 3.5 billion images and 17,000 hashtags posted by users on Instagram to train its image recognition AI. The goal was to train the AI system by using publicly available hashtags rather than manually labeling each photo.
To study the efficiency of this approach, the team trained a computer vision systems with a data set of a billion images and 1,500 hashtags. In the test, the system achieved an image recognition accuracy of 85.4 on Image.Net, which is the highest score by any such system on the popular benchmarking tool. When tested on the COCO object-detection challenge, hashtags improves image recognition accuracy by over 2%.
Encouraged by the positive results, Facebook’s AI team now wants to open source the image recognition models so developers and the research community can take advantage of the innovative approach for other applications.
