Memes are undoubtedly an indispensable part of social media. Unfortunately, people with visual impairments are missing out on memes as these images usually don’t include alternative text to be picked by screen reader software. Researchers at Carnegie Mellon University (CMU) have developed a technique for automatically identifying and adding alternative text to memes by applying prewritten templates.
“Memes may not seem like the most important problem, but a vital part of accessibility is not choosing for people what deserves their attention. Many people use memes, and so they should be made accessible.”, says Jeff Bigham, associate professor in the HCII.
For those wondering, the researchers make use of computer vision techniques to describe the image and optical character recognition to identify the text present on top of the image.
However, this approach might not be fruitful in all scenarios. Hence, they have attempted to create a platform to convert memes into sound accessible through a sound library. “One of the reasons we tried the audio memes was because we thought alt text would kill the joke, but people still preferred the text because they’re so used to it,” notes Cole Gleason, a Ph.D. student in CMU’s Human-Computer Interaction Institute (HCII).
Twitter does have an option to add alt-text but the interface is not straightforward and hence, people tend to skip it. The researchers note that only 0.1 percent of one million images present on Twitter had alt text.
As the researchers say, it depends on the meme template for the humor to reflect on the generated alt text. Nonetheless, it is better to have a somewhat relevant text rather than having no alternate text at all.
