ST. PAUL, MN - SEPTEMBER 05: Host Jon Stewart of Comedy Central's "The Daily Show with Jon Stewart" watches a video while taping "The Daily Show with Jon Stewart: Restoring Honor & Dignity to the White House" at the McNally Smith College of Music September 5, 2008 in St. Paul, Minnesota. The show is being taped in St. Paul during the week of the Republican National Convention. (Photo by Ethan Miller/Getty Images for Comedy Central)
Did'ja ever wonder how Jon Stewart manages to find just the right old clip to catch a politician in an inconsistency or lie? The Daily Show uses an expensive technology from SnapStream to search recorded video for actual words spoken. AT&T plans on bringing this contextual video search technology to your living room...sometime (they were a little hazy on the when), but the demo sure was cool.
Dubbed iMiracle, you use a smartphone or tablet (the demo used an iPad) to tell the system exactly what you're looking for, such as "Indonesia volcano eruption" and the system searches through all the programs recorded and stored in the cloud that mentioned this disaster and brings up a list of search results. You can then view the resulting clips on your portable device or transfer them to a wirelessly connected TV. In the demo I got, a dozen or so results sprang up in a few seconds.
According to its AT&T Labs developers, Bernard Renger and Behzad Shahraray, under the iMiracle hood is "a complex mashup of technologies, including video analysis and indexing, Watson speech recognition and language processing… [and] a multimodal speech-enabled user interface."
Using speech recognition, cloud content and your TV won't be restricted to searches of recorded programs. A sister Watson speech recognition project headed by Michael Johnston called iRemote (what's with all the i's? AT&T is still the home of iPhone, of course, and all the demos I saw were conducted either with iPhones and and iPads and were designed using AT&T's U-verse TV service) and VARC — Voice Activated Remote Control — uses an iPhone or iPad to search for future programs. You speak keywords — program name, actors, directors, channel, etc. — or multi-variable phrases such as "comedy movies on HBO tonight" or "all reality shows on ABC Thursday night" and the system finds all relevant future programs for you to DVR.
A third speech recognition/cloud technology led by Srinivas Bangalore called Smart TV does instant live language translation, displayed as closed captioning, sort of like Star Trek's universal translator, and adds instant Wikipedia searches of color highlighted words. An ever-changing array of four words are highlighted on-screen in red, blue, yellow or green as the closed captioning translation/transcription takes place; hitting the correspending word color button on the remote activates the Wikipedia entry.