What happens when AI eats your Web site? The information you've spent years publishing gets reassembled in a new way for people to interact with your content. So far, we've integrated text-based articles and audio files into this experience on Ask LukeW. Today we're adding over 6,000 minutes of video as well.
The Ask LukeW feature is powered by AI models that break down content into related concepts and reassemble it into new answers. When that content is sourced from an article, audio file, and now video file, we cite it and give you the ability to go deeper into the original source file.
In the example above, the information about loading indicators is sourced from two different videos which are linked in the source card to the right. Selecting one of these videos opens an integrated (within the broader conversational UI) video experience that looks like this:
The inline video player starting point is set to where the answer is mostly sourced from. I say "mostly" because a LLM generated answer may pull from multiple points in a video file to create a response, so we start you in the right vicinity of the topic being discussed. Below the video player is an indication of who is speaking that also serves as a scrubber to let you move through the file with context by displaying the current transcript directly underneath.
You can access the full speaker separated transcript and search within it as well. The transcript is generated with an automatic speech recognition AI model and diarized to indicate who is talking at any given time.
For each video file integrated within Ask LukeW, we make use of LLM language operations to:
- summarize the video file
- extract speakers, locations, key topics, and even generate a title from the video file (if we lack the data)
- create follow-on questions to ask based on the content of the video file
- enable people to ask questions using just the content within the video file
- generate answers in response to what people ask
It's like there's AI models everywhere. We even did a bit of machine learning to automate thumbnail selection for the videos. Just generate several thumbnails from each video, train on a few examples, and away you go.
Lastly but not least, we added the transcript search feature to all the audio files in Ask LukeW as well.
For more on how AI continues to eat this Web site, check out these earlier articles or go Ask LukeW about it.
Further Reading
- New Ways into Web Content: how AI enables a different way of interacting with a Web site
- Integrated Audio Experiences & Memory: enabling specific content experiences within a conversational UI
- Expanding Conversational User Interfaces: extending chat user interfaces to better support AI capabilities
- Integrated Video Experiences: adding video-specific experiences within conversational UI
Much thanks to Yangguang Li (front end), Thanh Tran (design), and Sam Pullara (back end) in helping these explorations move forward.