Link: Notes on an inexpensive and effective "video scraping" technique in which the user feeds a screen recording into Google AI Studio to extract data with Gemini (Simon Willison's Weblog)

I recently needed to sum values from multiple emails and opted to use video scraping on my Gmail using QuickTime and Google’s AI Studio.

I recorded my Gmail screen, navigated through the emails, and uploaded the video to AI Studio which converted the content into a JSON array.

Surprisingly, even without using the intended Gemini 1.5 Pro model, the AI accurately extracted all data.

The cost of using Gemini 1.5 Flash was negligible, totaling less than a cent, and currently, Google AI Studio offers free services.

Considering alternatives like manual copying, programmatic access, or browser automation, video scraping proved efficient and cost-effective.

Additionally, I developed a token pricing calculator with Claude 3.5 Sonnet to streamline such tasks in the future, illustrating its potential for broader applications in fields like data journalism. #

--

Yoooo, this is a quick note on a link that made me go, WTF? Find all past links here.