Jim Fan

QuHarrison Terry presents Dr. Jim Fan with the WTF Innovators Award for his pioneering research at NVIDIA AI into Autonomous Agents (AI Agents, or Generalist Agents), which self-learn, refine skills, and adapt to carry out tasks without human intervention.

The WTF Innovators Award recognizes excellence at the precipice of societal change, with the inaugural class focusing on AI innovators. As a memento, each of the 34 awardees are gifted a featured song by QuHarrison Terry and Genesis Renji. We present “Code Word: Dojo”, produced by Nimso, to Dr. Jim Fan.

AI Agents hold immense potential because they eliminate much of the back-and-forth in human-AI interaction. Dr. Jim Fan is unearthing a world where AI can learn and operate independently in any environment, which is why he should be on everyone’s radar. – QuHarrison Terry.

In June 2022, Dr. Jim Fan first published a framework for building AI Agents in the research paper, “MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge.” MineDojo used the open-world video game Minecraft as a training ground for their AI Agent framework. Its knowledge base came from hundreds of thousands of Minecraft YouTube videos, Wiki pages, and Reddit posts. This research won the Outstanding Paper Award at NeurIPS.

In May 2023, Dr. Jim Fan expanded on this research with Voyager, the first GPT-4-powered embodied lifelong learning agent in Minecraft that continuously explores the world, acquires diverse skills, and makes novel discoveries without human intervention.

Ideally, the frameworks Dr. Jim Fan has created will be applied outside of Minecraft, allowing Generalist Agents to operate as effective personal assistants, business partners, advanced research aides, and more.

I admire how openly Dr. Jim Fan shares his research and excitement for AI Agents. His mindset multiplies with every idea shared, thus elevating the entire research into advanced, self-capable AI. – QuHarrison Terry.

Prior to becoming a research scientist at NVIDIA AI, Dr. Jim Fan obtained his Ph.D. degree at Stanford Vision Lab and held research internship positions at NVIDIA, Google Cloud AI, OpenAI, Baidu Silicon Valley AI Lab, and Mila-Quebec AI Institute. During his 2016 internship at OpenAI, he worked on the OpenAI Universe project which was an AI Agent that could act on the OS and native software level.