James Fan
James Fan is an AI researcher whose work spans question answering, game-playing systems, speech analysis and LLM-agent security. After earning a PhD at UT Austin, he coauthored IBM's DeepQA overview and Watson strategy research; his later work includes a self-play poker system and speech-sentiment research. The IBM Watson team received the 2013 AAAI Feigenbaum Prize, and Fan went on to work on AI-powered customer support and co-found Tomato.ai, whose voice filter softens offshore call-center agents' accents. As of October 2026, he researches how to make LLM agents safe to deploy.
Timeline
-
2009
The Loose-Speak Interpreter
In 2009, he coauthors “Automatic interpretation of loosely encoded input” with Ken Barker and Bruce Porter. Its Loose-Speak Interpreter reaches 98% average precision and 90% average recall in empirical studies.
-
2011
Watson and DeepQA
In 2011, he is a member of IBM's Watson team and holds a PhD from The University of Texas at Austin. He coauthors the DeepQA overview, which describes Watson as a real-time Jeopardy! contestant built to compete at the human-champion level.
-
2013
A prize for IBM Watson
The IBM Watson team receives the 2013 AAAI Feigenbaum Prize for showing that a synthesis of AI techniques can reach human-level real-time factual question answering; Fan is named among its members. He coauthors a 2013 study that analyzes Watson's Jeopardy! strategy through simulation, learning and Monte Carlo methods.
-
2016
Poker-CNN learns through self-play
At Columbia University, he coauthors Poker-CNN in 2016, a poker system that learns through iterative self-play and improves to compete against human experts.
-
2017
HelloVERA in AI Nexus Lab
In 2017, HelloVERA joins NYU and ffVC's AI Nexus Lab cohort as an AI-powered customer-support startup. Fan and LiangLiang Cao point to the lab's presentation guidance.
-
2020
Speech sentiment analysis
In 2020, he coauthors a Google-affiliated study on pre-trained end-to-end ASR features for speech-sentiment analysis and a Switchboard speech-sentiment corpus. The corpus adds three human annotators' sentiment labels to each transcript segment and contains 49,500 labeled segments across 140 hours of audio.
-
March 2024
Co-founding Tomato.ai
In March 2024, he is Tomato.ai's co-founder and CTO. Its voice filter softens offshore call-center agents' accents as they speak, and Ofer Ronen credits Fan with the seed idea of using a French accent to sell wine.
-
As of October 2026
LLM agent security research
In July 2026, his LangGraph Agent Security series examines software-dependency and supply-chain risks. As of October 2026, he researches how to make LLM agents safe to deploy, including risks that arise when agents can use real tools; his earlier roles include leading Google Cloud's Speech Group and teaching at Columbia University.
Career
- From 2024Tomato.aiCo-founder, CTO
- 2019–2020GoogleGroup Lead
- 2019–2022Naval Postgraduate School
- 2012–2017Pennsylvania State University
- 2011–2014IBMTeam Member
- Columbia UniversityTeacher
Education
Most-cited works
- Building Watson: An Overview of the DeepQA ProjectWith Adam Lally, Aditya Kalyanpur, Chris Welty, David Ferrucci and David Gondek
- Efficient Clustering of Very Large Document CollectionsWith Inderjit S. Dhillon
- Fluorescence Excitation Emission Matrices of Human Tissue: A System for in vivo Measurement and Method of Data Analysis
- Project Halo: Towards a Digital AristotleWith Michael Witbrock and Steffen Staab
- Speech Sentiment Analysis via Pre-Trained Features from End-to-End ASR Models
Sources and credits
Photo: We have no published photo of James Fan yet. Send us a free-licensed photo.
- Automatic interpretation of loosely encoded input - IBM Research
- Three cheers for Watson - UT News
- Building watson: An overview of the deepQA project for AI Magazine - IBM Research
- AAAI Feigenbaum Prize - AAAI
- Analysis of Watson’s strategies for playing Jeopardy! for JAIR - IBM Research
- Poker-CNN: A Pattern Learning Strategy for Making Draws and Bets in Poker Games Using Convolutional Networks - Proceedings of the AAAI Conference on Artificial Intelligence
- ffVC and NYU are mixing researchers with investors in the name of machine intelligence - TechCrunch
- IEEE ICASSP 2020 || Barcelona, Spain || 4-8 May 2020
- A Large Scale Speech Sentiment Corpus - ACL Anthology
- Ofer Ronen, Co-Founder and CEO of Tomato.ai – Interview Series – Unite.AI
- James Fan — AI Agents, LLMs, GenAI & Security Research
- 17. Dependency and Supply Chain Security: The Threat Hiding in Your Requirements File — James Fan



