EXPERIENCE

Carnegie Mellon University 1989 - Present
School of Computer Science

Performed and directed research in the areas of natural language processing, machine learning, and cognitive architecture. Served as Principal Investigator for the Soar project at Carnegie Mellon following Allen Newell’s death. Current research interests: integrating ubiquitous sensing with self-narration to track medical procedures and neural-symbolic approaches to language-based interaction.

Senior Project Scientist, (2019 - present)
Adjunct Professor, (2000 - 2019)
Senior Research Computer Scientist, (1998 - 2000)
Research Computer Scientist, (1991 - 1998)
Core Faculty, Language Technologies Institute (1996 - 2000)
Core Faculty, Computational Linguistics Program (1995 - 1996)
Research Associate, (1989 - 1991)

Disney Research2010 - Present

Language-based character interaction with children and adults. Data-driven/machine learning algorithms for addressee identification, turn taking, responsive listening, and real-time, multimodal interaction with persistent interactive personalities.

Senior Research Scientist, (2014 - 2017)
Consultant, (2010 - 2014), (2018 - 2019), 2023

KidAccess, Inc. 1998 - 2021

Founded and lead a small business that provided products and services for children with developmental disabilities based on cognitive theory.

President, CEO
Director of Product Development

Carnegie Learning, Inc.2007 - 2008

Proof-of-concept implementation of a Curriculum Advisor that helps teachers maintain pedagogical constraints when creating custom curriculum sequences from established tutoring packages.

Consultant, (2007 - 2008)

Carnegie Speech, Inc.2004 - 2007

Proof-of-concept implementation of a listening comprehension tutor for young ESL students that provides continuous individualization of story content based on ongoing assessment of linguistic strengths and weaknesses.

Consultant, (2004 - 2007)

iCarnegie, Inc.2001 - 2004

Analysis of long-term data to pinpoint demographic influences on performance across course offerings and identify curriculum modifications for both U.S. and international campuses.

Consultant, (2001 - 2004)

The Rand Corporation1981 - 1985
Santa Monica, California

Consultant, (1982 - 1985)
Research Programmer, (1981 - 1982)

divider

EDUCATION

Ph.D Computer Science
Carnegie Mellon University

1989

M.S. Computer Science
Carnegie Mellon University

1987

B.S. Computer Science & Psychology
Yale University

1981
Summa cum laude
Distinction in the Major, Computer Science
Distinction in the Major, Psychology

divider

BOOKS

Private I: A story of intrigue, romance, murder, and artificial intelligence.
Private I:
A story of intrigue, romance, murder, and artificial intelligence.
Ashlei E. Watson, Jill Fain Lehman, and Paul Pangaro

Amazon, 2023

Cover of From Goals to Data and Back Again: Adding Backbone to Developmental Intervention for Children with Autism
From Goals to Data and Back Again:
Adding Backbone to Developmental Intervention for Children with Autism
Jill Fain Lehman and Rebecca Klaw

Jessica Kingsley Publishers
London and New York, 2003.

Cover of Adaptive Parsing: Self-Extruding Natural Language Interfaces
Adaptive Parsing:
Self-Extending Natural Language Interfaces
Jill Fain Lehman

Kluwer Academic Publishers
Norwell, MA, 1992.

divider

SELECTED PUBLICATIONS BY AREA

NATURAL LANGUAGE PROCESSING & MACHINE LEARNING

Arakawa, R., Yakura, H., Mollyn, V., Nie. S., Russell, E., Demeo, D., Reddy, H., Maytin, A., Carroll, B., Lehman, J. F., Goel, M., “PrISM-Tracker: A Framework for Multimodal Procedure Tracking using Wearable Sensors and State Transition Information with User-driven Handling of Errors and Uncertainty.” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2023;6(4):156:1-156:27. doi:10.1145/356950

Naik, A., Lehman, J. F., and Rosé, C., “Adapting to the Long Tail: A Meta-Analysis of Transfer Learning Research for Language Understanding Tasks.” Transactions of the Association for Computational Linguistics, Volume 10, August 2022.

Lovelace, J., Newman-Griffis, D., Vashishth, S., Lehman, J. F., Rosé, C.P., “Robust Knowledge Graph Completion with Stacked Convolutions and a Student Re-Ranking Network,” ACL-IJCNLP, 2021.

Newman-Griffis, D., Lehman, J. F., Rosé, C., Hochheiser, H., “Translational NLP: A New Paradigm and General Principles for Natural Language Processing Research,” Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT), 2021.

Green, N. and Lehman, J.F., “An Integrated Discourse Recipe-Based Model for Task-Oriented Dialogue,” Discourse Processes, Volume 33, Number 2, 2002.

Green, N. and Lehman, J. F., “An Application of Explanation-based Learning to Discourse Generation and Interpretation,” AAAI Spring Symposium on Applying Machine Learning to Discourse Processing, 1998.

Green, N. and Lehman, J. F., “Comparing Agent Modeling for Language and Action,” AAAI Workshop on Agent Modeling, 1996.

Green, N. and Lehman, J. F., “Goals for Future Computational Models of Conversational Implicature,” AAAI Spring Symposium on Computational Implicature, 1996.

Lehman, J. Fain, “Meaning Matters: Response to Miller,” in Mind Matters: A Tribute to Allen Newell, D. Steier and T. Mitchell, eds., Lawrence Erlbaum Associates, Hillsdale, NJ, 1996.

Lehman, J. F., Van Dyke, J. and Green, N., “Reactive Natural Language Processing: Comprehension and Generation in the Air Combat Domain,” AAAI Fall Symposium on Embodied Language and Action, 1995.

Lehman, J. F., Nelson, G., and Van Dyke, J., “Representation and Acquisition of Lexical Knowledge,” AAAI Spring Symposium on Representation and Acquisition of Lexical Knowledge, 1995.

Nelson, G., Lehman, J. F., John, B. E., “Experiences in Interruptible Language Processing,” in AAAI Spring Symposium on Active NLP, 1994.

Rubinoff, R. and Lehman, J. F., “Real-time Natural Language Generation in NL-Soar,” in 7th International Workshop on Natural Language Generation, 1994.

Lehman, J. Fain, “Toward the Essential Nature of Statistical Knowledge in Sense Resolution,” in 12th National Conference on Artificial Intelligence, 1994.

Steier, D. M., Lewis, R. L., Lehman, J. Fain, and Zacherl, A. L., “Combining multiple sources of knowledge in an integrated intelligent system,” IEEE Expert, Volume 8, Number 3, 1993.

Lehman, J. Fain, Lewis, R., and Newell, A., “Integrating Knowledge Sources in Language Comprehension,” in 13th Annual Conference of the Cognitive Science Society, 1991.

Lehman, J. Fain, “Supporting Linguistic Consistency and Idiosyncrasy with an Adaptive Interface Design,” in 12th Annual Conference of the Cognitive Science Society, 1990.

Lehman, J. Fain, “Adaptive Parsing: A General Method for Learning Idiosyncratic Grammars,” 6th International Conference on Machine Learning, 1990.

Lehman, J. F., and Carbonell, J. G., “Learning the User's Language: A Step Towards Automated Creation of User Models,” in User Modelling in Dialog Systems, Wahlster, W., and Kobsa, A., eds., Springer-Verlag, 1989.

Minton, S. N., Hayes, P. J., and Fain, J. E., “Controlling Search in Flexible Parsing,” 9th International Joint Conference on Artificial Intelligence (IJCAI), 1985.

Fain, J., Carbonell, J. G., Hayes, P. J., and Minton, S. N., “MULTIPAR: A Robust Entity-Oriented Parser,” 7th Annual Conference of the Cognitive Science Society, 1985.

LANGUAGE-BASED CHARACTER INTERACTION

Paetzel, M., Kennedy, J., Castellano, G., Lehman, J.F., “Incremental Acquisition and Reuse of Multimodal Affective Behaviors in a Conversational Agent.” 6th International Conference on Human-Agent Interaction (HAI), 2018.

Mota, P., Paetzel, M., Fox, A., Amini, A., Srinivasan, S., Kennedy, J., Lehman, J. F., “Expressing Coherent Personality with Incremental Acquisition of Multimodal Behaviors.” IEEE International Symposium on Robot and Human Interact Communication (RO-MAN), 2018.

Campanos, J., Kennedy, J., Lehman, J. F., “Challenges in Exploiting Dialog History with a Robot.” International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2018.

Kennedy, J., Leite, I., Pereira, A., Sun, M., Li, B., Jain, R., Cheng, R., Pincus, E. Carter, E., Lehman, J. F., “Learning and Reusing Dialog for Repeated Interactions with a Situated Social Agent.” 17th International Conference on Intelligent Virtual Agents (IVA), 2017.

Lehman, J. F. and Leite, I., “Turn-taking, Children and the Unpredictability of Fun.” AI Magazine, Volume 37, Number 4, Winter 2017.

Sun, M., Leite, I., Lehman, J. F., Li, B., “Collaborative Storytelling between Robot and Child: A Feasibility Study.” ACM SIGCHI Interaction Design and Children (IDC), 2017.

Leite, I., Pereira, A., Lehman, J. F., “Persistent Memory in Repeated Child-Robot Conversations.” ACM SIGCHI Interaction Design and Children (IDC), 2017.

Sadoughi, N., Pereira, A., Jain, R., Leite, I., and Lehman, J. F., “Creating Prosodic Synchrony for a Robot Co- player in a Speech-controlled Game for Children.” 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2017. Best Paper, Technical Advances Award.

Leite, I., Periera, A., Funkhouser, A., Li, A., and Lehman, J. F., “Semi-situated Learning of Verbal and Nonverbal Content for Repeated Human-Robot Interaction.” 18th ACM International Conference on Multimodal Interaction (ICMI), 2016.

Lehman, J. F., Wolfe, N., and Pereira, A., “Multi-party Language Interaction in a Fast-paced Game Using Multi-keyword Spotting.” 16th International Conference on Intelligent Virtual Agents (IVA), 2016.

Lehman, J. F., Wolfe, N., and Pereira, A., “G-g-go! Juuump! Online Performance of a Multi-keyword Spotter in a Real-time Game.” 5th Workshop on Child Computer Interaction (WOCCI), 2016.

Fringi, E., Lehman, J. F., and Russell, M., “The Role of Phonological Processes and Acoustic Confusability in Phone Errors in Children’s ASR.” 5th Workshop on Child Computer Interaction (WOCCI), 2016.

Chaspari, T. and Lehman, J. F., “An Acoustic Analysis of Child-Child and Child-Robot Interactions for Understanding Engagement during Speech-Controlled Computer Games.” Interspeech, 2016.

Lehman, J. F. and Singh, R., “Estimation of Children’s Physical Characteristics from their Voices.” Interspeech, 2016.

Leite, I. and Lehman, J. F., “The Robot Who Knew Too Much: Toward Understanding the Privacy/Personalization Trade-off in Child-Robot Conversation.” ACM SIGCHI Interaction Design and Children (IDC), 2016.

Chaspari, T., Al Moubayed, S., and Lehman, J. F., “Exploring Children’s Verbal and Acoustic Synchrony: Towards Promoting Engagement in Speech-Controlled Robot-Companion Games.” 17th ACM International Conference on Multimodal Interaction (ICMI), Workshop on Interpersonal Synchrony, 2015.

Al Moubayed, S. and Lehman, J. F., “Toward Better Understanding of Engagement in Multiparty Spoken Interaction with Children.” 17th ACM International Conference on Multimodal Interaction (ICMI), 2015.

Al Moubayed, S. and Lehman, J., “Regulating Turn-taking in Multi-child Spoken Interaction.” Interactive Virtual Agents (IVA), 2015.

Sundar, H., Lehman, J. F., and Singh, R., “Keyword spotting in multi-player voice driven games for children.” Interspeech, 2015.

Fringi, E., Lehman, J. F., and Russell, M., “Evidence of Phonological Processes in Automatic Recognition of Children’s Speech.” Interspeech, 2015.

Lehman, J. F. and Al Moubayed, S., “Mole Madness—A Multi-Child, Fast-Paced, Speech-Controlled Game.” AAAI Spring Symposium on Turn-taking and Coordination in Human-Machine Interaction, 2015.

Baljekar, P., Lehman, J. F., and Singh, R., “Online Word-spotting in Continuous Speech with Recurrent Neural Networks.” IEEE Spoken Language Technology Workshop (SLT), 2014.

Lehman, J. F., “Robo Fashion World: A Multimodal Corpus of Multi-child Human-Computer Interaction.” 16th ACM International Conference on Multimodal Interaction (ICMI), Workshop on Understanding and Modeling Multiparty, Multimodal Interactions, 2014.

Leite, I., Hajishirzi, H., Andrist, S., and Lehman, J. F., “Managing Chaos: Models of Turn-taking in Character- multichild Interactions.” 15th ACM International Conference on Multimodal Interaction (ICMI), 2013.

Andrist, S., Leite, I., and Lehman, J. F., “Fun and Fair: Influencing Turn-taking in a Multi-party Game with a Virtual Agent.” Interaction Design and Children (IDC), 2013.

Hajishirzi, H., Lehman, J. F., and Hodgins, J., “Using Group History to Identify Character-directed Utterances in Multi-child Interactions,” 13th Annual SIGdial Meeting on Discourse and Dialog, 2012. Best Paper Award.

Language & Cognition

Green, N. and Lehman, J. F., “The Role of a Cognitive Agent Architecture in the Design of an Integrated Discourse Processing Model,” 6th International Workshop on Agent Theories, Architectures, and Languages (ATAL-99), 1999.

Lehman, J. F., Laird, J. E., and Rosenbloom, P. S., “A Gentle Introduction to Soar, an Architecture for Human Cognition,” in Invitation to Cognitive Science, Volume 4, S. Sternberg and D. Scarborough, eds., MIT Press, Cambridge, Mass., 1998.

Lehman, J. Fain, Newell, A. N., and Lewis, R. L., “Architectural Influences on Language Comprehension,” in Constraining Cognitive Theories: Issues and Options, Pylyshyn, Z., ed., Ablex Publishing Corporation, 1998.

Van Dyke, J. and Lehman, J. F., “A Process Model of Learning Definiteness in a Foreign Language,” 19th Annual Conference of the Cognitive Science Society, 1997.

Nelson, G., Lehman, J. F., John, B., “Integrating Cognitive Capabilities in a Real-Time Task,” in 16th Annual Conference of the Cognitive Science Society, 1994.

Rosenbloom, P. S., Lehman, J. F., and Laird, J. E., “Overview of Soar as a unified theory of cognition: Spring 1993,” 15th Annual Conference of the Cognitive Science Society, 1993.

Lehman, J. Fain, Newell, A. N., Polk, T., and Lewis, R. L., “The Role of Language in Cognition,” in Conceptions of the Human Mind, Harman, G., ed., Lawrence Erlbaum Associates, Inc., 1993.

Lewis, R. L., Huffman, S. B., John, B. E., Laird, J. E., Lehman, J. Fain, Newell, A., Rosenbloom, P. S., Simon T., and Tessler, S. G., “Soar as a Unified Theory of Cognition: Spring 1990,” 12th Annual Conference of the Cognitive Science Society, 1990.

EDUCATION

Fringi, E., Lehman, J. F., and Russell, M., “Analysis of Phone Errors in Computer Recognition of Children’s Speech.” Workshop on Speech and Language Technology for Education (SLaTE), 2015.

Lehman, J. F. and Fisher, A., “Dynamic Curriculum Delivery,” Community College Journal, April/May 2004.

Miller, C., Lehman J. F., and Koedinger, K., “Goals and Learning in Microworlds,” Cognitive Science, Volume 23, Number 3, 1999.

Conati C., and Lehman, J. Fain, “EFH-Soar: Modeling Education in Highly Interactive Environments,” Advances in Artificial Intelligence, Lecture Notes in Artificial Intelligence, P. Torasso, ed., LNCS 728, Springer-Verlag, 1993.

Conati C. and Lehman, J. Fain, “Toward a Model of Student Education in Microworlds,” 15th Annual Conference of the Cognitive Science Society, 1993.

Lehman, J. Fain, “Three Uses of Adaptive Parsing in Intelligent Tutoring,” in Cognitive Modelling and Interactive Environments in Language Learning, Engel, F. L, Bouwhuis, D. G., Bosser, T., and d'Ydewalle, G., eds., NATO ASI Series, Springer-Verlag, 1992.

AUTISM

Carter, E. J., Williams, D. L., Hodgins, J. K., and Lehman J. F., “Are Children with Autism More Responsive to Animated Characters?” Journal of Autism and Developmental Disorders 05/2014; DOI:10.1007/s10803- 014-2116-8.

Carter, E.J., Williams, D.L., Minshew, N.J., and Lehman, J.F., “Is He Being Bad? Social and Language Brain Networks during Social Judgment in Children with Autism,” PLoS ONE 7(10): e47241. doi:10.1371/journal.pone.0047241.

Williams, D. L., Carter, E. J., and Lehman, J. F., “Visual Strategies to Remember Verbal Information: fMRI Study of Autism,” American Speech-Language-Hearing Association (ASHA), 2012.

Williams, D. L., Carter, E. J., Lehman, J. F., and Minshew, N. J., “Making Tough Decisions: The Neural Correlates of Categorization in Children with and without Autism,” International Meeting for Autism Research (IMFAR), 2012.

Carter, E. J., Williams, D. L., Lehman, J. F., and Minshew, N. J., “Using Visual Strategies to Remember Verbal Information: an fMRI Study of Working Memory in Children with and without Autism,” International Meeting for Autism Research (IMFAR), 2012.

Carter, E. J., Williams, D. L., Lehman, J. F., and Minshew, N. J., “Functional Neuroimaging During Social Judgment in Children with Autism,” American Speech-Language-Hearing Association, 2011.

Williams, D. L., Carter, E. J., Lehman, J. F., and Minshew, N. J., “Neuroimaging of Inferential and Theory of Mind Processing in Children with Autism,” American Speech-Language-Hearing Association (ASHA), 2011.

Lehman, J. F., “Hidden Assumptions in the Mainstream Curriculum: K-3 Reading & Writing, Autism Society’s 42nd National Conference on Autism Spectrum Disorders, 2011.

Carter, E. J., Williams, D. L., Lehman, J. F., and Minshew, N. J., “Is He Being Bad?: Brain Activation During Social Judgment in Children with Autism,” International Meeting for Autism Research (IMFAR), 2011.

Williams, D. L., Carter, E. J., Lehman, J. F., and Minshew, N. J., “Brain Activation During Inferential and Theory of Mind Processing in Children with Autism,” International Meeting for Autism Research (IMFAR), 2011.

Lehman, J. F., “Toward the Use of Speech and Natural Language Intervention for a Language-disordered Population,”' Behaviour and Information Technology, Volume 18, Number 1, 1999.

divider

PATENTS

divider

THE PUBLICATION I WILL PROBABLY BE KNOWN FOR...

Jill's Guide to Newborn Care