AI Modeling Thought & Language

Researchers: Steve DiPaola, Rafael Arias Gonzalez, Nilay Yalcin, Maryam Ahmadzadeh

The Research: Our AI work in cognitive modeling of thought, expression and language using advanced AI systems like AI Knowledge Graphs of thought and chained RAG based LLMs. We have ongoing research on VR simulators where females can practice negotiating for a raise with their boss, as well as being able to ask any question to a climate change expert (see video below). Also, allowing the public to talk faithfully (as much as possible) to inspiring historical figures like our Picasso and Van Gogh. With Van Gogh, we used our specific AI pre-processing of times of his life, historical events/people and the 700 letters he wrote in his own words to his brother Theo into a knowledge system that then goes through additional 3D facial expression, body gesture, talking, voice AI systems.

See samples of the work in the videos below:

**Video 1:** Our advanced talking AI character work on creating thinking Virtual Medical Patients for Medical School Education.

**Video 2:** Embodied Conversational AI for Women's Negotiation Training: A Regulatory-Focus-Informed Simulation and Evaluation.

See figures and videos of our work below:

**Figure 1:** Cognitive modeling of our virtual Van Gogh.

**Figure 2:** Sample of user conversation with our Van Gogh model.

**Figure 3:** Our VR characters even dynamically morph into painted forms based on their emotional state.

**Video 3:** Our interactive realtime advanced AI/VR character talking with DiPaola about any aspect of climate change via its knowledge base.

**Video 4:** Talking with our Virtual AI Picasso live (created with ivizlab research and [Virtro](https://www.virtro.ca/))

------ PAPERS: AI Modeling Thought & Language ------

Creating an Emotionally Aware Portrait System Prototype using Aesthetic Emotion Evaluations of AI Art Portraits by Abukhodair N, DiPaola S – Journal – 8 pages
Journal of Perceptual Imaging (2025)

Noel: A Chatbot Persona to Support Children Designing for Others, ACM Conf on Human Factors in Computing Systems by Lo P, Veldhuis A, Antle A, DiPaola S. – Conference – 25 pages
ACM CHI Conference on Human Factors in Computing Systems (CHI) (2025)

Artist-Centric XR/AI Sandbox for Co-Creation and Performance by Mah S, … and DiPaola S – Conference – Article No. 4
ACM CHI Conference on Human Factors in Computing Systems (CHI) (2024)

Rethinking Artificial Intelligence creativity and ideation systems by DiPaola S – Conference – 8 pages
NeurIPS 2024, NeurIPS Workshop on Creativity & Generative AI (2024)

Exploring Augmentation and Cognitive Strategies for Synthetic Personae by Gonzalez RA; DiPaola S – Conference – Workshop Paper
ACM CHI Conference on Human Factors in Computing Systems (CHI) (2024)

SAGA; Collaborative Storytelling with GPT-3 by Shakeri H; Neustaedter C; DiPaola S – Conference – pp. 163-166
ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW) (2021)

Aesthetic Judgments; Movement Perception and the Neural Architecture of the Visual System by Utz V; DiPaola S – Conference – Advances in Intelligent Systems and Computing. vol 948. Springer
Biologically Inspired Cognitive Architectures (BICA) (2019)

A multi-layer artificial intelligence and sensing based affective conversational embodied agent by DiPaola S; Yalcin ON – Conference – pp. 91-92
Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) (2019)

M-Path; A Conversational System for the Empathic Virtual Agent by Yalcin ON; DiPaola S – Conference – Advances in Intelligent Systems and Computing. vol 948. Springer
Biologically Inspired Cognitive Architectures (BICA) (2019)

Evaluating levels of emotional contagion with an embodied conversational agent by Yalcin ON; DiPaola S – Conference – pp. 3143-3149. Montreal
Annual Conference of the Cognitive Science Society (2019)

Transforming Kantian Aesthetic Principles into Qualitative Hermeneutics for Contemplative AGI Agents by Turner JO; DiPaola S – Conference – pp. 238-247
International Conference on Artificial General Intelligence (AGI) (2018)

Integrating Cognitive Architectures into Virtual Character Design by Turner J; Bernardet U; Nixon M; DiPaola S – Book – IGI Global
Full Book (2016)

Affective Response Patterns as Indicators of Personality in Virtual Characters by Bernardet U; DiPaola S – Conference – 2 pages. MIT. MA
Biologically Inspired Cognitive Architectures (BICA) (2014)