The Research:
Early Deep Learning systems are trained on huge datasets including text/image pairs that while can embed simple meaning, are still mainly trained on nouns like people and objects. Our main research area is to go beyond that to deeper Semantic Meaning & Analysis work. Which parses and models more multimodal and deeper semantic meaning to AI work. We do this by both analyzing meaning space (cognitive science of the arts, gesture, emotion, empathy, creativity, behavior etc.) but also then build models from cognitively and rigorously mapping these spaces that we can then use in our new emerging systems (with application in health, the arts, and education).
See samples of our work in the figures below:
Our work into a new method to understand, measure and model “aesthetic emotion” (see studies and papers):
Figure 1: Overview of the Emotion Wheel created by Song & Abukhodair
**Figure 2:** Explanation of the Emotion Wheel
**Figure 3:** Explanation of the Emotion Wheel - Secondary Emotion Selection
**Figure 4:** Reverse engineering generative visual AI systems to better understand current meaning making to improve them.
**Figure 5:** The diffusion based output where we then used an advanced system to reverse detect and heatmap the meaning of every word back through the system.
Deep studies with eye tracking, on how an art viewer perceives (cognitively) artwork — see Magic of Rembrandt’s Painting Technique Revealed:
Figure 6: Diagram from eye-tracking study showing that greater detail on a painted eye leads to quicker, longer and more frequent fixations.
Additional studies on deep meaning in aesthetics, emotions, creativity, empathy etc (see papers):
Figure 7: Desk setup for study on an artist’s creative process.
**Figure 8:** Diagram of eye-tracking data during an artist's creative process during a sketching task.
**Figure 9:** Outlining the Source-Path-Goal Schema as applied to creative interactivity.
**Video 1:** Video of the study shown in Figures 7, 8 and 9.
**Video 2:** Video of demo - system to track facial emotion expression to adapt visual appearance of video.
Designing a Wheel-based Assessment Tool to Measure Visual Aesthetic Emotions by
Abukhodair N; Song M; DiPaola S – Journal – 23 pages
Cognitive Systems Research (2023)
Engagement with artificial intelligence through natural interaction models by
Feldman S; Yalcin ON; DiPaola S – Conference – British Computer Society. pp. 296-303
Electronic Visualisation and the Arts (EVA) (2017)
Movement Awareness through Emotion Based Aesthetic Visualization by
Salevati S; DiPaola S; Carlson K – Conference – British Computer Society. 8 pages. London
Electronic Visualisation and the Arts (EVA) (2016)
A Creative Artificial Intelligence System to Investigate User Experience; Affect; Emotion and Creativity by
Salevati S; DiPaola S – Conference – British Computer Society. 8 pages. London
Electronic Visualisation and the Arts (EVA) (2015)
Enhancing Viewer's Emotional Connections to The Traditional Art Creative Process Via an AI Interactive System by
Salevati M; DiPaola S – Conference – British Computer Society. 8 pages. London
Electronic Visualisation and the Arts (EVA) (2015)
Using a Contextual Focus Model for an Automatic Creativity Algorithm to Generate Art Work by
DiPaola S – Journal – Special Issue; Bio Inspired Cognitive Architectures. Vol 41. pp. 212-219
Procedia Computer Science (2014)
How a Painter Paints; An Interdisciplinary Understanding of Embodied Creativity by
Choi S K; DiPaola S – Conference – British Computer Society. pp. 127-134. London
Electronic Visualisation and the Arts (EVA) (2013)
Rembrandts Textural Agency; A Shared Perspective in Visual Art and Science by
DiPaola S; Riebe C; Enns JT – Journal – Vol 43. No 3. pp. 145-151
Leonardo (2011)
Following the Masters; Viewer Gaze is Directed by Relative Detail in Painted Portraits by
Riebe C; DiPaola S; Enns JT – Journal – Vol 9. No 8. pp. 368-368 (abstract)
Journal of Vision (2009)
Following the masters; Viewer gaze is directed by relative detail in painted portraits by
Riebe C; DiPaola S; Enns JT – Conference – 9th Annual Meeting.
Abstracts of the Vision Sciences Society (2009)
Darwins Enduring Legacy by
DiPaola S – Journal – Images of my research in computer model of evolution ? selected by the Nature editors to accompany this essay (invited - not peer reviewed). Vol 451. pp. 632-633
The Journal Nature (2008)
The Trace and the Gaze; Textural Agency in Rembrandt?s Late Portraiture from a Vision Science Perspective by
DiPaola S – Conference – British Computer Society. 8 pages. London
Electronic Visualisation and the Arts (EVA) (2008)
Simulating Face to Face Collaboration for Interactive Learning Systems by
DiPaola S; Arya A; Chan J – Conference – 6 pages. Vancouver
E-Learn 2005 (2005)
Figure 1: Overview of the Emotion Wheel created by Song & Abukhodair
**Figure 2:** Explanation of the Emotion Wheel
**Figure 3:** Explanation of the Emotion Wheel - Secondary Emotion Selection
**Figure 4:** Reverse engineering generative visual AI systems to better understand current meaning making to improve them.
**Figure 5:** The diffusion based output where we then used an advanced system to reverse detect and heatmap the meaning of every word back through the system.
Figure 6: Diagram from eye-tracking study showing that greater detail on a painted eye leads to quicker, longer and more frequent fixations.
Figure 7: Desk setup for study on an artist’s creative process.
**Figure 8:** Diagram of eye-tracking data during an artist's creative process during a sketching task.
**Figure 9:** Outlining the Source-Path-Goal Schema as applied to creative interactivity.