See Related Papers and Related Projects:
## AI-based Exploration of Art and Design ## AI Anonymization ## AI Cognitive Creativity
## Sensing Humans (Bio/Brain/Face/Movement/VR) ## AI Modeling Thought & Language
Reseachers: Steve DiPaola, Suk Choi, Meehae Song, Nouf Abukhodair, Vanessa Utz
The Research:
Early Deep Learning systems are trained on huge datasets including text/image pairs that while can embed simple meaning, are still mainly trained on nouns like people and objects. Our main research area is to go beyond that to deeper Semantic Meaning & Analysis work. Which parses and models more multimodal and deeper semantic meaning to AI work. We do this by both analyzing meaning space (cognitive science of the arts, gesture, emotion, empathy, creativity, behavior etc.) but also then build models from cognitively and rigorously mapping these spaces that we can then use in our new emerging systems (with application in health, the arts, and education).
See samples of our work in the figures below:
Our work into a new method to understand, measure and model “aesthic emotion” (see studies and papers):
Figure 1: Overview of the Emotion Wheel created by Song & Abukhodair
Figure 2: Explanation of the Emotion Wheel
Figure 3: Explanation of the Emotion Wheel - Secondary Emotion Selection
Figure 4: Reverse engineering generative visual AI systems to better understand current meaning making to improve them. Here an emotionally complex prompt “an angry man punching a bag in a crowded joyful gym”.
Figure 5: The diffusion based output where we then used an advanced system to reverse detect and heatmap the meaning of every word back through the system.
Deep studies with eye tracking, on how an art viewer perceives (cognitively) artwork where we AI redraw masterwork (switch left/right eye detail & lost and found edges near chin). See several papers and over 200 press articles on our findings - like this one Magic of Rembrandt’s Painting Technique Revealed:
Figure 6: Diagram from eye-tracking study showing that greater detail on a painted eye leads to quicker, longer and more frequent fixations.
Additional studies on deep meaning in aesthics, emotions, creativity, empathy etc ( see papers):
Figure 7: Desk setup for study on an artist’s creative process.
Figure 8: Diagram of eye-tracking data during an artist’s creative process during a sketching task.
Figure 9: Outling the Source-Path-Goal Schema as applied to creative interactivity.
Video 1: Video of the study shown in Figures 7, 8 and 9.
Video 2: Video of demo - system to track facial emotion expression to adapt visual appearance of video (to reflect subject’s emotional state).
—— PAPERS: Semantic Meaning & Analysis ——
by
Abukhodair N; Song M; DiPaola S –
Cognitive Systems Research (2023)
by
Kitson A; DiPaola S; Riecke B –
ACM CHI Conference on Human Factors in Computing Systems (CHI) (2019)
by
Quesnel D; DiPaola S; Rieke B –
Semantic Ambient Media Experiences (SAME); Artificial Intelligence MEETS Virtual and Augmented Worlds (AIVR) with SIGGRAPH Asia (2018)
by
Feldman S; Yalcin ON; DiPaola S –
Electronic Visualisation and the Arts (EVA) (2017)
by
Salevati S; DiPaola S; Carlson K –
Electronic Visualisation and the Arts (EVA) (2016)
by
Choi S K; DiPaola S –
ACM Conf on Human Factors in Computing Systems (CHI) (2015)
by
Salevati S; DiPaola S –
Electronic Visualisation and the Arts (EVA) (2015)
by
Salevati M; DiPaola S –
Electronic Visualisation and the Arts (EVA) (2015)
by
DiPaola S –
Procedia Computer Science (2014)
by
DiPaola S –
Book Chapter. Electronic Visualisation in Arts and Culture (2013)
by
Bizzocchi J; Nixon M; DiPaola S; Funk N –
Digital Games Research Association Conference (DIGRA) (2013)
by
Choi S K; DiPaola S –
Electronic Visualisation and the Arts (EVA) (2013)
by
Gabora L; DiPaola S –
International Conference on Computational Creativity (ICCC) (2012)
by
Seifi H; DiPaola S; Enns JT –
International Symposium on Computational Aesthetics in Graphics; Visualization & Imaging (2012)
by
DiPaola S; Smith A –
Conceptual Structure; Discourse and Language (2012)
by
Choi S K; DiPaola S; Schiphorst T –
Conceptual Structure; Discourse and Language (2012)
by
DiPaola S; Riebe C; Enns JT –
Leonardo (2011)
by
Riebe C; DiPaola S; Enns JT –
Journal of Vision (2009)
by
DiPaola S; Gabora L –
Genetic Programming and Evolvable Machines Journal (2009)
by
Riebe C; DiPaola S; Enns JT –
Abstracts of the Vision Sciences Society (2009)
by
DiPaola S –
SPIE Human Vision and Imaging. Int. Society for Optical Engineering (2009)
by
DiPaola S –
The Journal Nature (2008)
by
DiPaola S –
Electronic Visualisation and the Arts (EVA) (2008)
by
DiPaola S; Arya A –
ACM SIGGRAPH - Symposium on Videogames (2006)
by
DiPaola S –
International Digital Media and Arts Association (IDMA) (2006)
by
DiPaola S; Arya A; Chan J –
E-Learn 2005 (2005)
by
Tolmie J; DiPaola S; Charles A –
Digital Games Research Association (2005)
by
DiPaola S; Arya A –
Electronic Visualisation and the Arts (EVA) (2004)
by
DiPaola S –
ACM SIGGRAPH - Conference Abstracts and Applications (2002)