Exploring AI Usage in Creativity & Art

January 2024 to May 2024

Week 00: Course Registration

I recently snagged the last seat in this AI design studio class at Penn! I'm so incredibly excited. I've been stunned by the number of AI-related classes being offered and taught at Penn in journalism, art, policy, business, marketing, and of course computer science department. Professors are adapting quickly to the changing landscape.

It's pretty unconventional for a PhD student at Penn's School of Medicine to enroll in an art & design class, but I'm grateful to my supportive mentors who let me chase down my curiosities. Check out how cool the syllabus sounds for Professor Lisa Park's DSGN 5580 AI in Art course. The purpose of this new course is to allow students to epxeriment with many different AI tools to try and expand our artistic practice.

DSGN 2580-401_ A.I. in Art Syllabus.pdf

We're directly being taught by an international artist, which is so cool! Lisa Park told us about how she studied Fine Art for her undergrad and then Interactive and Digital Art for her master's degree. When starting out in her career, she established and ran her art studio in Korea for several years. Since her projects are multimedia installations, she worked with a team of engineers, programmers, and designers to create her elaborate projects and exhibitions integrated with multisensory technologies. My professor has created so many fascinating projects: Luma, Harmony, Blooming, Twinkle Twinkle, and more!! I left the class so incredibly excited for the semester.

Week 01: Introductions

We started out the Spring 2024 semester by introducing ourselves to the rest of the class. This was a special group because my classmates come from a wide range of backgrounds: art & design, computer science, urban planning, animation, landscape architecture, digital design, graphic design, computational biology (me!), consumer psychology, 2D art, digital photography, landscape design, Chinese studies, UX/UI design, human-computer interactions, and more.

Below are my slides featuring a few pieces from my portfolio over the years. The last slide outlines my personal reasons for exploring AI in creativity!

2024-01-24-dsgn5580-van-truong-intro

Week 02: Evolution of AI in Art

First we dived into the social impact of AI. So many of us grew up watching sci-fi movies about AI, robots, and machines. For years, most people suspected these depictions were kind of far from reality but in recent years, the emergence of AI has significantly sent shockwaves across all our lives. Not only in society, but also art & design!

We discussed several movies which featured AI or robot themes.

Ex Machina (2014)
Black Mirror Season 2, Episode 1 (2013)
Her (2013)
Coded Bias (2020)

One movie was about a service that uses AI to replicate a person's online personality based on social networking, posts, interactions, and text messages. An individual decided to use this service to create a digital version of their ex-boyfriend and they go through this incredible emotional journey.

We also discussed several news articles of real-world examples of AI usage.

Replika Chatbot Program
Love and the Loss in the Age of AI by Josua Barbeau where an AI chatbot was modeled after his deceased fiance who died 10 years prior
Rosanna Ramos who married AI chatbot in 2023
The Rise of AI Companions Documentary (2022)
How This Holographic Anime Character Could Be Your Next Girlfriend (2017)

It's so interesting to see the monumental growth of computing which has accelerated and lead to this point! We briefly discussed Ray Kurzweil's concept of the Singularity (diagram below).

The coolest part was when Prof Park walked us through the "Evolution of AI in Art" starting with defining the concept of art. It used to be associated with certain skills, craftsmanship, and beauty then modern concepts of art began to challenge the traditional concepts.

In each of the different eras, we explored the art movements and artist's role respectively from the 14th to 16th century which was intent on depicting reality and symbolism. The artists role went from being a record keeper and storyteller painting historical events (as well as capturing/showcasing religious beliefs) to painting landscapes and portraits in the 17th to 18th Century.

Then came the invention of the camera in 1816! This really changed the role of artists from doing very labor-intensive captures to exploring new ways of representing reality, focusing on emotions, and creating subjective interpretations of art in the late 19th to early 20th century. One of my favorite eras is the mid-20th century which brought on an exploration of materials and conceptual art. Think Marcel Duchamp! Think very controversial and sensational art which challenges preconceptions of art, beauty, and the idea behind the artwork now holds more significance than the physical object.

Now, we're entering the 21st century which includes the incorporation of technology and digital media. Perhaps with the advent of widespread digital technology, maybe the artists role is to integrate new tools and media into their creative processes?! There are obviously challenges and downsides to digital and interactive art, such as having to troubleshoot software & hardware tools requiring larger interdisciplinary teams and resources rather than relying on a single artist. This is exciting especially with globalization and multiculturalism, which led to a diverse range of artistic styles, themes, and approaches as artists from around the world draw on inspiration from various cultures, perspectives, and lived experiences.

Then we spent the remaining class time using some entry-level AI image generation tools first, like ChatGPT-4, Bing Create, Adobe Firefly, Photoshop Beta Generative Fill, DALL-E, and more. At the end of class, we created our first images using generative AI tools. We also went around the room and shared our thoughts on two articles we read. Below are my takeaways!

2024-01-31-dsgn-5580-hw01

Week 03: Exploring AI Visual Tools

I showed up to class to a delightful surprise! Professor Park printed out all of our AI-generated images onto heat press paper and generously brought blank tote bags to share. We each spent a few minutes using the iron to warm up the tote bag. Once the surface was warm enough, the designs were transferred onto the front of the totebag and tada! I absolutely love how my totebag turned out. I enjoyed seeing all of the creative designs that my classmates generated.

Hands-on class session on heatpressing our own AI-generated art onto totebags!

Then we played around with several AI tools:

Playground AI
Leonardo AI, which has modes for text-to-image, real-time canvas, and finetune your own model
Midjourney via Discord, which has IMI Prompt, Prompt Hero, Niji Journey, and Image-to-Image modes which allow a ton of custom parameter tuning

Week 04: Copyright Issues & Deepfakes in AI

On Valentine's Day, we each took turns to share our comparisons of AI visual tools. I took some notes to refer back to later when using these tools again!

Galileo is a good tool for UX mobile/app prototype design
Genie is a text-to-3D model generation tool, which is kinda difficult and needs additional refinement
Luma AI via Discord can create cool 3D GIFs of mockups, but the output isn't great for many detailed products
Stable Diffusion via VSCode can be used to create images in a virtual environment with commands in Python
Veras AI requires an input image, but can be asked to fill in all the details and material choices of the desired output
Leonardo and Midjourney are also easy interfaces to use

Below are my slides for creating cultural village scenes using Canva AI, Bing Create, and Leonardo AI (including finetuning my own small image set and model!).

My comparative analysis on several AI tools for recreating idyllic scenes from my village

2024-02-14-dsgn5580-ai-comparative-analysis-van-truong

Experimenting with Leonardo AI

After uploading my own image set of photographs from my May 2017 visit to my seaside village, I used this model to generate some paintings of boat refugees. Check out the close-ups of these outputs! I'm quite impressed with how visually artistic and expressive these AI renderings turned out.

Week 05: A Field Trip to the ICA (Institute of Contemporary Art)

Our class got a private tour with the curators of the ICA! Am I dreaming?!

They pointed out all the extremely intentional curatorial decisions in each exhibition and space. As we walked through the front entrance, we were told to look up at the entryway glass to see custom window pane designs commissioned from a Black artist from New Haven who draws on influences from Zimbabwe and also the African continent. The designs featured 2D representations of wrought iron graphics, drawing on how the original blacksmiths were enslaved peoples who used to melt the metal entirely to create physical wrought iron fences.

Walk-Through with Philadelphia Institute for Contemporary Art Curators

One of the curators said that the process of curating is about bringing in artists whose medium is something that's appropriate to the space and helps bring it alive. As a curator, especially at ICA, their mission is to bring in artists who are up-and-coming and are learning along the way that artists themselves are their own small businesses too. So they feel a responsibility to help every artists understand what they can and can't do in any given space or exhibition.

We got to walk through all of the exhibits and were asked probing questions about color and material usage along with teasing apart what we believe the artists were trying to symbolize in their work.

Week 06: Midterm Idea & Studio Time

After brainstorming about AI tool usage for the past several weeks, every single student started thinking about our individual midterm projects. Our task for the start of class was to pin up our vision/mood boards of ideas and inspiration. Then we took turns examining each other's vision boards and presenting some of our ideas to the class. After each pitch, the professor and peers chimed in with critiques and feedback.

Vision Board Activity & Critiques for Individual AIxArt Midterm Projects

Potential Sources of Inspiration

I enjoy bringing multiple sensory dimensions into my work. Professor Park really liked my idea to create a highly-textured piece and paint an AI-generate image onto this textile canvas. The challenge here is taking AI-generated media off a computer screen and embedding it into a tangible form that's more engaging.

I'm trying to find other artists doing interesting sculpture work to serve as inspiration.

Osang Gwon

Korean American artist who cleverly pushes the boundaries of photography and sculpture

Tomashi Jackson

Tomashi Jackson is an American multimedia artist working across painting, video, textiles and sculpture

Ilaria Marzano

Ilaria Marzano is based in Milan, Italy working across digital design, map illustration, data visualization, generative AI, and fabric scraps to illustrating the expansive surface area occupied by textile waste

Week 07: Spring Break!

It's Spring Break! But, I got assigned one of the early midterm presentation slots right after we get back. So now I have to somehow figure out a way to work on my AI art project while I'm in another state for the entire week.

I spent some time researching about an old artisan handicraft practice of making woven cói mats from tall sedge grass.

Learning About Weaving Viet Sedge Mats

I started out by learning about traditional artisan practices like sedge mat weaving. Growing up, I remember our own sedge mat being the center of a lot of activities in our home. These mats are massive. Their surface area can cover the floor of a room, usually intended for gathering to sit, eating together, or as sleeping mats.

Many cultures have their own versions of these mats. These mats are pretty iconic in many Asian homes (i.e. Cambodian kantael, Japanese tatami, Korean wanchojang).

Building My Own Makeshift Loom & Collecting Weaving Materials

Inspired by these rural village weavers, I wanted to try my hand at weaving my own as a canvas for my AI images. Since I couldn't go out and grow, harvest, dye, and dry my own sedge grass, I decided to explore other materials I could create a similar textured surface with.

I remember seeing a couple sleeping mats woven from plastic grocery store bags and am aware of several efforts in different communities to create these types of mats for folks experiencing homelessness. With how abundant grocery store bags are, I figured this would be a good material choice!

I really wanted to use wood and proper nails, but with the time crunch, I went ahead with reusing materials that are readily available. I scoured for enough free cardboard boxes to make my loom frames.

What shocked me was how long it took me to weave this singular small mat (1-2 days)! The master weavers in Vietnam are able to work with 1-2 other people to weave a massive mat in approximately 1.5 hrs.

Week 08 & 09: Midterm Presentations (Round 1 & 2)

The first round of AI-generated midterm projects were truly mind-blowing! I still cannot get over the level of creativity, originality, and intentionality that my peers exerted on their individual projects. That's the amazing thing about both artists and AI tools - neither ever create the same thing.

Some of the cool AI-generated midterm projects:

AI-generated music video with chilling townscape scenes created using Midjourney for image generation and Runway AI for video editing
Massive collage of AI-generated environmental disaster/anxiety images printed via Risograph technique and assembled onto old cardboard boxes and paper shopping bags
Comparative analysis between interpretations of prompts for text-to-video AI platforms compared to how a human animator would create the intended scenes
AI-generated song/music production
Visual photo/video-story featuring philosophical commentary using Midjourney and Runway AI as well as Youcut for mobile phone editing of the final video

Below are my slides with my artistic process in figuring out which format and visual content to focus on.

2024-03-13-dsgn5580-ai-midterm-research

Week 10: AI in Music and Sound Art

We also several AI sound tools to generate two new songs based on any content of our choice. I'll add the audio here soon once I figure out the best way to embed it.

Week 11: Preliminary Concept for Final Project

Week 12: Gallery Site Visit (Stuart Weitzman Hall)

2024-04-03-museum-analysis-and-final-project-idea-van-truong

Week 13: Final Project Development

Week 14: Installation & Art Exhibition at Stuart Weitzman Hall

In late March, we went over to the Fine Arts Hall for a walk-through and to check out the space. From there, every artist was tasked with figuring out how to get the space they needed for their own pieces, while sharing and co-exhibiting in the same open floorplan as all the other artists.

After 3-4 months, it was finally time to set-up and exhibit our work!! I was in disbelief at the beauty and AI innovation embedded into everyone's process and final products. My piece ended up being the simplest to set up because it didn't require electricity, projector, sound, or interactive video.