Exploring AI Usage in Creativity & Art

From January through May 2024, I took DSGN 5580: AI in Art — Redefining Creativity in the 21st Century with Professor Lisa Park. It was a hands-on studio at Penn where we engaged visual and multimedia AI tools as part of our individual artistic practice — and spent just as much time interrogating what creativity, authorship, and craft mean when the tools generate, remix, and respond on their own.

What follows is a week-by-week journal of what I learned, made, and kept thinking about.

Week 00 — Course registration

I just snagged the last seat in this AI design studio class at Penn — so incredibly excited.

I’ve been stunned by the sheer number of AI-related classes being offered across Penn right now: journalism, art, policy, business, marketing, and of course computer science. Professors are adapting quickly to the changing landscape, and you can feel it in the catalog.

It’s pretty unconventional for a PhD student at Penn’s School of Medicine to enroll in an art & design class, but I’m grateful to my mentors for letting me chase down my curiosities. The purpose of Professor Park’s new course is to let students experiment with a wide range of AI tools and expand our artistic practice. The syllabus sounds incredible.

Week 01 — Introductions

We started the Spring 2024 semester by introducing ourselves to the rest of the class. This was a special group — my classmates come from an unusually wide range of backgrounds: art & design, computer science, urban planning, animation, landscape architecture, digital design, graphic design, computational biology (me!), consumer psychology, 2D art, digital photography, landscape design, Chinese studies, UX/UI design, human-computer interaction, and more.

We’re also being taught directly by an international artist, which is so cool. Lisa walked us through her path: undergrad in Fine Art, master’s in Interactive & Digital Art, and several years running her own art studio in Korea. Because her projects are multimedia installations, she works alongside engineers, programmers, and designers to pull off the elaborate exhibitions she’s known for — work that integrates multisensory technologies in ways I haven’t seen before.

She showed us a handful of her projects — Luma, Harmony, Blooming, Twinkle Twinkle, and more — and I left the room buzzing for the semester ahead.

Week 02 — Evolution of AI in Art

First we dove into the social impact of AI. So many of us grew up watching sci-fi movies about AI, robots, and machines. For years, most people suspected those depictions were pretty far from reality — but in recent years, the emergence of AI has sent shockwaves across all of our lives. Not only in society, but also in art and design.

A few of the films we discussed:

Ex Machina (2014)
Black Mirror Season 2, Episode 1 (2013)
Her (2013)
Coded Bias (2020)

One of the Black Mirror episodes is about a service that uses AI to replicate a person’s online personality based on their social posts, interactions, and text messages. A woman uses it to bring back a digital version of her deceased boyfriend — and the emotional arc is brutal.

We also looked at real-world examples:

The Replika chatbot
Love and Loss in the Age of AI about Joshua Barbeau, whose AI chatbot was modeled after his fiancée who had died ten years earlier
Rosanna Ramos, who married an AI chatbot in 2023
The Rise of AI Companions documentary (2022)
How This Holographic Anime Character Could Be Your Next Girlfriend (2017)

It’s wild to see how much compute has accelerated to land us here. We briefly touched on Ray Kurzweil’s concept of the Singularity.

The coolest part was when Prof Park walked us through the evolution of AI in art, starting with the definition of art itself. Art used to be associated with certain skills, craftsmanship, and beauty — and then modern movements started challenging all of that.

We went era by era:

14th–16th century: depicting reality and symbolism. The artist’s role was record-keeper and storyteller, painting historical events and religious belief.
17th–18th century: landscapes and portraits.
Late 19th–early 20th century: the invention of the camera in 1816 broke things wide open. Artists no longer had to do the labor-intensive job of “capturing” — they started exploring new ways of representing reality, focusing on emotion and subjective interpretation.
Mid-20th century (one of my favorites): exploration of materials and conceptual art. Think Marcel Duchamp. Think controversial, sensational work where the idea behind the artwork carries more weight than the physical object.
21st century: the incorporation of technology and digital media. Maybe the artist’s role now is to integrate new tools and media into their creative process? There are obvious challenges — troubleshooting software/hardware, needing interdisciplinary teams instead of a single lone artist — but globalization and multiculturalism also open the door to a wider range of styles, themes, and lived perspectives.

We spent the rest of class trying entry-level AI image generation tools: ChatGPT-4, Bing Create, Adobe Firefly, Photoshop Beta Generative Fill, DALL-E. By the end of the session, we’d each created our first generative AI images. We also went around the room and shared takeaways from two assigned articles.

Week 03 — Exploring AI Visual Tools

I showed up to class to a delightful surprise. Professor Park had printed all of our AI-generated images onto heat-press paper and brought blank tote bags to share. We each spent a few minutes ironing — warming the bag, pressing the design — and tada: my tote turned out gorgeous. I loved seeing what my classmates had made.

Then we played with several more tools:

Playground AI
Leonardo AI — has modes for text-to-image, real-time canvas, and fine-tuning your own model
Midjourney (via Discord) — IMI Prompt, Prompt Hero, Niji Journey, and Image-to-Image modes, all with a lot of custom parameter tuning

Week 04 — Copyright Issues & Deepfakes in AI

On Valentine’s Day, we each took turns sharing our comparisons of AI visual tools. Notes I took for future me:

Galileo — good for UX mobile/app prototype design
Genie — text-to-3D model generation, but kind of difficult and needs heavy refinement
Luma AI (via Discord) — can produce cool 3D GIFs of mockups, though output isn’t great for detailed products
Stable Diffusion (via VSCode) — generates images in a virtual environment using Python commands
Veras AI — requires an input image, then will fill in details and material choices for the desired output
Leonardo and Midjourney — the friendliest interfaces

For my own piece this week, I worked on recreating idyllic village scenes from my Vietnamese seaside hometown using Canva AI, Bing Create, and Leonardo AI — including fine-tuning my own small image set with Leonardo.

After uploading my image set from a May 2017 visit to my village, I used the fine-tuned Leonardo model to generate paintings of boat refugees. The close-ups blew me away — there’s something visually artistic and emotionally expressive about how those AI renderings turned out.

Week 05 — Field trip to the ICA (Institute of Contemporary Art)

Our class got a private tour with the curators of the ICA. Am I dreaming?

They walked us through every intentional curatorial decision in each exhibition. Right at the front entrance, we were told to look up at the entryway glass — custom window-pane designs commissioned from a Black artist from New Haven who draws on influences from Zimbabwe and the wider African continent. The designs featured 2D representations of wrought iron, drawing on the history of how enslaved blacksmiths once melted metal to forge physical wrought iron fences.

One of the curators told us that curating is about bringing in artists whose medium is appropriate to the space and helps bring it alive. At the ICA specifically, the mission is to support up-and-coming artists, including helping them understand that artists are small businesses — and that the institution has a responsibility to help them figure out what they can and can’t do in any given space or show.

We walked through each exhibit and got asked probing questions about color, material, and what we thought the artists were symbolizing.

Week 06 — Midterm idea & studio time

After weeks of experimenting, we each had to start landing on a midterm project. Today’s task was to pin up our vision/mood boards and walk the class through our concept. After each pitch, the professor and peers chimed in with critiques and feedback.

I’m drawn to bringing multiple sensory dimensions into my work. Professor Park really liked my idea: create a highly-textured piece and paint an AI-generated image onto a textile canvas. The challenge is taking AI-generated media off the screen and embedding it into a tangible form that’s more engaging in person.

I’ve been hunting for other artists doing interesting sculpture work for inspiration:

Osang Gwon — Korean American artist who cleverly pushes the boundaries of photography and sculpture
Tomashi Jackson — American multimedia artist working across painting, video, textiles, and sculpture
Ilaria Marzano — Milan-based artist working across digital design, map illustration, data visualization, generative AI, and fabric scraps to illustrate the expansive surface area of textile waste

Week 07 — Spring Break (working through it)

It’s Spring Break! But I got assigned one of the early midterm presentation slots right after we get back, so I have to figure out how to work on my AI art project from another state for the entire week.

I spent the week researching an old artisan handicraft practice: weaving cói mats from tall sedge grass.

Learning about Vietnamese sedge mat weaving

Growing up, our own sedge mat was the center of a lot of activity in our home. These mats are massive — their surface area can cover a whole room, and they’re used for gathering, eating together, or sleeping on.

Many cultures have their own version: the Cambodian kantael, the Japanese tatami, the Korean wanchojang.

Building my own makeshift loom

Inspired by these rural village weavers, I wanted to try my hand at weaving my own mat to serve as a canvas for my AI images. I couldn’t grow, harvest, dye, and dry my own sedge grass, so I had to think about what else could create a similar textured surface.

I remembered seeing sleeping mats woven from plastic grocery bags — and there are community efforts to make these for folks experiencing homelessness. Grocery bags are abundant. That decided it.

I wanted to use proper wood and nails for the loom, but with the time crunch I went with what was on hand: enough free cardboard boxes to assemble loom frames.

What shocked me was how long it took to weave a single small mat — one to two days. The master weavers in Vietnam can finish a massive mat in roughly 1.5 hours, working as a team of two or three.

Weeks 08 & 09 — Midterm presentations

The first round of AI-generated midterm projects was mind-blowing. I cannot get over the level of creativity, originality, and intentionality my peers brought to their projects. That’s the wild thing about both artists and AI tools — neither ever creates the same thing twice.

A few that stuck with me:

An AI-generated music video with chilling townscape scenes — Midjourney for image generation, Runway AI for video editing
A massive collage of AI-generated environmental disaster/anxiety images printed via Risograph and assembled onto old cardboard boxes and paper shopping bags
A comparative analysis between interpretations of prompts for text-to-video AI platforms versus how a human animator would create the same scenes
An AI-generated song/music production
A visual photo/video-story with philosophical commentary using Midjourney + Runway AI, edited on mobile with Youcut

Week 10 — AI in music and sound art

We tried several AI sound tools to generate two new songs based on any content of our choice.

Week 12 — Gallery site visit (Stuart Weitzman Hall)

Week 13 — Final project development

Week 14 — Installation & art exhibition at Stuart Weitzman Hall

In late March, we went over to the Fine Arts Hall for a walk-through to scope out the space. From there, every artist was tasked with figuring out how to claim the space they needed for their own piece while sharing the same open floor plan with everyone else.

After three to four months, it was finally time to set up and exhibit. I was in disbelief at the beauty and AI innovation embedded in everyone’s process and final pieces. My own piece ended up being the simplest to set up because it didn’t require electricity, a projector, sound, or interactive video.