How can interactions with one’s digital self influence your perception of self? Is your self perception malleable?
“Pixel Persona” is a interactive art piece that aims to answer this question by developing a digital avatar of oneself composed of one’s digital information on the internet. Social media posts, pictures and data for a user are scraped online and used to create a avatar that a user can manipulate using mid-air hand gestures to explore their digital identity. My role in this project was both design software development, and in particular, integrating the hand gesture recognition with the frontend visualization I developed using React Three Fiber.
Project Details
Role: Developer and Designer
Skills: React, React Three Fiber, Javascript, HTML/CSS,
Project Type: MIT Multimodal Interaction and Design
Team: Dominic Co, Maggie Chao
Project Date: 2024
To explore the above questions, we developed a mvp of explorative visualizations/avatars of one’s digital self based on their online personal data from Linkedin, Instagram and Facebook. Each visualization is crafted based on one’s social media profile, and stylized algorithmically to enhance the visual differences between avatars. “Connection” between the digital avatar of the user and the user him/herself is enhanced by mimicking the actual user’s head movements and hand movements, enabled by Google Mediapipe’s hand landmark detection and by FaceCap in react three fiber; the virtual avatars act like a mirror to one’s digital self. We were particularly inspired by Tom Cruise’s Minority report UI:
Demos and Trials
As the user explores his/hers avatars, he/she can begin to drag and create their own perception of self by dragging voxels from each of their virtual selves to an empty profile, helping the user literally craft their own perception of digital self. Below are some of the demos we built featuring the user interaction we envisioned: