In Cookie We Trust
In Cookie We Trust is an interactive window installation display that reinterpret the classic fortune cookie in new media to know one's future. It's a bit chunk of fun that divines people's fortune specific to the person using Artificial Intelligence.
Role
Experience & Interaction Designer, Creative Technologist
Project Type
Interactive window installation display
Technology
Arduino, Processing, Facial Recognition
Collaborators
Chelsea Chen, Daniel Castaño, Arnav Wagh
Project Timeline
September 5th - November 13th 2018
Display
November 9th - 13th 2018 at New York University Skirball Windows, LaGuardia Pl, New York City
How fun is it to know your fortune?
Fortune-telling has been part of human life for millenia. Some people does take it to heart some just find it for fun. Fortune cookie is a unique phenomenon of fortune-telling which people get their fortune on a small slip of paper inside a cookie. The most interesting part of fortune cookie is how people experience the anticipation and enthusiasm while breaking the cookie. For some people, the fortune coming from the cookie defines how the Universe provides a belief to reinforce what one already wants and wishes for. Even though some others don't take it seriously, people always find it fun to see what a cookie can tell human about life. The art of classic fortune cookie is the randomness of fortunes. But, what if it can predict your fortune specifically to you? Do you believe it? Or just find it for fun?
The most interesting part of fortune cookie is how people experience the anticipation and enthusiasm while breaking the cookie.
To enhance people's enthusiasm and surprise, we hide the camera so that user wouldn't notice if the fortune was based on their facial recognition. The goal of this project is to bring fun into fortune-telling and make people ask their beliefs if the fortunes are made specifically for them.
Facial recognition to divine people's fortune
One of many technologies to know specific human presence is facial recognition. In order to bring more value to personalized fortunes, we used facial recognition technology to detect audiences' faces and categorize them based on the highest confidence rate. This data then will be used to map the categories into our fortunes database. We have hundreds of fortunes for each appearance category. That way, each person will get personalized fortune based on their facial recognition. We used Microsoft Azure API to detect human faces.
Engaging people on a window display
The biggest challenge of this project is how to engage people to interact with window installation display. Interactive window installation is one of the hardest installations to interact with. To engage the audiences, we paid attention to every detail interaction.
Interaction design
Visual display also plays a big role in engaging people's attention for window display. We used contrast colors and fabricated a giant fortune cookie as part of affordance to people's attention. For the interaction itself, we put hand sign to let people know how to interact with the installation.
Display at NYU Skirball
In the making
Technology
Code and fortunes database
Making a giant fortune cookie interactive installation
The process took about 2 months with the first month for developing the concept. Roughly we took 3 weeks of production from turning the visual sketch into an actual window installation display. The production was very intense from fabricating the giant fortune cookie by cutting woods using CNC machine, creating the cookie surface using foam, making the movement mechanism to open and close the cookie, and developing all physical computing parts; from LED strips, vibration speaker, spotlight, and linear actuator.
We used four separate microcontrollers (Arduino UNO) to make the installation robust and to reduce problems rate. Each Arduino connects to: LED 12V as spotlight, linear actuator to move the cookie open and close, capacitive touch as a main trigger from user, and Fadecandy LED driver to animate the LED Strips. We used processing as a main controller to control the entire sequence; starting from the user touched capacitive touch, then it would take user's picture, triggered LED thunder animation, thunder sound, spotlight increased brightness, and the cookie started to open. When it opened, the fortune message would be shown on the monitor display for 7 seconds until the cookie started to close. We had a fortune database contained of hundreds messages. A message that appeared was based on user's appearance using facial recognition technology.
Make people smile
I was so happy to see how people were smiling and excited to interact with the installation and very enthusiastic to see their personalized fortune. They were very surprised about how could the cookie knew them and gave the message very related to their personal appearance.
"It's hard not to feel happy when you make someone smile." - Unknown