{"id":3936,"date":"2020-04-22T22:32:30","date_gmt":"2020-04-23T02:32:30","guid":{"rendered":"https:\/\/www.hajim.wdev.rochester.edu\/senior-design-day\/pianosaic-ar-musical-experience\/"},"modified":"2020-05-02T17:06:43","modified_gmt":"2020-05-02T21:06:43","slug":"pianosaic-ar-musical-experience","status":"publish","type":"post","link":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/pianosaic-ar-musical-experience\/","title":{"rendered":"Pianosaic: AR musical experience"},"content":{"rendered":"\n
\"\"<\/figure>\n\n\n\n

Team Members<\/h2>\n\n\n\n

Zo\u00eb James
Molly Robins
Elizabeth Shaher
Kyle Tworek<\/p>\n\n\n\n

Supervisors<\/h3>\n\n\n\n

Michael Heilemann, Sarah Smith, Daniel Phinney, Mark Bocko<\/p>\n\n\n\n

\n\n\n\n

Description<\/h2>\n\n\n\n

Pianosaic is an augmented reality experience in which players get to create musical environments with a larger than life piano. Players are immersed in accurately spatialized audio. Players are able to use the keys of the piano around to create an individualize experience. Players are able to create a full production utilizing accompanying instruments in the game.<\/p>\n\n\n\n

\"Black
Black and white composite of group members wearing Magic Leap headsets. <\/figcaption><\/figure><\/div>\n\n\n\n
\n\n\n\n

Process<\/h2>\n\n\n\n

Motivation<\/h3>\n\n\n\n
\"\"<\/figure>
\n

We were all interested in spatial audio coding and interactive experiences. This diagram shows some of our early planning of the features we wanted to include, and how all of the parts would work together. <\/p>\n<\/div><\/div>\n\n\n\n

\"\"<\/figure>
\n

Our app uses Magic Leap’s MSA Spatializer to create spatial audio. For the first few weeks of planning, we explored the capabilities of the Magic Leap from a user standpoint by being the users ourselves. This photo shows Kyle exploring the Magic Leap’s capabilities. <\/p>\n<\/div><\/div>\n\n\n\n

Methods<\/h3>\n\n\n\n
\"\"<\/figure>
\n

The app was developed using Unity game engine with scripting in C#, and the audio was developed in Logic Pro X. This is a picture of what our Unity session looks like, which shows how it was all put together. <\/p>\n<\/div><\/div>\n\n\n\n

While we had many ideas on musical applications, we decided on the piano as a practical and physical option. We developed the app as we developed our skills in Unity, adding new features along the way until we combined what we had learned into one single product. Below is a flowchart showing the projects we created along the way to Pianosaic. <\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n
\n\n\n\n

The Product<\/h2>\n\n\n\n

Our final product is an application for the MagicLeap headset called Pianosaic. Features include: <\/p>\n\n\n\n

  • 2-Octave Piano, with individually moving tiles<\/li>
  • 18 Unique Loops, including a Drum Wall for customizing drum sounds, also individually movable and synchronized<\/li>
  • Recording capabilities, for the user to create their own 4-bar loop<\/li>
  • A metronome for keeping time<\/li>
  • A reset function, to bring all objects back to original positions<\/li>
  • A tutorial mode, to describe all of the functions, accessible after any reset<\/li><\/ul>\n\n\n\n
    \n\n\n\n

    Demos<\/h2>\n\n\n\n

    Tutorial<\/h3>\n\n\n\n
    \n