Skip to content

Flat Panel Smart Speaker

Team

  • Benjamin Kevelson
  • Jun Qiu
  • Julia Weinstock
  • Zilin Zeng

Mentors

Mike Heilemann, Sarah Smith, and Dan Phinney

 

Abstract

Flat panel technology has been an emerging field of research at the U of R over the past few years, and recent developments have shown the promise of flat panels as acoustic surfaces. Our project aims to demonstrate the multifaceted capabilities of flat panel technology by building a standalone smart speaker, which utilizes a compound speaker and microphone on a single panel. The virtual assistant software is a program developed in-house for MATLAB, and specialized for panel devices. Through our work, we hope to show the capabilities of flat-panels as an affordable and functional alternative to existing technology for smart speakers such as Amazon Alexa and Google Home.

Introducing Flat Panel Technology

Flat panel devices create sound through use of drivers mounted on the rear of the device. These drivers are placed in an array pattern to optimize the frequency response of the panel by cancelling the most modes, which creates a better sound for the listener. Additionally, our smart speaker utilizes a tweeter driver, which controls the high frequencies and allows for the other driver array to be tuned for lower frequencies.

Fig. 1: Demonstration of flat panel concepts.

In order to receive sound, we use a vibration sensor which is also mounted on the rear of the device. The recorded audio signal is then transcribed by a speech-to-text software, such as IBM Watson, to recognize key words that correspond to a predefined set of commands stored in a dictionary within the panelAssistant.m file.

Flat Panel Smart Speaker – Software

Voice Transcription – panelAssistant uses IBM Watson to transcribe user voice into text, then uses the text for command recognition. The link below is a demo for our panelAssistant:

https://www.youtube.com/shorts/Rl4R3K-eEN4

 

Music Cancellation – To accurately transcribe speech while playing music, panelAssistant applies audio processing to the input signal in order to remove recorded audio that may corrupt the speech signal. This process is illustrated in Fig. 2b, and the results are shown by the spectrograms in Fig. 2a. The top spectrogram is the raw sensor input before filtering, and the bottom spectrogram is the cleaned speech after processing. Notice that the formants of the speech signal originally masked by the music are visible in the bottom figure after the audio processing is applied.

Fig. 2a-1: Spectrograms: raw sensor input.
Noisy Speech Recording.
Fig. 2a-2: Spectrograms: clean speech after processing.
Clean Speech after Audio Processing.
Fig. 2b: Diagram of the cancellation process.

Flat Panel Smart Speaker – Hardware

Fig. 3: Hardware setup for the flat panel smartspeaker.

Piezo (I) and Piezo Amplifier (II) – As our user talks to the panel, the piezo (vibration sensor) detects the vibration on the panel and thus receives the speech signal. The piezo amplifier boosts the input signal by 40dB, and filters to cut out the low-frequency noise (<400Hz). 

Speech Recorded using the Piezo and the Piezo Amplifier
Fig.4a: Piezo Amplifier.
Fig.4b: LTspice Schematic for the Piezo Amplifier.
Fig.4c: Noise from the Piezo Amplifier.

Power Amplifier (III) – A power amplifier boosts signal current and converts a low-power signal to a higher power one in order for the loudspeaker drivers to operate.

Fig.5: Power Amplifier.

Loudspeaker Driver Array (IV & V) and Crossover Network – We performed acoustic measurements for the panel and determined the corresponding positions that best excite bass-mid frequencies and treble frequencies before designing our driver array based on these locations. We also coded a digital crossover network  so the four drivers attached to the center of the panel are assigned for outputting bass-mid frequency signal (IV) , while the single driver at the top of the panel is assigned for the treble frequency signal (V) .

Fig.6: Crossover Network Filtering.

Audio Interface (VI) – The audio interface connects the hardware and software components of this project. It receives speech signal from the piezo sensor to be processed by panelAssistant, and sends audio output to the drivers.

Fig.7: Audio Interface.
Fig. 8: Signal flow for the flat panel smart speaker.

Conclusions and Future Work

Now that we have a working demo for the flat panel smart speaker, our future work will focus on simplifying our system setup and improving the system stability:

Hardware – We will finish our PCB design for the piezo amplifier. Currently, our piezo amp is built on a perf board, which is fragile. Substituting the perf board version amp with a PCB version can substantially improve our hardware stability. We will also design a better enclosure for our flat panel smart speaker as it is unsafe to have exposing electronics. A case will be a better and more stable enclosure comparing to mounting the electronics to the frame.

Software – We plan to develop a Python version for our virtual assistant software so that we can install it in a tiny Raspberry Pi. By doing that, we will no longer need our bulky PC, and we can drastically improve the portability of our system. We also plan to add the music streaming feature to our smart speaker.

Acknowledgments

Thank you to Professor Heilemann and Tre DiPassio. It has been a great honor to be a part of the flat panel loudspeaker research, and we couldn’t have made it without your guidance, encouragement, and inspiration. We also want to thank Professor Smith and Professor Phinney for their consistent attention and help to with our  project.