{"id":154892,"date":"2024-05-03T09:32:32","date_gmt":"2024-05-03T13:32:32","guid":{"rendered":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/?p=154892"},"modified":"2025-05-02T13:49:12","modified_gmt":"2025-05-02T17:49:12","slug":"wheeltalk-controlling-electric-wheelchairs-using-voice-commands","status":"publish","type":"post","link":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/wheeltalk-controlling-electric-wheelchairs-using-voice-commands\/","title":{"rendered":"WheelTalk: Controlling Electric Wheelchairs Using Voice Commands"},"content":{"rendered":"\r\n

Team Members<\/h2>\r\n\r\n\r\n\r\n

Paula Sedlacek, Audio and Music Engineering \u201924<\/p>\r\n\r\n\r\n\r\n

Qixin Deng, Audio and Music Engineering \u201924<\/p>\r\n\r\n\r\n\r\n

Jay Peskin, Audio and Music Engineering \u201924<\/p>\r\n\r\n\r\n\r\n

Tianyu Qi, Audio and Music Engineering \u201924<\/p>\r\n

\"\"<\/p>\r\n\r\n\r\n\r\n

Mentors<\/h2>\r\n\r\n\r\n\r\n

Michael Heilemann, Tre DiPassio, Dan Phinney, Sarah Smith<\/p>\r\n\r\n\r\n\r\n

Project Description<\/h2>\r\n\r\n\r\n\r\n

\u00a0Electric wheelchairs require users to interact with a joystick, alienating individuals with compromised motor skills. A compact and inexpensive voice-controlled device that can interface with the joystick of electric wheelchairs can assist users with limited mobility options. This project aims to design an attachment for powered wheelchairs that can control the chair\u2019s joystick in response to a user\u2019s voice commands. The attachment is meant to be a relatively inexpensive add-on unit to an already functioning powered wheelchair, and the unit should fit various available chairs. The project will require a system for capturing the user\u2019s voice, processing the captured audio to interpret the user\u2019s commands, and a mechanism for controlling the joystick based on what the user said. The device must be compact and inexpensive without compromising functionality and safety.<\/p>\r\n\r\n\r\n\r\n

Design<\/h2>\r\n\r\n\r\n\r\n

Block Diagram<\/h3>\r\n\r\n\r\n\r\n

The block diagram demonstrates the underlying logic of the speech recognition of WheelTalk.<\/p>\r\n\r\n\r\n\r\n

<\/p>\r\n\r\n\r\n\r\n

Starting at the top left, the diagram shows that the \u201cstop\u201d function of the WheelTalk module is completely independent of all other functions. No matter what is happening, if the user gives the \u201cstop\u201d command then the chair will stop immediately.\u00a0<\/p>\r\n\r\n\r\n\r\n

The red diamond shows the IDLE mode. When the module is in this state, it is only listening for \u201cstop\u201d and \u201chey chair\u201d, the wake word. Once the wake word is given, it moves on to the green diamond: AWAKE mode.\u00a0<\/p>\r\n\r\n\r\n\r\n

From here, the user can move the chair at its default speed by giving one of the direction commands: \u201cforward\u201d, \u201cback\u201d, \u201cleft\u201d, and \u201cright\u201d. This will move the joystick of the chair in the appropriate direction. The user can also change the speed of the chair to 1, 2, or 3 by saying \u201cspeed\u201d, the blue LED will light up, and then select a speed. This changes the amount that the joystick will move in the given direction.\u00a0<\/p>\r\n\r\n\r\n\r\n

In addition, any command (besides \u201cstop\u201d) resets a timer. If no new command is given within ten seconds of the previous command, the timer will reach its maximum and revert the module back to IDLE mode.\u00a0<\/p>\r\n\r\n\r\n\r\n

Hardware<\/h3>\r\n\r\n\r\n\r\n

Joystick Controller<\/h4>\r\n\r\n\r\n\r\n

The joystick controller has five main components: the brackets, rack and pinion gears, linear bearings, servo motors, and the outer cage. Most joystick controller components were designed using the CAD software Fusion 360 and printed using a Prusa i3 3D printer.<\/p>\r\n\r\n\r\n\r\n

\"\"\r\n
Side view and top view of the 3D CAD model of the controller<\/figcaption>\r\n<\/figure>\r\n\r\n\r\n\r\n
\"\"\r\n
Side view of joystick controller and top-down view of the longitudinal axes<\/figcaption>\r\n<\/figure>\r\n\r\n\r\n\r\n

Mount<\/h4>\r\n\r\n\r\n\r\n

The mount secures the joystick controller onto the wheelchair’s joystick and is predominantly made of metal. It sits atop the arm of the wheelchair, with the joystick protruding from the center of the mount. The joystick controller is screwed on the surface of the mount. An elastic band connected to the mount surrounding the arm of the wheelchair ensures that the mount is secure in different axes.<\/p>\r\n\r\n\r\n

\r\n
\"\"\r\n
Mounting of the joystick controller to the wheelchair<\/figcaption>\r\n<\/figure>\r\n<\/div>\r\n\r\n\r\n

 <\/p>\r\n\r\n\r\n\r\n

Microphone<\/h4>\r\n\r\n\r\n\r\n

Knowles FB-EI-31552-000 boom microphone is selected as the audio input. The Knowles is an electro-mechanical microphone that does not require biasing. The microphone will not detect audio unless the sound source is in close proximity to the microphone. The microphone features are listed below<\/p>\r\n\r\n\r\n\r\n

    \r\n
  • Type: Electro-mechanical<\/li>\r\n\r\n\r\n\r\n
  • Polar Pattern: Bi-directional<\/li>\r\n\r\n\r\n\r\n
  • Impedance: 300 Ohm<\/li>\r\n\r\n\r\n\r\n
  • Frequency Response Range:300-5k Hz<\/li>\r\n\r\n\r\n\r\n
  • Sensitivity @ 1 kHz: -67 dBV<\/li>\r\n\r\n\r\n\r\n
  • Humidity Tolerance: 50%<\/li>\r\n\r\n\r\n\r\n
  • Connection: 2 Wire<\/li>\r\n<\/ul>\r\n\r\n\r\n\r\n
    \"\"<\/figure>\r\n\r\n\r\n\r\n
    \"\"\r\n
    The datasheet and frequency response of Knowles FB-EI-31552-000<\/figcaption>\r\n<\/figure>\r\n\r\n\r\n\r\n

    Software<\/h3>\r\n\r\n\r\n\r\n

    Microcontroller<\/h4>\r\n\r\n\r\n\r\n
    \"\"<\/figure>\r\n\r\n\r\n\r\n

    For the voice recognition module, we chose the Elechouse V3, which is a compact voice recognition module that operates either via serial ports or built-in GPIO pins. It can store up to 80 voice commands and recognizes commands by comparing them to pre-recorded samples. However, the module presents a limitation of only loading up to seven of the stored commands each time.<\/p>\r\n\r\n\r\n\r\n

    The Arduino IDE is utilized to train the VR3 module and to control LEDs and motors based on the recognized voice commands. To address the limitation, the program is designed to dynamically load and clear commands within a conditional loop, which is detailed in the hierarchical graph underneath.<\/p>\r\n\r\n\r\n\r\n

    Speech Recognition<\/h4>\r\n\r\n\r\n\r\n
    \"\"\r\n
    The hierarchy of dynamic commands loading and clearing<\/figcaption>\r\n<\/figure>\r\n\r\n\r\n\r\n

    We designed our program to meet the process described in the block diagram. To solve the limitation of the voice recognition module only recognizing up to seven commands each time. The program is designed to dynamically load and clear commands according to each mode. The program is initialized with \u201cIdle\u201d mode, in which only two commands \u201cHey Chair\u201d and \u201cStop\u201d are loaded from the pre-recorded stop commands. If \u201cHey Chair\u201d is recognized, the module enters \u201cAwake\u201d mode and clears the loaded commands, and reloads six commands that control the direction(\u201cLeft\u201d, \u201cRight\u201d, \u201cForward\u201d, \u201cBack\u201d), a \u201cSpeed\u201d command,\u00a0 and additional stop commands. If \u201cSpeed\u201d is recognized, the module enters \u201cSpeed Selection\u201d mode and clears the loaded commands from the \u201cAwake\u201d mode, and loads commands \u201cOne\u201d, \u201cTwo\u201d,\u201d Three\u201d and an additional \u201cStop\u201d which allows users to control the speed of the wheelchair.\u00a0<\/p>\r\n\r\n\r\n\r\n

    A timer is also programmed with the speech recognition process. If the module has not recognized any commands in 30 seconds, it will reset the mode to \u201cIdle\u201d and the user needs to say \u201cHey Chair\u201d to re-wake it up.<\/p>\r\n\r\n\r\n\r\n

    The designed program successfully incorporates combinations of commands to seamlessly control the wheelchair with less than seven commands loaded each time and incorporates an emergent stop function for safety.<\/p>\r\n\r\n\r\n\r\n

     <\/p>\r\n\r\n\r\n\r\n

    Safety Features<\/h3>\r\n\r\n\r\n\r\n

    Fail-safe<\/strong>: Both mechanical and digital fail-safes were implemented to ensure the joystick returned to the\u00a0normal (stop) position if the device failed.<\/p>\r\n\r\n\r\n\r\n

    Input validation<\/strong>: The voice recognition module is speaker-dependent. Only the device\u2019s intended user can input voice\u00a0commands. Other voices will not be recognized responded.<\/p>\r\n\r\n\r\n\r\n

    Stop Command<\/strong>: The stop command is the only command that does not require a wake word. In an emergency, time is\u00a0paramount, and requiring a wake word to stop the device could be potentially fatal. By simply saying\u00a0\u201cstop,\u201d the user can stop the wheelchair from moving without any wake word commands.<\/p>\r\n\r\n\r\n\r\n

     <\/p>\r\n\r\n\r\n\r\n

    Demo Video<\/h2>\r\n\r\n\r\n\r\n
    <\/figure>\r\n