Eye and Voice-Controlled Human Machine Interface System for Wheelchairs Using Image Gradient Approach

Saba Anwer, Asim Waris* (Corresponding Author), Hajrah Sultan, Shahid Ikramullah Butt, Muhammad Hamza Zafar, Moaz Sarwar, Imran Khan Niazi, Muhammad Shafique, Amit Pujari

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

16 Citations (Scopus)
6 Downloads (Pure)


Rehabilitative mobility aids are being used extensively for physically impaired people. Efforts are being made to develop human machine interfaces (HMIs), manipulating the biosignals to better control the electromechanical mobility aids, especially the wheelchairs. Creating precise control commands such as move forward, left, right, backward and stop, via biosignals, in an appropriate HMI is the actual challenge, as the people with a high level of disability (quadriplegia and paralysis, etc.) are unable to drive conventional wheelchairs. Therefore, a novel system driven by optical signals addressing the needs of such a physically impaired population is introduced in this paper. The present system is divided into two parts: the first part comprises of detection of eyeball movements together with the processing of the optical signal, and the second part encompasses the mechanical assembly module, i.e., control of the wheelchair through motor driving circuitry. A web camera is used to capture real-time images. The processor used is Raspberry-Pi with Linux operating system. In order to make the system more congenial and reliable, the voice-controlled mode is incorporated in the wheelchair. To appraise the system’s performance, a basic wheelchair skill test (WST) is carried out. Basic skills like movement on plain and rough surfaces in forward, reverse direction and turning capability were analyzed for easier comparison with other existing wheelchair setups on the bases of controlling mechanisms, compatibility, design models, and usability in diverse conditions. System successfully operates with average response time of 3 s for eye and 3.4 s for voice control mode
Original languageEnglish
Article number5510
Number of pages13
Issue number19
Early online date26 Sept 2020
Publication statusPublished - 1 Oct 2020


  • human machine interface (HMI)
  • rehabilitation
  • wheelchair
  • quadriplegia
  • image gradient
  • AMR-voice
  • Open-CV
  • image processing
  • Raspberry Pi
  • Image processing
  • Quadriplegia
  • Image gradient
  • Human machine interface (HMI)
  • Wheelchair
  • Rehabilitation
  • AMR voice


Dive into the research topics of 'Eye and Voice-Controlled Human Machine Interface System for Wheelchairs Using Image Gradient Approach'. Together they form a unique fingerprint.

Cite this