Project Introduction

Project Introduction

Message

This research starts from a corporative research ”development of supporting technology, safe and comfortable support for people with disabilities” between the National Institute of Advanced Industrial Science and Technology and National Rehabilitation Centre for Persons with Disabilities (then) in 2004 to 2007.

At that time, in a part of the project, we researched and developed an orphan product about a gesture interface that makes it possible to run autonomously indoor and outdoor circumstances by head movement for people with disabilities who cannot use electric wheel chair because of cerebral palsy.

It has been 10 years after the end of the project, nowadays; truly useful product is coming because of the price down and improvement technology of the image range censor and personal computer.

The research objective and approach method

The research objective is that to provide low-priced, a flexible and adaptable switch interface with using an image range censor sell at a market for people with disabilities who cannot use ordinary interfaces. Our objective is to provide truly useful system for every user as to make the price of the system contains of personal computer, less than 100 thousand yen.

The approach method for research is that, we collect and classified the actual data of people with disabilities according to body part and used to develop basic a modulated gesture recognition engine to apply to various kinds of people with disabilities more and more.

Also, there are big feature in system as to make it possible to recognize multi gestures at the same time.

Research Team

  • National Institute of Advanced Industrial Science and Technology (AIST)
  •  
  • National Rehabilitation Centre for Persons with Disabilities (NRC)
  •  
  • National Center of Neurology and Psychiatry (NCNP)

Hot news !

  • July 9th, 2018
  • This research was introduced in a British magazine “IMPACT”, famous among research institutions around the world, research fund subsidiary organizations, government officials, and others. You can download the article here.

     

History

  • April 2018 – March 2020 Ministry of Internal Affairs and Communications, Strategic Information and Communications Research and Development Promotion Project (SCOPE)
    Priority Area Research and Development “Research and development of adaptive gesture interface for improving accessibility”

Representative Ikushi Yoda (AIST), sharing member Yoko Kobayashi, Awazawa Hiroyuki (NCNP)

Summery : We research and develop a gesture interface for people with disabilities who cannot use ordinary devices to operate PCs. We develop a unconstrained and non-contactable interfaces using commercially available distance image sensor to reduce the price. Our first target is people with disabilities, and we develop the technology to adapt their movements to develop standard technology in the future.
 
  • April 2016 – March 2019 Ministry of Education, Culture, Sports, Science and Technology, Science and research basis B “Research of adaptable gesture interface for people with disabilities to support communication ”

Representative Ikushi Yoda (AIST), sharing member Tsuyoshi Nakayama, Kazuyuki Ito (NRC), Yoko Kobayashi (NCNP)

Research result : We have been researching mainly of basic learning system using collected more than 100 part of body gestures, and we are about to complete gesture recognition module group of our basic suggestion. The basic part of this recognition engine will be complete at the end of this project.
 
  • April 2015 – March 2018 AMED (Total research and development project for persons with disabilities), ”The development of non-contact gesture recognition interface for cerebral palsy and cerebral stroke patients to support communication”

Representative Kazuyuki Ito (NRC), sharing member Ikushi Yoda (AIST)

Summery : We did long term adjustment experiment for actual users mainly research for cerebral palsy patient continuously, we collected gesture data from people with disabilities continuously, and we developed operation menu for the first time mainly of to operate personal computers, home electronics, bell pressing.
Research result : We collected data total number of 51 people, total number of the body part is 181 continuously. Also, we did long adjustment evaluation for the first time with state transition type of operation menu. Also, we made web site to open the software later and we will open foot gesture module as for the first version of prototype.
 
  • April 2014 – March 2015 Ministry of Health, Labor and Welfare, Health, Labor, Welfare and Science Research Trust Fund (total research project for persons with disabilities) “Research and development of module that is non-contact and unconstraint gesture interface to support of a person with disabilities”

Representative Ikushi Yoda (AIST), sharing member Tsuyoshi Nakayama, Kazuyuki Ito (NRC)

Summery : We research and developed gesture interfaces that make it possible to operate personal computer with more easily, as a continuous research mentioned above. Especially, we did long term adjustment experiment (3 to 4 month or more) for 3 examinee to develop a system that can customize the system for each user with more easily and more low cost. Also, we developed recognition engine module with collecting and classifying various kinds of movement from people with disabilities same as before.
Research result : We collected gestures with using these parts from total number of 36 people, adding to the previous project with hearing ideas from users that is person with disabilities of their own and their caregivers. And we collected total number of 125 parts of the body movement because the users want to use these parts to operate equipment as for a switch. Also, we newly developed leaning motion recognition modules that recognize movement of the body part that do not regulate part of the body, even though 5 parts of recognition modules, same as before. We did long term additional experiment for 3 examinee for over 3 month with using this new module, head part recognition module and finger recognition module.
 
  • April 2013 – March 2014 Ministry of Public Management, Home Affairs, Posts and Telecommunications SCOPE Priority area type (ICT innovation creation type) research and development ”Research and development of module that is non-contact and unconstraint gesture interface to support information illiterate”

Representative Ikushi Yoda (AIST), sharing member Tsuyoshi Nakayama, Kazuyuki Ito (NRC)

Summery : We researched and developed the interface that can operate information-processing equipment with easy gestures for persons with disabilities who cannot use ordinary switches or sight line input devices because of convulsion or involuntary movement. It is first in the world developing the system with collecting many gestures from people with disabilities.
Research result : We collected 36 parts of gestures that users want and users can do voluntary movement from 22 examinee (not total number but all different person). We classified these gestures with person concerned such as rehabilitation medical doctor, rehabilitation engineer and occupational therapist. At the same time we developed a prototype and make total 5 kinds of recognition modules (2 types of hands and arms, 2 types of head parts, 1 type of legs parts).
 
  • April 2012 – March 2013 Tateishi Research Foundation A“The promotion of Human and machine harmony” “Development of interface for cerebral palsy by image range sensor”

Representative Ikushi Yoda (AIST)、sharing member Tsuyoshi Nakayama, Kazuyuki Ito (NRC)

There are some cases that it is difficult to cope with the problem about recognition of gestures that it is difficult to recognize because of movement of the recognition area or spasticity even though it is easy to understand by familiar people with ordinary equipment. So we research and develop system for one people who have cerebral palsy. As a result, we made success as to develop system specialize for one user with using gesture, mainly for movement of finger and shake neck and open and close moth.

 
  • April 2004 – March 2006 Science and Technology Promotion Fund “Development of supporting technology safe and comfortable support for people with disabilities”

We developed a stereo camera and our system recognize the head motion in real time, and we succeeded to run electric wheelchair operated by people with disabilities of their own at indoor and outdoor condition.

 

Main papers

  1. Ikushi Yoda, Kazuyuki Itoh, and Tsuyoshi Nakayama, “Modular Gesture Interface for People with Severe Motor Dysfunction: Foot Recognition,” Proceedings of AAATE 2017, (Harnessing the Power of Technology to Improve Lives), IOS Press, pp.725-732 (2017)
  2. Ikushi Yoda, Kazuyuki Itoh, and Tsuyoshi Nakayama, “Long-Term Evaluation of a Modular Gesture Interface at Home for Persons with Sever Motor Dysfunction,” Proceedings of Universal Access in Human-Computer Interaction 2016, (Springer LNCS 9738), pp.102-116 (2016)
  3. Ikushi Yoda, Kazuyuki Itoh, and Tsuyoshi Nakayama; “Basic Long-term Experimental Evaluation on Modular Gesture Interface by People with Severe Motor Dysfunction,” IEICE Technical Report, WIT2014-93, pp.45-50 (2015) (in Japanese)
  4. Ikushi Yoda, Kazuyuki Itoh, and Tsuyoshi Nakayama: “Collection and Classification of Gestures from People with Severe Motor Dysfunction for Developing Modular Gesture Interface,” Springer Lecture Note in Computer Science (LNCS) HCI International 2015 (2015 in printed)
  5. Ikushi Yoda, Kazuyuki Itoh, and Tsuyoshi Nakayama: “Collection and Classification of Gestures from People with Severe Motor Dysfunction for Developing Modular Gesture Interface,” Springer Lecture Note in Computer Science (LNCS) HCI International 2015 (2015 in printed)
  6. Ikushi Yoda, Kazuyuki Itoh, and Tsuyoshi Nakayama: “Collection and Classification of Gestures from People with Severe Motor Dysfunction for Developing Modular Gesture Interface,” Correspondences on Human Interface Vol.16,No.2, SIG-ACI-12, PP.23-28 (2014) (in Japanese)
  7. I. Yoda, T. Nakayama, and K. Itoh: “Development of Interface for Cerebral Palsy Patient by Image Range Sensor,” Tateishi Science and Technology Foundation, Grand Research Results Vol. 22 2013, pp.122-125 (2013) (in Japanese)
  8. N. Sato, I. Yoda and T. Inoue: “Shoulder Gesture Interface for Operating Electric Wheelchair,” IEEE International Workshop on Human-Computer Interaction in conjunction with ICCV2009, pp.2048-2055 (2009)
  9. I. Yoda, J. Tanaka, Y. Kimura, B. Raytchev, K. Sakaue, and T. Inoue: “Non-contact Non-constraining Head Gesture Interface System for Electric Wheelchair,” IEICE Trans. Inf.& Syst.(Japanese Edition) Vol. J91-D, No.9, 2008 (2008) (in Japanese)