Mathematical methods and applied information technologies of modeling, translation and training for the Ukrainian sign language

Work number - M 9 AWARDED

This is an abstract of the scientific work of  A.S. Ternov, O.V. Lozynska, D.V. Shkilnyuk

entitled V. M. Glushkov Institute of Cybernetics of National Academy of Sciences of Ukraine nominates this paper for the annual award of the President of Ukraine for young scientists.

The objective of this scientific workis to develop the mathematical methods and form a complex of the applied information technologies of  modeling, processing and translation of the Ukrainian sign language.

Scientific novelty of the studyis that the research proved for the first time and solved a number of important scientific and applied tasks related to the development of mathematical methods and the formation of a complex of applied information technologies of modeling, processing and translation of the Ukrainian sign language. It is analyzed and proposed the new methods of recognition and dactyl identification of the sign alphabet, methods of computer translation of the annotated Ukrainian sign language. It is performed the original research of the linguistic peculiarities of the Ukrainian sign language and developed the grammatically augmented ontology of the Ukrainian verbal and Ukrainian sign language. It is designed the multimedia technology of the analysis of the visual sound images and the emotional facial gesture, allowing to implement the animation of the emotional speech channel of the sign language speaker.

Practical significance of the obtained resultsconsists in the development of the information technologies complex for the dactyl alphabet of the sign language recognition, for the lips mimic recognition within pronouncing the Ukrainian language on the basis of the flexible templates that are represented using non-uniform rational B-splines, and the information technology of the annotated Ukrainian sign language translation. The developed mathematical methods and applied information technologies used for the design of the computer system synthesis and analysis of the visual component of mimic-articulatory process for the Ukrainian sign language reproduction and recognition, as well as in the development of the correct articulation training system, lip-reading system, preparation and creation of the educational materials, presented with the annotated Ukrainian sign language.

The results of the scientific research of the young scientists are implemented and used in Lviv special secondary boarding school for deaf children named after Maria Pokrova, in Kamianets-Podilskyi multidisciplinary education and rehabilitation center, in Ternopil specialized secondary boarding school for levels I-III. The results of the scientific research used for the development of the industrial software in the "Infoservice" scientific-technical company.

The authors of the research work has published 83 research papers (20 publications in professional journals, peer-reviewed in the international scientometric databases such as Scopus, Google Scholar, eLibrary and Russian science citation index, 10 publications in professional journals of Ukraine, 52 publications in the proceedings of the international scientific and technical conferences and 1 preprint).

n/a