Payam Nikdel Research Assistant at Simon Fraser University.

Lip Reading Using Dual Attention Model

Designed an audio-visual lipreading system that can translate a sequence of face images to natural language. To do so, we generated a data set containing a series of people’s mouse images aligned with audio and subtitle from YouTube videos then trained a dual attention model. Tools: Pytorch, Python, OpenCV