Muhammad Jaleed Khan

Biography

Jaleed is a PhD researcher in Artificial Intelligence at the Data Science Institute at the University of Galway. He has over five years of R&D experience in academic and industrial setups and has published extensively in peer-reviewed journals and presented at highly regarded conferences. He is a member of the International Association for Pattern Recognition (IAPR), the Pakistan Pattern Recognition Society (PPRS), and the Artificial Intelligence & Computer Vision (iVision) Lab.

Talks and Events

Neural-Symbolic Visual Understanding and Reasoning using Deep Learning and Knowledge Graphs

Visual AI has made incredible progress in basic vision tasks using deep learning techniques that can detect concepts in visual scenes accurately and quickly. However, the existing techniques rely on labelled datasets that lack common sense knowledge about visual concepts and have biased distribution of visual semantic relationships. As a result, these techniques have limited visual relationship prediction performance, limiting the expressiveness and accuracy of semantic representation and downstream reasoning. We employed deep neural networks to predict visual concepts, including objects and visual relationships, and linked them to generate symbolic image representation. To alleviate the challenges above, we leveraged rich and diverse common sense knowledge in heterogenous knowledge graphs to systematically refine and enrich the generated image representation. As a result, we observed significant improvement in recall rates of visual relationship prediction (7% increase in Recall@100), expressiveness of the representation, and the performance of downstream visual reasoning tasks, including image captioning (15% increase in SPICE score) and image reconstruction. The encouraging results depict the effectiveness of the proposed approach and the impact on downstream visual reasoning.

Track: Deep Learning for and with Knowledge Graphs

Session Topics:

• Deep Learning for and with Knowledge Graphs