Here are a few illustrations and short texts describing some of my past works. After reading them, I hope you will consider checking out my full length papers, which describe these projects in more detail.
My work
I study visual perception and cognition through computational models. What does this mean? It means that I first study a certain human cognitive capability, like the way humans can perceive or recognize things when observing an image. Then, I try to build a computational model for it. For example, a computer program that can automatically produce the thing that humans perceive or recognize when given an image.
I have used this method for several projects over the past few years. The mathematics toolbox for computational modeling in visual science is large and rapidly growing. I am both a user and contributor. The tools in my works are from the fields of differential geometry, numerical optimization, and statistical machine learning.
Full Interpretation of Minimal Images
Left:
What are the semantic internal parts that humans can identify in this image of a horse? When the image is small and reduced, but still recognizable, there are not many parts humans can identify. Here are 8 parts that humans can consistently identify.
Right:
We modeled humans ability to identify and localize semantic features and parts (termed ‘full interpretation’), in small and reduced image regions (termed ‘minimal images’). The model studies structural representations in the human visual system.
Interpretation of social interactions
Left:
What are the types and tones of social interactions in this image? Humans are remarkably good at recognizing social interactions from very limited image regions. These regions contain body parts and geometrical relations between body parts, which are crucial for the recognition task.
right:
We model visual recognition and interpretation of social interactions. The model can identify the body parts and their geometrical relations in images. We study the minimal recognizable interaction images, and their structural learning and representation in the human visual system
visual contour completion in the tangent bundle
Left:
Are there boundaries and shapes that you "see," but are not really there? The human visual system has clever mechanisms to deal with partial information, for example, due to occlusions.
right:
We model the mechanism of visual contour completion, by imposing the principle of minimal energy consumption in the primary visual cortex (V1). We show how completion is computed in artificial neural networks, in a bottom up manner without any knowledge about the object or scene.
curvature based visual texture segregation
Left:
Can you notice salient contours and boundaries here? Perhaps on the bottom, but not the top? These images, composed from multiple small oriented bars, are designed to show the sensitivity of the human visual system to two geometrical features: orientation and change of orientation (curvature).
right:
We model the perceptual saliencies and boundaries in visual texture (texture segregation) based on curvature. We show how the computation of texture curvature is feasible with the mechanisms of the early visual cortex.