Sen Fang

I'm Sen Fang, a first-year Ph.D. student at Rutgers University, supervised by Prof. Dimitris Metaxas.

My main research field is Multimodal. My research interest covers Audio-Visual (talking-face and representation of text/audio, Audio Generated Image), AIGC (AI-generated content, Multi-view learning, NeRF/3D), Self-Supervised Learning (Pose recognition and modeling, object & action detection/recognition in videos, Medical Image Analysis) and VR/AR/DCG and Visual Perception (Enables the agent to make plan and navigate)!

Recently I have been interested in the following topic:

LLM/HCI/Embodied Intelligence - I will bring agents that can interact with scenes or people to reality, whatever it takes.
Identify and Model Behavior - particularly in perceiving and synthesizing dynamic humans, objects, and scenes. I think controlling robots by self-supervised training through real-world data and Computer Vision related technologies is the right path.
Sign Language - it's a worthwhile direction for AP to add to your almost any lab. I can introduce you to this field. (It has high entry threshold, small competition, annual tech hot-spots/Your past technology can be continuously applied, and it is easy to get funding or medical funding.), which is a robotics topic based on Computer Vision.

I am open to academic collaborations, and please drop me an email if you are interested in collaborating with me. I am currently looking for a 25 Summer Intern, if your company has a position, please contact me.

Selected Papers

* Equivalent contribution, † Corresponding author

SignLLM: Sign Languages Production Large Language Models
Sen Fang, Lei Wang, Ce Zheng, Yapeng Tian, Chen Chen^†
arXiv:2405.10718
[Paper] [Project Website] [Prompt2Sign Dataset]

Bridging the Gap between Text, Audio, Image, and Any Sequence: A Novel Approach using Gloss-based Annotation
Sen Fang, Sizhou Chen^*, Yalin Feng, Xiaofeng Zhang, Teik Toe Teoh^†
Preprint'24
[Paper] [Local-PDF]

SignDiff: Learning Diffusion Models for American Sign Language Production
Sen Fang, Chunyu Sui^*, Yanghao Zhou, Xuedong Zhang, Hongbin Zhong, Minyu Zhao, Yapeng Tian^†, Chen Chen^†
arXiv:2308.16082
[Paper] [Project Page] [Preprocessed Data]

→ Full list of Publication

Part of Friends (random sort)

Extend

Georgia Institute of Technology: Hongbin Zhong

Columbia University: Chunyu Sui

Tsinghua University: Jiayin Zhu

University of Waterloo: Yifan Wang

Nanyang Technological University: Yalin Feng

University of Washington: Yangyang Wu

Sen Fang

Recent News

Selected Papers

Part of Friends (random sort)