Research Topics

Edge AI

We research sensing technologies that quickly capture real-world changes by combining new vision sensors such as event cameras with lightweight and efficient AI. In addition to image restoration and estimation of motion and three-dimensional structures that leverage the characteristics of sensors, we also work on reducing inference processing and communication volume, as well as improving labeling efficiency through active learning. Our goal is to realize AI that operates with high accuracy and in real time under limited computing resources and data.

• Improving Inference Efficiency in Dense Event Camera Tasks (MIRU2026)
• Event Representation Learning Considering Communication Bandwidth (MIRU 2026)
• EMARS: Event-based Motion-Aware Correction, Deblurring and Interpolation of Rolling Shutter Images（ICIP 2026）
• Geometric-Photometric Event-based 3D Gaussian Ray Tracing （CVPR 2026 Highlight）
• Simultaneous Motion And Noise Estimation with Event Cameras（ICCV2025）
• Non-invasive estimation of air convection using schlieren imaging technology with event cameras
• Three-dimensional human body scanning using event cameras
• Self-Supervised Noise Removal for Event-Based Optical Flow Estimation
• Optical Flow and Ego-Motion Estimation Using Event Cameras
• Non-Deep Active Learning for Deep Neural Networks
• Optical Flow Estimation Using Vehicle-Mounted Event Cameras
• Robust 2D Code Recognition Using Event Cameras Under Varying Lighting and Motion Blur Conditions

Physical AI

We conduct research to understand and predict human and robot actions, connecting this to generating appropriate behaviors and control. Based on person detection and tracking, pose estimation, action recognition, and environmental understanding from video, we work on motion generation considering physical consistency and control via reinforcement learning. We also explore methods to efficiently learn motion representations, such as utilizing pseudo-videos generated from still images, aiming for intelligence that enables humans and AI to cooperate in the real world.

• Data Collection-free Masked Video Modeling
• Guided by the Way: The Role of On-the-route Objects and Scene Text in Enhancing Outdoor Navigation
• Boosting Outdoor Vision-and-Language Navigation with On-the-route Objects
• Automatic Procedure Classification in Plastic Surgery Considering Surgical Instrument Information
• Human Posture Estimation Using Acoustic Information
• Simultaneous Detection of Objects and Relationships in Dynamic Scene Graph Generation
• Retrieving and Highlighting Action with Spatiotemporal Reference
• Moment-Sentence Matching from Temporal Action Proposals
• Daily Action Recognition Using Human and Object Presence Probabilities
• Detailed Action Recognition in Factory Production Lines
• Person Re-identification Using Distance Learning with CNN
• Online Multiple Object Tracking Using Re-identification of Tracking Trajectories
• Time-Series Action Recognition in Action Transition Videos
• Calibration-Free Gaze Estimation

Multimodal AI

We research AI that links different types of information such as images, videos, language, and sound to understand the world and create new expressions. Utilizing models trained on image-language correspondences and generative models, we tackle tasks including image recognition from limited data, language-guided segmentation, and integrated understanding and generation of audio and video. Through complementary use of diverse information and effective utilization of pre-trained models, we aim to develop AI that flexibly adapts to unknown objects and situations.

• Scalable Semantic Gesture Generation Utilizing Multimodal Prior Knowledge (MIRU2026)
• Zero-Shot Music-to-Image Generation Based on Common Textual Embedding Representations Using Textual Inversion (MIRU2026)
• DPPE: Rethinking Camera-Based Positional Encoding for Scaling Multi-View Transformers (Accepted at NeurIPS 2026)
• Overlap-aware ViT Harmonizer for Automatic Color Tone Adjustment of Different Optical Satellite Images (SSII2026)
• Reference-Free Image Quality Assessment for Virtual Try-On via Human Feedback（ECCV2026）
• Learning to Assist: Physics-Grounded Human-Human Control via Multi-Agent Reinforcement Learning （CVPR 2026）
• Listening without Looking: Modality Bias in Audio-Visual Captioning（ICIP 2026）
• 4D Reconstruction from Sparse Dynamic Cameras（CVPR 2026 Workshop 4DV）
• Sign-to-Speech Prosody Transfer （ICPR2026）
• Pre-training with Synthetic Patterns for Audio
• Language-Guided Contrastive Audio-Visual Masked Autoencoder with Automatically Generated Audio-Visual-Text Triplets from Videos
• Text-guided Synthetic Geometric Augmentation for Zero-shot 3D Understanding
• Formula-Supervised Sound Event Detection: Pre-Training Without Real Data
• Rethinking Image Super-Resolution from Training Data Perspectives
• Proto-Adapter: Efficient Training-Free CLIP-Adapter for Few-Shot Image Classification
• Construction of a large-scale dataset for image recognition targeting retail product shelves
• MaskDiffusion: Exploiting Pre-trained Diffusion Models for Semantic Segmentation
• TAG: Guidance-free Open-Vocabulary Semantic Segmentation
• Boosting Semantic Segmentation by Conditioning the Backbone with Semantic Boundaries
• Efficient Video Recognition Based on Important Patch Selection
• Peripheral Completion of 360-Degree Images for Efficient 3DCG Background Production
• Shadow Removal from Document Images by Learning with Fully Synthetic Images
• Introduction of Temporal Cross-Modal Attention for Event Estimation Using Audio and Visual Information in Videos
• Fast Soft Color Segmentation
• Attribute Transformation of Object Images Based on Natural Language Instructions
• Graph Convolutional Neural Networks on Superpixels for Segmentation
• Saliency Map Generation for Image Classification Using CNN Classifiers
• Simultaneous Color Adjustment and Image Completion Using GANs

Field AI

We advance research to deliver sensing and AI research outcomes to actual fields such as space, sports, fashion, and medical care. Facing unique challenges in each domain—such as satellite image analysis, sports motion evaluation, virtual fitting, and recognition of surgical procedures and diagnostic support—we refine our technologies through collaboration with experts and companies. Considering constraints of real environments and user perspectives, we aim to create systems useful on-site and generate new value.

• Phase Detection of Golf Swings Using Combined Action Spotting and Action Segmentation
• Surgical Instrument Detection in Open Surgery Using Feature Fusion Based on Hand and Instrument Interaction (MIRU2026)
• Quality evaluation of figure skating jumps using expert gaze information
• Quality evaluation of figure skating jumps using expert gaze information
• Detection of Osteoarthritis Using Multimodal Hand Data
• Diagnosis of Scoliosis Using Depth Images
• Estimation of Object Manipulation Methods Based on Learning of Stable Arrangements
• Estimation of Functional Parts of Objects Based on Operation Task Input
• Tactile Logging: A Method for Describing Operation History on Object Surfaces Based on Human Motion Analysis
• Six Degrees of Freedom Pose Estimation of Similar-Shaped Objects Focusing on Spatial Arrangement of Functional Attributes
• Scoliosis Screening by Estimating Spinal Alignment from Back Moiré Images
• Shot Detection in Tennis Match Videos
• Rugby Video Analysis System
• American Football Video Analysis System
• Swimmer Tracking System