Publications
Self-Supervised Shape Alignment for Sports Field Registration
This paper presents an end-to-end self-supervised learning approach for cross-modality image registration and homography estimation, with a particular emphasis on registering sports field templates onto broadcast videos as a practical application. Rather then using any pairwise labelled data for…
Detecting and Matching Related Objects with One Proposal Multiple Predictions
Yang Liu, Luiz G Hafemann, Michael Jamieson, Mehrsan Javan // CVPRW 2021
Tracking players in sports videos is commonly done in a tracking-by-detection framework, first detecting players in each frame, and then performing association over time. While for some sports tracking players is sufficient for game analysis, sports like hockey, tennis and polo may require…
Exploiting Prunability for Person Re-Identification
Hugo Masson, Amran Bhuiyan, Le Thanh Nguyen-Meidine, Mehrsan Javan, Parthipan Siva, Ismail Ben Ayed, Eric Granger
Recent years have witnessed a substantial increase in the deep learning (DL) architectures proposed for visual recognition tasks like person re-identification, where individuals must be recognized over multiple distributed cameras. Although these architectures have greatly improved the…
Learning Agent Representations for Ice Hockey
Guiliang Liu, Oliver Schulte, Pascal Poupart, Mike Rudd, Mehrsan Javan // NeurIPS 2020
Team sports is a new application domain for agent modeling with high real-world impact. A fundamental challenge for modeling professional players is their large number (over 1K), which includes many bench players with sparse participation in a game season. The diversity and sparsity of player…
Group Activity Detection from Trajectory and Video Data in Soccer
Ryan Sanford, Siavash Gorji, Luiz G. Hafemann, Bahareh Pourbabaee, Mehrsan Javan // CVPR 2020
Group activity detection in soccer can be done by using either video data or player and ball trajectory data. In current soccer activity datasets, activities are labelled as atomic events without a duration. Given that the state-of-the-art activity detection methods are not well-defined for atomic…
Actor-Transformers for Group Activity Recognition
Kirill Gavrilyuk, Ryan Sanford, Mehrsan Javan, Cees G. M. Snoek // CVPR 2020
This paper strives to recognize individual actions and group activities from videos. While existing solutions for this challenging problem explicitly model spatial and temporal relationships based on location of individual actors, we propose an actor-transformer model able to learn and selectively…
Unsupervised Domain Adaptation in Person re-ID via k-Reciprocal Clustering and Large-Scale Heterogeneous Environment Synthesis
Devinder Kumar, Parthipan Siva, Paul Marchwica, Alexander Wong // The IEEE Winter Conference on Applications of Computer Vision (WACV), 2020, pp. 2645-2654
An ongoing major challenge in computer vision is the task of person re-identification, where the goal is to match individuals across different, non-overlapping camera views. While recent success has been achieved via supervised learning using deep neural networks, such methods have limited…
Pose Guided Gated Fusion for Person Re-identification
Amran Bhuiyan, Yang Liu, Parthipan Siva, Mehrsan Javan, Ismail Ben Ayed, Eric Granger // The IEEE Winter Conference on Applications of Computer Vision (WACV), 2020, pp. 2675-2684
Person re-identification is an important yet challenging problem in visual recognition. Despite the recent advances with deep learning (DL) models for spatio-temporal and multi-modal fusion, re-identification approaches often fail to leverage the contextual information (e.g., pose and illu-…
Optimizing Through Learned Errors for Accurate Sports Field Registration
Wei Jiang, Juan Camilo Gamboa Higuera, Baptiste Angles, Weiwei Sun, Mehrsan Javan, Kwang Moo Yi // The IEEE Winter Conference on Applications of Computer Vision (WACV), 2020, pp. 201-210
We propose an optimization-based framework to register sports field templates onto broadcast videos. For accurate registration we go beyond the prevalent feed-forward paradigm. Instead, we propose to train a deep network that regresses the registration error, and then register images by finding the…
Learning Person Trajectory Representations for Team Activity Analysis
Nazanin Mehrasa, Yatao Zhong, Frederick Tung, Luke Bornn, Greg Mori // Sloan Sports Analytics Conference, 2018
Activity analysis in which multiple people interact across a large space is challenging due to the interplay of individual actions and collective group dynamics. We propose an end-to-end approach for learning person trajectory representations for group activity analysis. The learned representations…
Deep Learning of Appearance Models for Online Object Tracking
Mengyao Zhai, Lei Chen, Greg Mori, Mehrsan Javan Roshtkhari // The European Conference on Computer Vision (ECCV) Workshops, 2018, pp. 0-0
This paper introduces a deep learning based approach for vision based single target tracking. We address this problem by proposing a network architecture which takes the input video frames and directly computes the tracking score for any candidate target location by estimating the probability…
Time Perception Machine: Temporal Point Processes for the When, Where and What of Activity Prediction
Y. Zhong, B. Xu, G. Zhou, L. Bornn, G. Mori // arXiv preprint, 2018
Numerous powerful point process models have been developed to understand temporal patterns in sequential data from fields such as health-care, electronic commerce, social networks, and natural disaster forecasting.
Deep reinforcement learning in ice hockey for context-aware player evaluation
G. Liu, O. Schulte // International Joint Conference on Artificial Intelligence, 2018
A variety of machine learning models have been proposed to assess the performance of players in professional sports. However, they have only a limited ability to model how player performance depends on the game context. This paper proposes a new approach to capturing game context.
A Markov Game model for valuing actions, locations, and team performance in ice hockey
O. Schulte, M. Khademi, S. Gholami, Z. Zhao, M. Javan, P. Desaulniers // Data Mining and Knowledge Discovery, 2017
We apply the Markov Game formalism to develop a context-aware approach to valuing player actions, locations, and team performance in ice hockey. The Markov Game formalism uses machine learning and AI techniques to incorporate context and look-ahead.
Apples-to-apples: Clustering and ranking NHL players using location information and scoring impact
O. Schulte, Z. Zhao, M. Javan, P. Desaulniers // Sloan Sports Analytics Conference, 2017
Using new game events and location data, we introduce a player performance assessment system that supports drafting, trading, and coaching decisions in the NHL. Players who tend to play in similar locations are clustered together using machine learning techniques.
The evaluation of pace of play in hockey
R. Silva, J. Davis, T. Swartz // Journal of Sports Analytics, 2017
This paper explores new definitions for pace of play in ice hockey. Using detailed event data from the 2015-2016 regular season of the National Hockey League (NHL), the distance of puck movement with possession is the proposed criterion in determining the pace of a game.
Sports field localization via deep structured models
Namdar Homayounfar, Sanja Fidler, Raquel Urtasun // Computer Vision and Pattern Recognition – CVPR 2017
In this work, we propose a novel way of efficiently localizing a sports field from a single broadcast image of the game. Related work in this area relies on manually annotating a few key frames and extending the localization to similar images, or installing fixed specialized cameras in the stadium…
Hockey action recognition via integrated stacked hourglass network
Mehrnaz Fani, Helmut Neher, David A. Clausi, Alexander Wong, John Zelek // Computer Vision and Pattern Recognition Workshops – CVPRW 2017
A convolutional neural network (CNN) has been designed to interpret player actions in ice hockey video. The hourglass network is employed as the base to generate player pose estimation and layers are added to this network to produce action recognition. As such, the unified architecture is referred…
Classification of Puck Possession Events in Ice Hockey
Moumita Roy Tora, Jianhui Chen, James J. Little // Computer Vision and Pattern Recognition Workshops – CVPRW 2017
Group activity recognition in sports is often challenging due to the complex dynamics and interaction among the players. In this paper, we propose a recurrent neural network to classify puck possession events in ice hockey. Our method extracts features from the whole frame and appearances of the…
Tracking without Appearance Descriptors, Handbook of Pattern Recognition and Computer Vision
Mehrsan Javan Roshtkhari and Martin D. Levine, 2016
Multiple target tracking is still one of the most challenging computer vision problems. In this chapter, we present an algorithm for multiple-object tracking without using object detection, and also provide a framework for including the detection response within a tracking system. We concentrate on…
Discovering human interactions in videos with limited data labeling
Mehran Khodabandeh, Arash Vahdat, Guang-Tong Zhou, Hossein Hajimirsadeghi, Mehrsan Javan Roshtkhari, Greg Mori, Stephen Se // The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2015, pp. 9-18
We present a novel approach for discovering human interactions in videos. Activity understanding techniques usually require a large number of labeled examples, which are not available in many practical cases. Here, we focus on recovering semantically meaningful clusters of human-human and…
Deep structured models for group activity recognition
Zhiwei Deng, Mengyao Zhai, Lei Chen, Yuhao Liu, Srikanth Muralidharan, Mehrsan Javan Roshtkhari, Greg Mori // British Machine Vision Conference – BMVC 2015
This paper presents a deep neural-network-based hierarchical graphical model for individual and group activity recognition in surveillance scenes. Deep networks are used to recognize the actions of individual people in a scene. Next, a neural-network-based hierarchical graphical model refines the…