Arjun Ashok

Arjun Ashok

I am a Visiting Researcher (Full-Time) at ServiceNow Research, Montreal and a PhD student at MILA-Quebec AI Institute and CERC-AAI, Université de Montréal advised by Irina Rish and Alexandre Drouin. At ServiceNow, I also work closely with Étienne Marcotte, Valentina Zantedeschi and Nicolas Chapados. My current research interests are in time series forecasting and decision-making, with a focus on designing scalable general-purpose models for time series prediction tasks (forecasting, interpolation, imputation etc).

My email address is arjun.ashok [at] servicenow [dot] com. I'd love to connect and chat with you if we have shared interests, be it in research or music or anything else. Please shoot me an email if you'd like to connect!



Descriptive Alt Text

News

May '24 Presented TACTiS-2 at ICLR 2024. TACTiS-2 is a highly flexible model for multivariate probabilistic time series prediction tasks. Check out the tweet thread and poster here!
Feb '24 The full version of Lag-Llama released with open-source model checkpoints! Check the announcement here!
Jan '24 I gave a talk on our efforts Towards General-Purpose Models for Time-Series Prediction at the Winter 2024 Montreal Time Series Meetup.
Jan '24 TACTiS-2 accepted at ICLR 2024!
Dec '23 I gave a talk on Building Foundation Models for Time Series Data at the 6th workshop on Neural Scaling Laws co-located with NeurIPS 2023.
Oct '23 TACTiS-2 is out on arXiv.
Oct '23 A preliminary version of Lag-Llama is out on arXiv.
Jan '23 One paper on out-of-distribution detection accepted to ICLR 2023. This is work in collaboration with folks at ML Collective mentored by Rosanne Liu.
Jan '23 Started as a Visiting Researcher (Full-Time) at ServiceNow Research, Montreal. Excited to continue working on problems in time series representation learning!
Aug '22 Preliminary work on self-supervised learning objectives for weather time series accepted at the AAAI 2022 Fall Symposium on Climate Change.
Jul '22 One paper on Class-Incremental Learning accepted as a full paper at ECCV 2022.
Jun '22 Started as a Research Intern at IBM Research, India. I'll be working on building self-supervised learning objectives and pre-trained models for geospatial weather time series.
Jun '22 One paper on cross-task generalization in NLP submitted to EMNLP 2022 (Update: Accepted).
Apr '22 One paper on Class-Incremental Learning accepted at the CLVISION Workshop at CVPR 2022 as a non-archival paper (Update: Accepted at ECCV 2022).
Apr '22 One reproducibility report on Self-Supervision and Few-shot Learning accepted at the ML Reproducibility Challenge 2021 (Fall Edition) and published at ReScience-C.
Oct '21 One paper on out-of-distribution generalization accepted as AAAI 2022 as a student abstract.
Jun '21 Started as a Research Assistant at IIT Hyderabad under Prof. Vineeth Balasubramanian.

Latest Papers

TACTiS-2: Better, Faster, Simpler Attentional Copulas for Multivariate Time Series
Arjun Ashok, Étienne Marcotte, Valentina Zantedeschi, Nicolas Chapados, Alexandre Drouin
ICLR 2024

arXiv Code OpenReview Tweet Poster Blog 15-min Video

A flexible model for multivariate probabilistic time series prediction, simplifying the training of attentional copulas, with state-of-the-art accuracy on diverse forecasting tasks, while supporting interpolation and learning from irregular data.
We introduce a new model for multivariate probabilistic time series prediction, designed to flexibly address a range of tasks including forecasting, interpolation, and their combinations. Building on copula theory, we propose a simplified objective for the recently-introduced transformer-based attentional copulas (TACTiS), wherein the number of distributional parameters now scales linearly with the number of variables instead of factorially. The new objective requires the introduction of a training curriculum, which goes hand-in-hand with necessary changes to the original architecture. We show that the resulting model has significantly better training dynamics and achieves state-of-the-art performance across diverse real-world forecasting tasks, while maintaining the flexibility of prior work, such as seamless handling of unaligned and unevenly-sampled time series.
Lag-Llama: Towards Foundation Models for Time Series Forecasting
Kashif Rasul*, Arjun Ashok*, Andrew Robert Williams, Hena Ghonia, Rishika Bhagwatkar, Arian Khorasani, Mohammad Javad Darvishi Bayazi, George Adamopoulos, Roland Riachi, Nadhir Hassen, Marin Biloš, Sahil Garg, Anderson Schneider, Nicolas Chapados, Alexandre Drouin, Valentina Zantedeschi, Yuriy Nevmyvaka, Irina Rish
(* Co-first authorship, equal contribution, order arbitrary)
Preprint. Preliminary work presented at NeurIPS 2023 Workshop on Robustness of Few-shot and Zero-shot Learning in Foundation Models

Paper Code Weights Demo Tweet 15-min Video

A foundation model for probabilistic time series forecasting with strong zero-shot and few-shot capabilities
Over the past years, foundation models have caused a paradigm shift in machine learning due to their unprecedented capabilities for zero-shot and few-shot generalization. However, despite the success of foundation models in modalities such as natural language processing and computer vision, the development of foundation models for time series forecasting has lagged behind. We present Lag-Llama, a general-purpose foundation model for univariate probabilistic time series forecasting based on a decoder-only transformer architecture that uses lags as covariates. Lag-Llama is pretrained on a large corpus of diverse time series data from several domains, and demonstrates strong zero-shot generalization capabilities compared to a wide range of forecasting models on downstream datasets across domains. Moreover, when fine-tuned on relatively small fractions of such previously unseen datasets, Lag-Llama achieves state-of-the-art performance, outperforming prior deep learning approaches, emerging as the best general-purpose model on average. Lag-Llama serves as a strong contender to the current state-of-art in time series forecasting and paves the way for future advancements in foundation models tailored to time series data.

Previous Work

I previously worked on problems in out-of-distribution generalization, continual learning, and few-shot learning, spanning the domains of computer vision and natural language processing. Please check my Google Scholar for a list of previous publications.

Music

I am a Carnatic Vocalist and a student of Vidwan Bharat Sundar. I have performed Carnatic concerts in multiple venues in India, and continue to perform in and around Montréal regularly.
May 2022

Jan 2019

Invited Talks

Academic Service

  • Served as a reviewer at several conferences and journals: ICLR 2024, NeurIPS 2023, AISTATS 2022, AISTATS 2021, CVPR 2022, CVPR 2021, TMLR, TPAMI.
  • Organized the ICLR 2024 Time Series Meetup in Vienna in May 2024.