Biography

I am an Associate Professor at the School of Statistics, University of Minnesota, with graduate faculty appointments at the Department of Electrical and Computer Engineering and Data Science Program. As a core member of the Data Science Initiative, I am co-developing the AID-H interdisciplinary working group, which focuses on healthcare, data science, and AI. I am also a visiting scholar at the Amazon AGI Team.

My research is at the intersection of AI, statistics, and scientific computing. I joined the University of Minnesota in September 2018 and got early tenure promotion in September 2023. Before that, I received a Ph.D. in Engineering Sciences from Harvard University in 2017 and worked as a post-doctoral fellow at Duke University in 2018. I obtained a B.S. degree from Tsinghua University, where I was selected in the Math & Physics Academic Talent Program (2008-2010) and also enrolled in the Electrical Engineering program (2010-2012).

Research Interest

I like to formulate and address novel research problems that are often inspired by foundational thoughts in statistics, information theory, signal processing, and optimization.  I address those research problems often by establishing new mathematical models, using asymptotic statistics and probability theory, and performing real-world data studies.

As AI rapidly transitions from research labs to a broad spectrum of disciplines and industries, my research focuses on the following interconnected directions:

  • AI Foundations to unravel fundamental principles to augment the interpretability and trustworthiness of data-driven decisions;
  • Efficient Training and Deployment of Large Models to make AI more scalable and accessible to the general public;
  • Decentralized and Collaborative AI to transcend the limitations of single-machine capabilities by catalyzing machine-to-machine interactions across networks;
  • AI Safety to address emerging societal concerns related to privacy, security, and watermarking in the training and deployment of AI models.

More information about these endeavors is included in the sections below. I am deeply grateful for the support received in my research journey, including from the National Science Foundation, Army Research Office, Office of Naval Research, Defense Advanced Research Projects Agency, National Institutes of Health, and Cisco Research. Some special acknowledgments include the NSF CAREER Award, ARO Young Investigator Program (YIP/ECP) Award, Cisco Research Award, Meta Research Award, and Amazon Cloud Credits for Research.

I am offering a new course in Fall 2024 titled STAT8901 Generative AI: Principles and Practices, which explores foundation models and cutting-edge advancements in AI. The course notes are open source and continuously updated here. Accompanying the course, we will leverage a custom web tool, designed to enrich the learning experience for both enrolled students and auditors.

Industry Engagement

We highly value joint work with industry partners. Our recent collaboration with Cisco Research, showcased in this press release, exemplifies our commitment to advancing the field through meaningful partnerships. We welcome discussions with all industry companies on potential collaborations that leverage research insights to address real-world challenges.

 

Click here for a voluntary, remote-based, open-source program that provides a great opportunity to get hands-on experience in cutting-edge R&D of generative AI!

Topics include efficient training of large foundation models such as LLMs and Diffusion Models, compression and deployment of large models, Vector Databases, and various applications to Information Technology, Healthcare, Consumer Products, and Finance.

Research Topics

  • AI Foundations

    to unravel fundamental principles to augment the interpretability and trustworthiness of data-driven decisions

    Foundations of Statistics, Machine Learning, and Quality Evaluation

    I have been interested in addressing foundational questions such as the following.

    Q1. Given a data set and a task, is there any “Limit of Learning”? How to learn that limit (if any)?

    Q2. Is a learned model good enough? Is its performance close to the oracle one could expect (in hindsight)? Note that this cannot be assessed by accuracy (or metrics alike)! Consider, for example, flipping a fair coin. A 50% prediction accuracy is not great, but we cannot further improve it.

    Q3. Which one of the competing machine learning models/procedures is the best? What model selection criterion should one choose–AIC, BIC, or k-fold/leave-one-out cross-validations?

     

    Representative Work

    G. Li, G. Wang, J. Ding, “Provable Identifiability of Two-Layer ReLU Neural Networks via LASSO Regularization,” IEEE Transactions on Information Theory, 2023. pdf

    J. Ding, J. Zhou, and V. Tarokh, “Asymptotically Optimal Prediction for Time-Varying Data Generating Processes,” IEEE Transactions on Information Theory, 2019. pdf

    J. Zhang, J. Ding, Y. Yang, “Targeted Cross-Validation,” Bernoulli, 2022. pdf

    J. Zhang, J. Ding, Y. Yang, “Is a Classification Procedure Good Enough? — A Goodness-of-Fit Assessment Tool for Classification Learning,” Journal of the American Statistical Association, 2022. pdf

    J. Ding, E. Diao, J. Zhou, V. Tarokh, “On Statistical Efficiency in Learning,” IEEE Transactions on Information Theory, 2021. pdf

    J. Ding, V. Tarokh and Y. Yang, “Bridging AIC and BIC: A New Criterion for Autoregression,” IEEE Transactions on Information Theory, 2018. pdf

  • Efficient Training and Deployment of Large Models

    to make AI more scalable and accessible to the general public and various industry sectors

    As we navigate the digital age, we face an ever-growing volume of online streaming data, including recordings from intelligent sensors, transactions from e-commerce platforms, and media content from edge devices. This data expansion will exceed the processing capabilities of computing hardware in the foreseeable future. Therefore, the need for real-time analysis of this data becomes imperative, driven by constraints in decision-making time, hardware capacity, and communication bandwidth.

    Simultaneously, with the rise of large generative models, there is an increasing requirement for high-performance GPU clusters, making them less accessible to many research labs or industry sectors. This requires innovative model architectures and computing approaches for efficient model training and deployment.

    Our team is addressing critical problems such as the following.

    Q1: How to fuse data sources heterogeneous in quality, size, and modality without catastrophic forgetting?

    Q2. How can machine learners build “human-like memory” to learn recurring tasks faster and new tasks more easily?

    Q3. What is the limit of model compression and how to algorithmically attain that?

    Q4. How to leverage low-end GPUs or CPUs to perform large model training and inference?

    Representative Work

    J. Du, X. Zhang, H. Shen, X. Xian, G. Wang, J. Zhang, Y. Yang, N. Li, J. Liu, J. Ding, “Drift to Remember.” manuscript, 2024. pdf

    E. Diao, Q. Le, S. Wu, X. Wang, A. Anwar, J. Ding, V. Tarokh. “ColA: Collaborative Adaptation with Gradient Learning.” manuscript, 2023. pdf

    J. Du, Y. Yang, and J. Ding, “Adaptive Continual Learning: Rapid Adaptation and Knowledge Refinement.” manuscript, 2024. pdf

    E. Diao, G. Wang, J. Zhang, Y. Yang, J. Ding, V. Tarokh, “Pruning Deep Neural Networks from a Sparsity Perspective.” ICLR, 2023. pdf

    X. Tang, J. Zhang, Y. He, X. Zhang, Z. Lin, S. Partarrieu, E. Hanna, Z. Ren, H. Shen, Y. Yang, X. Wang, N. Li, J. Ding, J. Liu. “Explainable Multi-Task Learning for Multi-Modality Biological Data Analysis.” Nature Communications (Editors’ Highlight), 2023. pdf

  • Decentralized and Collaborative AI

    to transcend the limitations of single-machine capabilities by catalyzing machine-to-machine interactions across networks

    Assisted Learning

    Humans develop knowledge from individual studies and joint discussions with peers, even though each individual observes and thinks differently. Likewise, in many emerging application domains, collaborations among organizations or intelligent agents of heterogeneous nature (e.g., different institutes, commercial companies, and autonomous agents) are often essential to resolving challenging problems that are otherwise impossible to be dealt with by a single organization. However, an organization typically enforces stringent security measures to avoid leaking useful and possibly proprietary information, significantly limiting such collaboration. Motivated by these, my team is developing an “Assisted Learning” framework for agents to assist each other in an autonomous, decentralized, and task-adaptive manner. We are also working on an open-source project. Stay tuned!

    Federated Learning

    Federated Learning is a distributed learning framework that enables model training across a multitude of distributed devices. It aggregates locally-computed updates to a shared model, thereby enhancing the model’s performance without directly accessing individual datasets, ensuring data remains on the original device. Key issues involve federated learning under scenarios with hardware and distributional heterogeneity.

     

    Representative Work

    X. Xian, X. Wang, J. Ding, R. Ghanadan, “Assisted Learning: A Framework for Multi-Organization Learning,” NeurIPS, Spotlight Presentation, 2020. pdf

    E. Diao, J. Ding, V. Tarokh, “Gradient Assisted Learning,” NeurIPS, 2022. pdf

    X. Wang, J. Zhang, M. Hong, Y. Yang, J. Ding, “Parallel Assisted Learning,” IEEE Transactions on Signal Processing, 2022. pdf

    H. Chen, J. Ding, E. Tramel, S. Wu, A. Sahu, S. Avestimehr, T. Zhang, “Self-Aware Personalized Federated Learning,” NeurIPS, 2022. pdf

    E. Diao, J. Ding, V. Tarokh, “HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients,” ICLR, 2021. pdf

  • AI Safety

    to address emerging societal concerns related to privacy and security in the training and deployment of AI models

    Data Privacy

    Motivated by emerging data privacy issues, my team is working on a new data privacy methodology that aims to address the following challenges:

    Q1. How to alleviate privacy concerns from the data collection stage by developing human-centric private data-collecting interfaces where individuals have a perceptible, transparent, and simple way of sharing sensitive data?

    Q2. How to model and infer from vaguely valued data (e.g., subsets, intervals) collected from non-traditional ways due to privacy constraints?

    Model Privacy/Security

    Unlike data privacy which concerns the protection of raw data/identity information, model privacy/security aims to protect an already-learned model that is to be deployed for public use (e.g., via machine-learning-as-a-service platforms). My team is developing foundational concepts and theories of model privacy/security to address the following challenges:

    Q3. How to understand model-stealing attacks and backdoor attacks in a principled manner?

    Q4. How to assess whether a given model is susceptible to adversarial attacks and how to mitigate that?

     

    Recent Work

    G. Wang, X. Xian, X. Bi, J. Srinivasa, A. Kundu, M. Hong, J. Ding,  “RAW: A Robust and Agile Plug-and-Play Watermark Framework for AI-Generated Images with Provable Guarantees,” NeurIPS, 2024. pdf

    G. Wang, X. Xian, X. Bi, J. Srinivasa, A. Kundu, C. Fleming,  M. Hong, J. Ding, “On the Vulnerability of Retrieval-Augmented Generation within Knowledge-Intensive Application Domains,” Arxiv, 2024. pdf

    G. Wang, X. Xian, X. Bi, J. Srinivasa, A. Kundu, M. Hong, J. Ding, “Demystifying Poisoning Backdoor Attacks from a Statistical Perspective,” ICLR, 2024. pdf

    J. Ding, B. Ding, “Interval Privacy: A Framework for Privacy-Preserving Data Collection,” IEEE Transactions on Signal Processing, 2022. pdf

    X. Xian, G. Wang, J. Srinivasa, A. Kundu, X. Bi, M. Hong, J. Ding, “A Unified Framework for Inference-Stage Backdoor Defenses,” NeurIPS, 2023. pdf

    X. Xian, G. Wang, J. Srinivasa, A. Kundu, X. Bi, M. Hong, J. Ding, “Understanding Backdoor Attacks through the Adaptability Hypothesis,” ICML, 2023. pdf

    X. Wang, Y. Xiang, J. Gao, J. Ding, “Information Laundering for Model Privacy,” ICLR, Spotlight Presentation, 2021. pdf

Teaching

  •    I am creating a new course of the theme “Generative AI: Principles and Practices“, focusing on the study of foundation models. The course in Fall 2024 will be listed as:

    STAT 8931 – Advanced Topics in Statistics (for Ph.D. students in Stat, EE, CS, etc.)

    This course will blend instruction with in-class group research and programming, offering a unique opportunity to bridge fundamental concepts with the latest advancements in AI applications. Registration will open in the Fall 2024 registration period—be sure to mark your calendars and register early to reserve your seat in this exciting course.

    The course syllabus can be found here. Course lecture notes will be updated here.

  •    I have taught the following courses:

    PHY/AST/CSCI/STAT 8581 – Big Data in Astrophysics (for Ph.D. students in astrophysics, CS, and Stat)
    STAT 8112 – Mathematical Statistics II (for Ph.D. students in Stat)
    STAT 4052 – Introduction to Statistical Learning (for undergraduate students in Stat)
    STAT 5302 – Applied Regression Analysis (for Master-level students in other schools)
    STAT 5021 – Statistical Analysis (for Master-level students in other schools)

    The detailed course coverage can be found here.

  •    I have served on defense committees in Psychology, Mass Communication, Design, Applied Plant Sciences, and Computer Science, in addition to Statistics and Electrical Engineering. You are welcome to contact me if you need my expertise.

Outreach

  •    In the summer of 2022, I created a High School Apprenticeship program to promote K-12 science, technology, engineering, and mathematics (STEM) education, with generous support from the Army’s Educational Outreach Program (AEOP). The program recruited talented high school students and provided them with exposure to research conducted by professional scientists and engineers at the University of Minnesota.

    A detailed post can be found here.

Academic Positions

  • 20242023

    Associate Professor

    University of Minnesota, School of Statistics

  • 20222018

    Tenure-Track Assistant Professor

    University of Minnesota, School of Statistics

  • 20182018

    Postdoctoral Researcher

    Duke University, Information Initiative at Duke

  • 20172012

    Research Assistant

    Harvard University, School of Engineering and Applied Sciences

Current Students

Xun Xian          (EE PhD)
Jiaying Zhou      (Stat PhD)
Jin Du                (Stat PhD)
An Luo               (Stat PhD)
Fangqiao Tian    (Stat PhD)

Harsh Shah         (Stat MS)
Amy Zhang        (High School)

Past Students

Jiawei Zhang       (Stat PhD, Assistant Professor in Stat at the University of Kentucky)
Ganghua Wang   (Stat PhD, Postdoctoral Fellow at the University of Chicago)

Minh Nguyen     (Data Science MS)
Ana Uribe           (Data Science MS)
Colin Ornelas     (Data Science MS)
Tianyang Xie     (Statistics M.S.)
Yilin Hou           (Statistics M.S.)
Eunji Min           (Statistics M.S.)
Yi Rong              (Statistics M.S.)
Suya Wu             (Statistics M.S.)
Yiqing Shen        (Statistics M.S.)

Contact

dingj at umn dot edu

366 Ford Hall

224 Church Street, Minneapolis, MN 55455

Acknowledgement

We are extremely grateful to the following research sponsors for their generous support.