Biography

Invitation to Participate in a Voluntary Research

Research Topics

• AI Foundations

  to unravel fundamental principles to augment the interpretability and trustworthiness of data-driven decisions

  Foundations of Statistics, Machine Learning, and Quality Evaluation

  I have been interested in addressing foundational questions such as the following.

  Q1. Given a data set and a task, is there any “Limit of Learning”? How to learn that limit (if any)?

  Q2. Is a learned model good enough? Is its performance close to the oracle one could expect (in hindsight)? Note that this cannot be assessed by accuracy (or metrics alike)! Consider, for example, flipping a fair coin. A 50% prediction accuracy is not great, but we cannot further improve it.

  Q3. Which one of the competing machine learning models/procedures is the best? What model selection criterion should one choose–AIC, BIC, or k-fold/leave-one-out cross-validations?

   

  Representative Work

  G. Li, G. Wang, J. Ding, “Provable Identifiability of Two-Layer ReLU Neural Networks via LASSO Regularization,” IEEE Transactions on Information Theory, 2023. pdf

  J. Ding, J. Zhou, and V. Tarokh, “Asymptotically Optimal Prediction for Time-Varying Data Generating Processes,” IEEE Transactions on Information Theory, 2019. pdf

  J. Zhang, J. Ding, Y. Yang, “Targeted Cross-Validation,” Bernoulli, 2022. pdf

  J. Zhang, J. Ding, Y. Yang, “Is a Classification Procedure Good Enough? — A Goodness-of-Fit Assessment Tool for Classification Learning,” Journal of the American Statistical Association, 2022. pdf

  J. Ding, E. Diao, J. Zhou, V. Tarokh, “On Statistical Efficiency in Learning,” IEEE Transactions on Information Theory, 2021. pdf

  J. Ding, V. Tarokh and Y. Yang, “Bridging AIC and BIC: A New Criterion for Autoregression,” IEEE Transactions on Information Theory, 2018. pdf

• Efficient Training and Deployment of Large Models

  to make AI more scalable and accessible to the general public and various industry sectors

  As we navigate the digital age, we face an ever-growing volume of online streaming data, including recordings from intelligent sensors, transactions from e-commerce platforms, and media content from edge devices. This data expansion will exceed the processing capabilities of computing hardware in the foreseeable future. Therefore, the need for real-time analysis of this data becomes imperative, driven by constraints in decision-making time, hardware capacity, and communication bandwidth.

  Simultaneously, with the rise of large generative models, there is an increasing requirement for high-performance GPU clusters, making them less accessible to many research labs or industry sectors. This requires innovative model architectures and computing approaches for efficient model training and deployment.

  Our team is addressing critical problems such as the following.

  Q1: How to fuse data sources heterogeneous in quality, size, and modality without catastrophic forgetting?

  Q2. How can machine learners build “human-like memory” to learn recurring tasks faster and new tasks more easily?

  Q3. What is the limit of model compression and how to algorithmically attain that?

  Q4. How to leverage low-end GPUs or CPUs to perform large model training and inference?

  Representative Work

  E. Diao, G. Wang, J. Zhang, Y. Yang, J. Ding, V. Tarokh, “Pruning Deep Neural Networks from a Sparsity Perspective.” ICLR, 2023. pdf

  X. Tang, J. Zhang, Y. He, X. Zhang, Z. Lin, S. Partarrieu, E. Hanna, Z. Ren, H. Shen, Y. Yang, X. Wang, N. Li, J. Ding, J. Liu. “Explainable Multi-Task Learning for Multi-Modality Biological Data Analysis.” Nature Communications (Editors’ Highlight), 2023. pdf

  J. Du, Y. Yang, and J. Ding, “Adaptive Continual Learning: Rapid Adaptation and Knowledge Refinement.” preprint, 2023. pdf

  E. Diao, Q. Le, S. Wu, X. Wang, A. Anwar, J. Ding, V. Tarokh. “Parameter-Free Fine-Tuning of Large Foundation Models.” preprint, 2023. pdf

• Decentralized and Collaborative AI

  to transcend the limitations of single-machine capabilities by catalyzing machine-to-machine interactions across networks

  Assisted Learning

  Humans develop knowledge from individual studies and joint discussions with peers, even though each individual observes and thinks differently. Likewise, in many emerging application domains, collaborations among organizations or intelligent agents of heterogeneous nature (e.g., different institutes, commercial companies, and autonomous agents) are often essential to resolving challenging problems that are otherwise impossible to be dealt with by a single organization. However, an organization typically enforces stringent security measures to avoid leaking useful and possibly proprietary information, significantly limiting such collaboration. Motivated by these, my team is developing an “Assisted Learning” framework for agents to assist each other in an autonomous, decentralized, and task-adaptive manner. We are also working on an open-source project. Stay tuned!

  Federated Learning

  Federated Learning is a distributed learning framework that enables model training across a multitude of distributed devices. It aggregates locally-computed updates to a shared model, thereby enhancing the model’s performance without directly accessing individual datasets, ensuring data remains on the original device. Key issues involve federated learning under scenarios with hardware and distributional heterogeneity.

   

  Representative Work

  X. Xian, X. Wang, J. Ding, R. Ghanadan, “Assisted Learning: A Framework for Multi-Organization Learning,” NeurIPS, Spotlight Presentation, 2020. pdf

  E. Diao, J. Ding, V. Tarokh, “Gradient Assisted Learning,” NeurIPS, 2022. pdf

  X. Wang, J. Zhang, M. Hong, Y. Yang, J. Ding, “Parallel Assisted Learning,” IEEE Transactions on Signal Processing, 2022. pdf

  H. Chen, J. Ding, E. Tramel, S. Wu, A. Sahu, S. Avestimehr, T. Zhang, “Self-Aware Personalized Federated Learning,” NeurIPS, 2022. pdf

  E. Diao, J. Ding, V. Tarokh, “HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients,” ICLR, 2021. pdf

• AI Safety

  to address emerging societal concerns related to privacy and security in the training and deployment of AI models

  Data Privacy

  Motivated by emerging data privacy issues, my team is working on a new data privacy methodology that aims to address the following challenges:

  Q1. How to alleviate privacy concerns from the data collection stage by developing human-centric private data-collecting interfaces where individuals have a perceptible, transparent, and simple way of sharing sensitive data?
  

  Q2. How to model and infer from vaguely valued data (e.g., subsets, intervals) collected from non-traditional ways due to privacy constraints?

  Model Privacy/Security

  Unlike data privacy which concerns the protection of raw data/identity information, model privacy/security aims to protect an already-learned model that is to be deployed for public use (e.g., via machine-learning-as-a-service platforms). My team is developing foundational concepts and theories of model privacy/security to address the following challenges:

  Q3. How to understand model-stealing attacks and backdoor attacks in a principled manner?

  Q4. How to assess whether a given model is susceptible to adversarial attacks and how to mitigate that?

   

  Representative Work

  G. Wang, X. Xian, X. Bi, J. Srinivasa, A. Kundu, M. Hong, J. Ding, “Demystifying Poisoning Backdoor Attacks from a Statistical Perspective,” ICLR, 2024. pdf

  J. Ding, B. Ding, “Interval Privacy: A Framework for Privacy-Preserving Data Collection,” IEEE Transactions on Signal Processing, 2022. pdf

  X. Xian, G. Wang, J. Srinivasa, A. Kundu, X. Bi, M. Hong, J. Ding, “A Unified Framework for Inference-Stage Backdoor Defenses,” NeurIPS, 2023. pdf

  X. Xian, G. Wang, J. Srinivasa, A. Kundu, X. Bi, M. Hong, J. Ding, “Understanding Backdoor Attacks through the Adaptability Hypothesis,” ICML, 2023. pdf

  X. Wang, Y. Xiang, J. Gao, J. Ding, “Information Laundering for Model Privacy,” ICLR, Spotlight Presentation, 2021. pdf

Current Students

Teaching

•    I will create and teach the topic of “Generative AI: Principles and Practices“, focusing on the study of foundation models, through the following topic course in Fall 2024.
  

  STAT 8931 – Advanced Topics in Statistics (for Ph.D. students in Stat, EE, and CS)

  This course will blend instruction with in-class group research and programming, offering a unique opportunity to bridge fundamental concepts with the latest advancements in AI applications. Registration will open in the Fall 2024 registration period—be sure to mark your calendars and register early to reserve your seat in this exciting course.
  

  The detailed course coverage can be found here.

•    I have taught the following courses:

  PHY/AST/CSCI/STAT 8581 – Big Data in Astrophysics (for Ph.D. students in astrophysics, CS, and Stat)
  STAT 8112 – Mathematical Statistics II (for Ph.D. students in Stat)
  STAT 4052 – Introduction to Statistical Learning (for undergraduate students in Stat)
  STAT 5302 – Applied Regression Analysis (for Master-level students in other schools)
  STAT 5021 – Statistical Analysis (for Master-level students in other schools)

  The detailed course coverage can be found here.

•    I have served on defense committees in Psychology, Mass Communication, Design, Applied Plant Sciences, and Computer Science, in addition to Statistics and Electrical Engineering. You are welcome to contact me if you need my expertise.

Outreach

•    In the summer of 2022, I created a High School Apprenticeship program to promote K-12 science, technology, engineering, and mathematics (STEM) education, with generous support from the Army’s Educational Outreach Program (AEOP). The program recruited talented high school students and provided them with exposure to research conducted by professional scientists and engineers at the University of Minnesota.

  A detailed post can be found here.

Academic Positions

• 20222018

  Tenure-Track Assistant Professor

  University of Minnesota, School of Statistics

• 20182018

  Postdoctoral Researcher

  Duke University, Information Initiative at Duke

• 20172012

  Research Assistant

  Harvard University, School of Engineering and Applied Sciences

Contact

Acknowledgement