As we navigate the digital age, we face an ever-growing volume of online streaming data, including recordings from intelligent sensors, transactions from e-commerce platforms, and media content from edge devices. This data expansion will exceed the processing capabilities of computing hardware in the foreseeable future. Therefore, the need for real-time analysis of this data becomes imperative, driven by constraints in decision-making time, hardware capacity, and communication bandwidth.
Simultaneously, with the rise of large generative models, there is an increasing requirement for high-performance GPU clusters, making them less accessible to many research labs or industry sectors. This requires innovative model architectures and computing approaches for efficient model training and deployment.
Our team is addressing critical problems such as the following.
Q1: How to fuse data sources heterogeneous in quality, size, and modality without catastrophic forgetting?
Q2. How can machine learners build “human-like memory” to learn recurring tasks faster and new tasks more easily?
Q3. What is the limit of model compression and how to algorithmically attain that?
Q4. How to leverage low-end GPUs or CPUs to perform large model training and inference?
Representative Work
J. Du, X. Zhang, H. Shen, X. Xian, G. Wang, J. Zhang, Y. Yang, N. Li, J. Liu, J. Ding, “Drift to Remember.” manuscript, 2024. pdf
E. Diao, Q. Le, S. Wu, X. Wang, A. Anwar, J. Ding, V. Tarokh. “ColA: Collaborative Adaptation with Gradient Learning.” manuscript, 2023. pdf
J. Du, Y. Yang, and J. Ding, “Adaptive Continual Learning: Rapid Adaptation and Knowledge Refinement.” manuscript, 2024. pdf
E. Diao, G. Wang, J. Zhang, Y. Yang, J. Ding, V. Tarokh, “Pruning Deep Neural Networks from a Sparsity Perspective.” ICLR, 2023. pdf
X. Tang, J. Zhang, Y. He, X. Zhang, Z. Lin, S. Partarrieu, E. Hanna, Z. Ren, H. Shen, Y. Yang, X. Wang, N. Li, J. Ding, J. Liu. “Explainable Multi-Task Learning for Multi-Modality Biological Data Analysis.” Nature Communications (Editors’ Highlight), 2023. pdf