I. Gradient-based Optimization Methods for Machine Learning
1. Advances of Variance Reduction Methods
- SARAH: A Novel Method for Machine Learning Problems Using Stochastic Recursive Gradient. Lam M. Nguyen, Jie Liu, Katya Scheinberg, Martin Takac. The 34th International Conference on Machine Learning (ICML 2017), PMLR volume 70, 2613-2621, 2017. (25% acceptance rate) [PDF] [Poster] [VIDEO] [Bibtex]
- Stochastic Recursive Gradient Algorithm for Nonconvex Optimization. Lam M. Nguyen, Jie Liu, Katya Scheinberg, Martin Takac. Technical report, arXiv preprint, 2017 [PDF] [Bibtex]
- Inexact SARAH Algorithm for Stochastic Optimization. Lam M. Nguyen, Katya Scheinberg, Martin Takac. Optimization Methods and Software (GOMS), volume 36(1), 237-258, 2020. DOI 10.1080/10556788.2020.1818081 [PDF] [Poster] [Bibtex]
- Finite-Sum Smooth Optimization with SARAH. Lam M. Nguyen, Marten van Dijk, Dzung T. Phan, Phuong Ha Nguyen, Tsui-Wei Weng, Jayant R. Kalagnanam. Computational Optimization and Applications (COAP), 2022 [PDF] [Bibtex]
- ProxSARAH: An Efficient Algorithmic Framework for Stochastic Composite Nonconvex Optimization. Nhan H. Pham, Lam M. Nguyen, Dzung T. Phan, Quoc Tran-Dinh. Journal of Machine Learning Research (JMLR), volume 21, 1-48, 2020 [PDF] [CODE] [Bibtex]
- A Hybrid Stochastic Optimization Framework for Composite Nonconvex Optimization. Quoc Tran-Dinh, Nhan H. Pham, Dzung T. Phan, Lam M. Nguyen. Mathematical Programming (MAPR), 2021 [PDF] [Bibtex]
- A Hybrid Stochastic Policy Gradient Algorithm for Reinforcement Learning. Nhan H. Pham, Lam M. Nguyen, Dzung T. Phan, Phuong Ha Nguyen, Marten van Dijk, Quoc Tran-Dinh. The 23rd International Conference on Artificial Intelligence and Statistics (AISTATS 2020), PMLR volume 108, 2020 [PDF] [Bibtex]
- Stochastic Gauss-Newton Algorithms for Nonconvex Compositional Optimization. Quoc Tran-Dinh, Nhan H. Pham, Lam M. Nguyen. The 37th International Conference on Machine Learning (ICML 2020), PMLR volume 119, 2020 (21.8% acceptance rate) [PDF] [Bibtex]
- Hybrid Variance-Reduced SGD Algorithms For Nonconvex-Concave Minimax Problems. Quoc Tran-Dinh, Deyi Liu, Lam M. Nguyen. The 34th Conference on Neural Information Processing Systems (NeurIPS 2020), 2020 (20.1% acceptance rate) [PDF] [Bibtex]
2. Theoretical Aspects of Stochastic Gradient Algorithms
- SGD and Hogwild! Convergence Without the Bounded Gradients Assumption. Lam M. Nguyen, Phuong Ha Nguyen, Marten van Dijk, Peter Richtarik, Katya Scheinberg, Martin Takac. The 35th International Conference on Machine Learning (ICML 2018), PMLR volume 80, 3747-3755, 2018. (25% acceptance rate) [PDF] [Poster] [VIDEO] [Bibtex]
- New Convergence Aspects of Stochastic Gradient Algorithms. Lam M. Nguyen, Phuong Ha Nguyen, Peter Richtarik, Katya Scheinberg, Martin Takac, Marten van Dijk. Journal of Machine Learning Research (JMLR), volume 20(176), 1-49, 2019 [PDF] [Bibtex]
- Characterization of Convex Objective Functions and Optimal Expected Convergence Rates for SGD. Marten van Dijk, Lam M. Nguyen, Phuong Ha Nguyen, Dzung T. Phan. The 36th International Conference on Machine Learning (ICML 2019), PMLR volume 97, 6392-6400, 2019 (22.5% acceptance rate) [PDF] [Bibtex]
- Tight Dimension Independent Lower Bound on the Expected Convergence Rate for Diminishing Step Sizes in SGD. Phuong Ha Nguyen, Lam M. Nguyen, Marten van Dijk. The 33th Conference on Neural Information Processing Systems (NeurIPS 2019), 2019 (21.17% acceptance rate) [PDF] [Bibtex]
- Hogwild! over Distributed Local Data Sets with Linearly Increasing Mini-Batch Sizes. Nhuong V. Nguyen, Toan N. Nguyen, Phuong Ha Nguyen, Quoc Tran-Dinh, Lam M. Nguyen, Marten van Dijk. The 24th International Conference on Artificial Intelligence and Statistics (AISTATS 2021), 2021 (29.8% acceptance rate) [PDF] [Bibtex]
- Proactive DP: A Multiple Target Optimization Framework for DP-SGD. Marten van Dijk, Nhuong Van Nguyen, Toan N. Nguyen, Lam M. Nguyen, Phuong Ha Nguyen. The 41th International Conference on Machine Learning (ICML 2024), 2024 (27.5% acceptance rate) [PDF] [Poster] [Bibtex]
- Stochastic ISTA/FISTA Adaptive Step Search Algorithms for Convex Composite Optimization. Lam M. Nguyen, Katya Scheinberg, Trang H. Tran. Journal of Optimization Theory and Applications (JOTA), 2024 [PDF] [Bibtex]
3. Advances of Shuffling-Type Gradient Methods
- A Unified Convergence Analysis for Shuffling-Type Gradient Methods. Lam M. Nguyen, Quoc Tran-Dinh, Dzung T. Phan, Phuong Ha Nguyen, Marten van Dijk. Journal of Machine Learning Research (JMLR), 2021 [PDF] [Bibtex]
- SMG: A Shuffling Gradient-Based Method with Momentum. Trang H. Tran, Lam M. Nguyen, Quoc Tran-Dinh. The 38th International Conference on Machine Learning (ICML 2021), PMLR volume 139, 2021 (21.47% acceptance rate) [PDF] [Poster] [VIDEO] [Bibtex]
- Nesterov Accelerated Shuffling Gradient Method for Convex Optimization. Trang H. Tran, Katya Scheinberg, Lam M. Nguyen. The 39th International Conference on Machine Learning (ICML 2022), 2022 (21.9% acceptance rate) [PDF] [Poster] [Bibtex]
- On the Convergence to a Global Solution of Shuffling-Type Gradient Algorithms. Lam M. Nguyen, Trang H. Tran. The 37th Conference on Neural Information Processing Systems (NeurIPS 2023), 2023 (26.1% acceptance rate) [PDF] [Bibtex]
- Shuffling Momentum Gradient Algorithm for Convex Optimization. Trang H. Tran, Quoc Tran-Dinh, Lam M. Nguyen. Vietnam Journal of Mathematics (VJOM), Special issue dedicated to Dr. Tamás Terlaky on the occasion of his 70th birthday, 2024 [PDF] [Bibtex]
- Shuffling Gradient-Based Methods for Nonconvex-Concave Minimax Optimization. Quoc Tran-Dinh, Trang H. Tran, Lam M. Nguyen. The 38th Conference on Neural Information Processing Systems (NeurIPS 2024), 2024 (25.8% acceptance rate) [PDF] [Bibtex]
4. On the Convergence to a Global Solution for Deep Learning Applications
- Finite-Sum Optimization: A New Perspective for Convergence to a Global Solution. Lam M. Nguyen, Trang H. Tran, Marten van Dijk. Technical report, arXiv preprint, 2022 [PDF] [Bibtex]
- On the Convergence to a Global Solution of Shuffling-Type Gradient Algorithms. Lam M. Nguyen*, Trang H. Tran*. The 37th Conference on Neural Information Processing Systems (NeurIPS 2023), 2023 (26.1% acceptance rate) [PDF] [Bibtex]
II. Time Series, Dynamical Sytems, and Reinforcement Learning
1. Time Series
- Improving Time Series Encoding with Noise-Aware Self-Supervised Learning and an Efficient Encoder. Anh Duy Nguyen, Trang H. Tran, Hieu H. Pham, Phi Le Nguyen, Lam M. Nguyen. The 24th IEEE International Conference on Data Mining (ICDM 2024), 2024 (19.5% acceptance rate) [PDF] [Bibtex]
- Abstracted Shapes as Tokens - A Generalizable and Interpretable Model for Time-series Classification. Yunshi Wen, Tengfei Ma, Tsui-Wei Weng, Lam M. Nguyen, Anak Agung Julius. The 38th Conference on Neural Information Processing Systems (NeurIPS 2024), 2024 (25.8% acceptance rate) [PDF] [Bibtex]
- An End-to-End Time Series Model for Simultaneous Imputation and Forecast. Trang H. Tran, Lam M. Nguyen, Kyongmin Yeo, Nam Nguyen, Dzung Phan, Roman Vaculin, Jayant Kalagnanam. Technical report, arXiv preprint, 2023 [PDF] [Bibtex]
- Correlated Attention in Transformers for Multivariate Time Series. Quang Minh Nguyen, Lam M. Nguyen, Subhro Das. Technical report, arXiv preprint, 2023 [PDF] [Bibtex]
- A Supervised Contrastive Learning Pretrain-Finetune Approach for Time Series. Trang H. Tran, Lam M. Nguyen, Kyongmin Yeo, Nam Nguyen, Roman Vaculin. Technical report, arXiv preprint, 2023 [PDF] [Bibtex]
2. Dynamical Systems for Machine Learning
- ConCerNet: A Contrastive Learning Based Framework for Automated Conservation Law Discovery and Trustworthy Dynamical System Prediction. Wang Zhang, Tsui-Wei Weng, Subhro Das, Alexandre Megretski, Luca Daniel, Lam M. Nguyen. The 40th International Conference on Machine Learning (ICML 2023), 2023 (27.9% acceptance rate) [PDF] [Poster] [Bibtex]
- One Step Closer to Unbiased Aleatoric Uncertainty Estimation. Wang Zhang, Martin Ma, Subhro Das, Lily Weng, Alexandre Megretsky, Luca Daniel, Lam M. Nguyen. The 38th AAAI Conference on Artificial Intelligence (AAAI 2024), 2024 (23.75% acceptance rate) [PDF] [Bibtex]
- Guaranteeing Conservation Laws with Projection in Physics-Informed Neural Networks. Anthony Baez, Wang Zhang, Ziwen Ma, Subhro Das, Lam M. Nguyen, Luca Daniel . The 38th Conference on Neural Information Processing Systems (NeurIPS 2024), Data-driven and Differentiable Simulations, Surrogates, and Solvers (D3S3), 2024 [PDF] [Bibtex]
3. Stability of Dynamical Systems
- A Service System with Randomly Behaving On-demand Agents. Lam M. Nguyen, Alexander L. Stolyar. The 42nd ACM International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS 2016), ACM SIGMETRICS Performance Evaluation Review, volume 44, issue 1, 365-366, 2016. DOI 10.1145/2964791.2901484 (25% acceptance rate) [PDF] [Poster] [Bibtex]
- A Queueing System with On-demand Servers: Local Stability of Fluid Limits. Lam M. Nguyen, Alexander L. Stolyar. Queueing Systems (QUESTA), volume 89, issue 3-4, 243-268, 2018, (Springer). DOI 10.1007/s11134-017-9564-8 [PDF] [Bibtex]
4. Reinforcement Learning
- A Hybrid Stochastic Policy Gradient Algorithm for Reinforcement Learning. Nhan H. Pham, Lam M. Nguyen, Dzung T. Phan, Phuong Ha Nguyen, Marten van Dijk, Quoc Tran-Dinh. The 23rd International Conference on Artificial Intelligence and Statistics (AISTATS 2020), PMLR volume 108, 2020 [PDF] [Bibtex]
- Fast Convergence for Unstable Reinforcement Learning Problems by Logarithmic Mapping. Wang Zhang, Lam M. Nguyen, Subhro Das, Alexandre Megretski, Luca Daniel, Tsui-Wei Weng. The 39th International Conference on Machine Learning (ICML 2022), Decision Awareness in Reinforcement Learning, 2022 [PDF] [Bibtex]
- c-MBA: Adversarial Attack for Cooperative MARL Using Learned Dynamics Model. Nhan H. Pham, Lam M. Nguyen, Jie Chen, Hoang Thanh Lam, Subhro Das, Tsui-Wei Weng. The 36th Conference on Neural Information Processing Systems (NeurIPS 2022), ML Safety, 2022 [PDF] [Bibtex]
- Attacking c-MARL More Effectively: A Data Driven Approach. Nhan H. Pham, Lam M. Nguyen, Jie Chen, Hoang Thanh Lam, Subhro Das, Tsui-Wei Weng. The 23rd IEEE International Conference on Data Mining (ICDM 2023), 2023 (19.94% acceptance rate) [PDF] [Bibtex]
III. Applications
1. Other Optimization Methods and Models for Machine Learning
- Pruning Deep Neural Networks with L0-constrained Optimization. Dzung T. Phan, Lam M. Nguyen, Nam H. Nguyen, Jayant R. Kalagnanam. The 20th IEEE International Conference on Data Mining (ICDM 2020), 2020 (19.7% acceptance rate) [PDF] [Bibtex]
- A Scalable MIP-based Method for Learning Optimal Multivariate Decision Trees. Haoran Zhu, Pavankumar Murali, Dzung T. Phan, Lam M. Nguyen, Jayant R. Kalagnanam. The 34th Conference on Neural Information Processing Systems (NeurIPS 2020), 2020 (20.1% acceptance rate) [PDF] [Bibtex]
- Regression Optimization for System-level Production Control. Dzung T. Phan, Lam M. Nguyen, Pavankumar Murali, Nhan H. Pham, Hongsheng Liu, Jayant R. Kalagnanam. The 2021 American Control Conference (ACC 2021), 2021 [PDF] [Bibtex]
- Ensuring the Quality of Optimization Solutions in Data Generated Optimization Models. Segev Wasserkrug, Orit Davidovith, Evgeny Shindin, Dharmashankar Subramanian, Parikshit Ram, Pavankumar Murali, Dzung Phan, Nianjun Zhou, Lam M. Nguyen. The 30th International Joint Conference on Artificial Intelligence (IJCAI 2021), Data Science Meets Optimisation, DSO@IJCAI2021, 2021 [PDF] [Bibtex]
- Interpretable Clustering via Multi-Polytope Machines. Connor Lawless, Jayant Kalagnanam, Lam M. Nguyen, Dzung Phan, Chandra Reddy. The 36th AAAI Conference on Artificial Intelligence (AAAI 2022), 2022 (15% acceptance rate) [PDF] [Bibtex]
- AI-based Real-time Site-wide Optimization for Process Manufacturing. Jayant Kalagnanam, Dzung Phan, Pavankumar Murali, Lam M. Nguyen, Nianjun Zhou, Dharmashankar Subramanian, Raju Pavuluri, Xiang Ma, Crystal Lui, Giovane Cesar da Silva. INFORMS Journal on Applied Analytics (IJAA), 2022 [PDF] [Bibtex]
- StepDIRECT - A Derivative-Free Optimization Method for Stepwise Functions. Dzung Phan, Hongsheng Liu, Lam M. Nguyen. SIAM International Conference on Data Mining (SDM22), 2022 (27.8% acceptance rate) [PDF] [Bibtex]
- Addressing Solution Quality in Data Generated Optimization Models. Orit Davidovich, Parikshit Ram, Segev Wasserkrug, Dharmashankar Subramanian, Nianjun Zhou, Dzung Phan, Pavankumar Murali, Lam M. Nguyen. The 36th AAAI Conference on Artificial Intelligence (AAAI 2022), AI for Decision Optimization, AI4DO, 2022 [PDF] [Bibtex]
- Automated Decision Optimization: Data and Knowledge Driven Optimization Model Generation with Human-in-the-loop. Lisa Amini, Arunima Chaudhary, Yishai Feldman, Pavankumar Murali, Lam M. Nguyen, Dzung Phan, Aviad Sela, Carolina Spina, Dharmashankar Subramanian, Abel Valente, Long Vu, Dakuo Wang, Segev Wasserkrug, Ritesh Yadav, Nianjun Zhou. The 36th AAAI Conference on Artificial Intelligence (AAAI 2022), AI for Decision Optimization, AI4DO, 2022 [PDF] [Bibtex]
- Optimal Control via Linearizable Deep Learning. Vinicius Lima, Dzung T. Phan, Lam M. Nguyen, Jayant R. Kalagnanam. The 2023 American Control Conference (ACC 2023), 2023 [PDF] [Bibtex]
- Multi-polytope Machine for Classification. Dzung Phan, Lam M. Nguyen, Jayant Kalagnanam, Chandra Reddy. SIAM Conference on Data Mining (SDM 2024), 2024 (29.2% acceptance rate) [PDF] [Bibtex]
2. Optimal Transport
- On Unbalanced Optimal Transport: Gradient Methods, Sparsity and Approximation Error. Quang Minh Nguyen, Hoang H. Nguyen, Yi Zhou, Lam M. Nguyen. Journal of Machine Learning Research (JMLR), 2023 [PDF] [Bibtex]
- On Partial Optimal Transport: Revising the Infeasibility of Sinkhorn and Efficient Gradient Methods. Anh Duc Nguyen, Tuan Dung Nguyen, Quang Nguyen, Hoang Nguyen, Lam M. Nguyen, Kim-Chuan Toh. The 38th AAAI Conference on Artificial Intelligence (AAAI 2024), 2024 (23.75% acceptance rate) [PDF] [Bibtex]
3. Trustworthy and Explainability in AI/ML
- PROVEN: Verifying Robustness of Neural Networks with a Probabilistic Approach. Tsui-Wei Weng, Pin-Yu Chen, Lam M. Nguyen, Mark S. Squillante, Akhilan Boopathy, Ivan Oseledets, Luca Daniel. The 36th International Conference on Machine Learning (ICML 2019), PMLR volume 97, 6727-6736, 2019 (22.5% acceptance rate) [PDF] [Bibtex]
- On the Equivalence between Neural Network and Support Vector Machine. Yilan Chen, Wei Huang, Lam M. Nguyen, Tsui-Wei Weng. The 35th Conference on Neural Information Processing Systems (NeurIPS 2021), 2021 (26% acceptance rate) [PDF] [Bibtex]
- Besting the Black-Box: Barrier Zones for Adversarial Example Defense. Kaleel Mahmood, Phuong Ha Nguyen, Lam M. Nguyen, Thanh Nguyen, Marten van Dijk. IEEE Access, 2022 [PDF] [Bibtex]
- Robust Randomized Smoothing via Two Cost-Effective Approaches. Linbo Liu, Trong Nghia Hoang, Lam M. Nguyen, Tsui-Wei Weng. The 10th International Conference on Learning Representations (ICLR 2022), PAIR2Struct: Privacy, Accountability, Interpretability, Robustness, Reasoning on Structured Data, 2022 [PDF] [Bibtex]
- c-MBA: Adversarial Attack for Cooperative MARL Using Learned Dynamics Model. Nhan H. Pham, Lam M. Nguyen, Jie Chen, Hoang Thanh Lam, Subhro Das, Tsui-Wei Weng. The 36th Conference on Neural Information Processing Systems (NeurIPS 2022), ML Safety, 2022 [PDF] [Bibtex]
- Label-free Concept Bottleneck Models. Tuomas Oikarinen, Subhro Das, Lam M. Nguyen, Tsui-Wei Weng. The 11th International Conference on Learning Representations (ICLR 2023), 2023 [PDF] [Bibtex]
- Promoting Robustness of Randomized Smoothing: Two Cost-Effective Approaches. Linbo Liu, Trong Nghia Hoang, Lam M. Nguyen, Tsui-Wei Weng. The 23rd IEEE International Conference on Data Mining (ICDM 2023), 2023 (19.94% acceptance rate) [PDF] [Bibtex]
- Attacking c-MARL More Effectively: A Data Driven Approach. Nhan H. Pham, Lam M. Nguyen, Jie Chen, Hoang Thanh Lam, Subhro Das, Tsui-Wei Weng. The 23rd IEEE International Conference on Data Mining (ICDM 2023), 2023 (19.94% acceptance rate) [PDF] [Bibtex]
4. Distributed and Federated Learning
- Hogwild! over Distributed Local Data Sets with Linearly Increasing Mini-Batch Sizes. Nhuong V. Nguyen, Toan N. Nguyen, Phuong Ha Nguyen, Quoc Tran-Dinh, Lam M. Nguyen, Marten van Dijk. The 24th International Conference on Artificial Intelligence and Statistics (AISTATS 2021), 2021 (29.8% acceptance rate) [PDF] [Bibtex]
- FedDR -- Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimization. Quoc Tran-Dinh, Nhan H. Pham, Dzung T. Phan, Lam M. Nguyen. The 35th Conference on Neural Information Processing Systems (NeurIPS 2021), 2021 (26% acceptance rate) [PDF] [CODE] [Bibtex]
- Scalable and Secure Federated XGBoost. Quang Nguyen, Nhan Khanh Le, Lam M. Nguyen. The 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2023), 2023 [PDF] [Bibtex]
- Federated Learning: Theory and Practice. Lam M. Nguyen, Trong Nghia Hoang, Pin-Yu Chen. Elsevier publisher, ISBN 9780443190377, 2024 [Bibtex]
- Proactive DP: A Multiple Target Optimization Framework for DP-SGD. Marten van Dijk, Nhuong Van Nguyen, Toan N. Nguyen, Lam M. Nguyen, Phuong Ha Nguyen. The 41th International Conference on Machine Learning (ICML 2024), 2024 (27.5% acceptance rate) [PDF] [Bibtex]
- Probabilistic Federated Prompt-Tuning in Data Imbalance Settings. Pei-Yau Weng, Minh Hoang, Lam M. Nguyen, My T. Thai, Tsui-Wei Weng, Trong Nghia Hoang. The 38th Conference on Neural Information Processing Systems (NeurIPS 2024), 2024 (25.8% acceptance rate) [PDF] [Bibtex]
5. Natural Language Processing
- Ensembling Graph Predictions for AMR Parsing. Thanh Lam Hoang, Gabriele Picco, Yufang Hou, Young-Suk Lee, Lam M. Nguyen, Dzung T. Phan, Vanessa López, Ramon Fernandez Astudillo. The 35th Conference on Neural Information Processing Systems (NeurIPS 2021), 2021 (26% acceptance rate) [PDF] [Bibtex]