Chulhee Yun

Chulhee Yun

Assistant Professor

Kim Jaechul Graduate School of AI

KAIST

My name is Chulhee (I go by Charlie), and I am an assistant professor at KAIST Kim Jaechul Graduate School of AI (KAIST AI). I direct the Optimization & Machine Learning (OptiML) Laboratory at KAIST AI.

I finished my PhD from the Laboratory for Information and Decision Systems at Massachusetts Institute of Technology, where I was fortunate to study under the joint supervision of Prof. Suvrit Sra and Prof. Ali Jadbabaie. Before MIT, I was a master’s student in Electrical Engineering at Stanford University, where I worked with Prof. John Duchi. I finished my undergraduate program in Electrical Engineering at KAIST.

Email: {firstname}.{lastname}@kaist.ac.kr
Phone: +82-2-958-3919
Office: KAIST Seoul Campus Building #9, 9401


For prospective students: If you are curious about what kinds of research I do, please see this interview article (in Korean). I look for self-motivated graduate students with strong mathematical backgrounds. If you are an undergraduate student interested in interning at our lab, consider applying for summer/winter KAIST AI Research Internship (KAIRI) programs.


Interests
  • Deep Learning Theory
  • Optimization
  • Machine Learning Theory
Education
  • PhD in Elec. Eng. & Comp. Sci., 2016–2021

    Massachusetts Institute of Technology

  • MSc in Electrical Engineering, 2014–2016

    Stanford University

  • BSc in Electrical Engineering, 2007–2014

    KAIST

News

[Mar 2024] One paper was accepted to the ICLR 2024 Workshop on Bridging the Gap Between Practice and Theory in Deep Learning. We prove that alternating gradient descent-ascent (GDA) converges faster than simultaneous GDA, and propose an even faster variant!
[Jan 2024] Our paper on the optimization characteristics of linear Transformers got accepted to ICLR 2024!
[Jan 2024] I had the honor to take part in the Fifth Korean-American Kavli Frontiers of Science Symposium as a session speaker.
[Oct 2023] Two papers got accepted to the NeurIPS 2023 Workshop on Mathematics of Modern Machine Learning (M3L)! (update: both papers were selected for oral presentations!)
[Sep 2023] Four papers got accepted to NeurIPS 2023; among them, our SAM analysis paper was selected for a spotlight presentation! Congratulations and thank you to my co-authors!

Publications

Fundamental Benefit of Alternating Updates in Minimax Optimization  arXiv
arXiv preprint, early version at ICLR 2024 Workshop on Bridging the Gap Between Practice and Theory in Deep Learning
Large Catapults in Momentum Gradient Descent with Warmup: An Empirical Study  arXiv
arXiv preprint, early version at NeurIPS 2023 Workshop on Mathematics of Modern Machine Learning (Oral)
Linear attention is (maybe) all you need (to understand transformer optimization)  arXiv
ICLR 2024, short version at NeurIPS 2023 Workshop on Mathematics of Modern Machine Learning (Oral)
Fair Streaming Principal Component Analysis: Statistical and Algorithmic Viewpoint  arXiv
NeurIPS 2023
PLASTIC: Improving Input and Label Plasticity for Sample Efficient Reinforcement Learning  arXiv
NeurIPS 2023
Practical Sharpness-Aware Minimization Cannot Converge All the Way to Optima  arXiv
NeurIPS 2023 (Spotlight)
Outstanding Paper Award at KAIA Conference 2023 (CKAIA 2023)
Tighter Lower Bounds for Shuffling SGD: Random Permutations and Beyond  Paper arXiv
ICML 2023 (Oral)
SGDA with shuffling: faster convergence for nonconvex-PŁ minimax optimization  Paper arXiv
ICLR 2023
NAVER Outstanding Theory Paper Award at KAIA-NAVER Joint Conference 2022 (JKAIA 2022)
Minibatch vs Local SGD with Shuffling: Tight Convergence Bounds and Beyond  Paper arXiv
ICLR 2022 (Oral)
Provable Memorization via Deep Neural Networks using Sub-linear Parameters  Paper arXiv
COLT 2021
Presented as part of a contributed talk at DeepMath 2020
A Unifying View on Implicit Bias in Training Linear Neural Networks  Paper arXiv
ICLR 2021, short version at NeurIPS 2020 Workshop on Optimization for Machine Learning (OPT 2020)
Minimum Width for Universal Approximation  Paper arXiv
ICLR 2021 (Spotlight)
Presented as part of a contributed talk at DeepMath 2020
$O(n)$ Connections are Expressive Enough: Universal Approximability of Sparse Transformers  Paper arXiv
NeurIPS 2020
Low-Rank Bottleneck in Multi-head Attention Models  Paper arXiv
ICML 2020
Are Transformers universal approximators of sequence-to-sequence functions?  Paper arXiv
ICLR 2020, short version at NeurIPS 2019 Workshop on Machine Learning with Guarantees
Honorable Mention at NYAS Machine Learning Symposium 2020 Poster Awards
Are deep ResNets provably better than linear predictors?  Paper arXiv
NeurIPS 2019
Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity  Paper arXiv
NeurIPS 2019 (Spotlight)
Minimax Bounds on Stochastic Batched Convex Optimization  Paper
COLT 2018
Global optimality conditions for deep neural networks  Paper arXiv
ICLR 2018, short version at NIPS 2017 Workshop on Deep Learning: Bridging Theory and Practice
Face detection using Local Hybrid Patterns  Paper
ICASSP 2015

Teaching

AI709 Advanced Deep Learning Theory (2024S)
AI616 Deep Learning Theory (2022S/F, 2023S/F)

Research Group

I direct the Optimization & Machine Learning (OptiML) Laboratory at KAIST. I ambitiously pronounce it as the “Optimal Lab”—although my students may disagree!

PhD Students (all students are in KAIST AI)
Master’s Students (all students are in KAIST AI)
Undergraduate Students/KAIRI Interns
  • Hyunji Jung (POSTECH Math/CS)
  • ChangMin Kang (KAIST Math)
  • Donghwa Kim (KAIST Math)
  • Yujun Kim (KAIST Math/CS)
  • Chaewon Moon (SNU IE/Math)
  • Minhak Song (KAIST ISysE/Math)
Former Graduate Students

Service

Conference Area Chair
  • NeurIPS 2023
Conference/Workshop Reviewer
  • ICLR 2019–2024
  • ICML 2019–2024
  • COLT 2020–2024
  • NeurIPS 2018–2020, 2022
  • AISTATS 2019
  • CDC 2018
  • ICLR 2024 Privacy Regulation and Protection in Machine Learning Workshop
Journal Reviewer
  • Journal of Machine Learning Research
  • SIAM Journal on Mathematics of Data Science
  • Annals of Statistics
  • IEEE Transactions on Neural Networks and Learning Systems
  • IEEE Transactions on Information Theory
  • Mathematical Programming
(Co-)Organizer
  • Mathematical Theory of AI Sessions at KAIA-NAVER Joint Conference 2022
  • The LIDS & Stats Tea Talk Series Fall 2019–Spring 2020
  • The 24th Annual LIDS Student Conference 2019