Yefan Zhou

Hi, I'm Yefan. I am a CS PhD candidate (2023- ) at Dartmouth College advised by Prof. Yaoqing Yang. I am a researcher at UC Berkeley/ICSI working with Prof. Michael Mahoney.
I earned my Master's degree in EECS at UC Berkeley.

Scholar  /   /   /  Linkedin  /  Email  /  Resume

I'm interested in improving efficiency and transparency of machine learning models. My current research is focused on model diagnosis, utilizing high-dimension features such as loss landscapes, weight matrix analysis. This research contributes to

• Neural network optimization ([TempBalance]), LLM fine-tuning ([TempBalance_v2]), Ensembling ([SharpBalance]), LLM MoE ([AlphaExpert])
• LLM pruning ([AlphaPruning]), Model pruning ([ThreeRegime_Prune])
• Post-training diagnosis and hyperparameter tuning ([MD_tree])

[NN optimizer, Efficient training, Layer quality analysis]
We introduce a new learning rate scheduler TempBalance that analyzes the layer-wise training quality via "diagnosis" and dynamically adjusts learning rates per layer. This method quantifies and utilizes the heavy-tailed structure of the weight matrix eigenspectrum.

[LLM pruning, Efficient inference]
We introduce a LLM pruning method AlphaPruning, that assigns the layer-wise sparsities to pre-trained LLM based on weight-analysis based metrics.

[Ensembling, Out-of-distribution, Loss landscape analysis]
We discover sharpness-diversity trade-off: minimizing the sharpness in the loss landscape tends to diminish the diversity of members within the ensemble, adversely affecting the ensemble's improvement. We introduce a new ensemble training method called SharpBalance to address it.

[(Post-training) model diagnosis, Model selection, Scaling law, Hyperparameter tuning]
This work studies how to predict the source of model failure from a set of failure modes (such as a wrong hyperparameter, inadequate model size) without knowing the training configuration of the pre-trained NN. We propose a diagnosis method called MD tree based on loss landscape metrics and demonstrate its advantage over classical validation-based approaches.

[NN pruning, Model selection, Losslandscape analysis]
The study identifies a transition phenomenon in neural network pruning, where the effect of increasing the temperature-like parameter (e.g. training epochs) depends on the value of the load-like parameter (e.g. pruning ratio), leading to different pruning outcomes. The findings are then applied to three practical scenarios, including optimizing hyperparameters for improved pruning and selecting the most suitable model for pruning.

[Dataset diagnosis, 3D reconstruction]
A SVR model can be disposed towards recognition (classification-based) or reconstruction depending on how dispersed the training data becomes.
We propose "dispersion score", which is a data-driven metric used to measure the tendency of SVR models to perform recognition or reconstruction. It can also be used to diagnose problems from the training data and guide the design of data augmentation schemes.

[LLM fine-tuning, Layer quality analysis]
We show that optimizer TempBalance effectively improves LLM fine-tuning, especially in the cases of low downstream data.

[LLM efficient fine-tuning, Mixture-of-expert]
We introduced a new LoRA-MoE fine-tuning approach called AlphaExpert, which utilizes the layer-wise weight analysis to assign expert numbers.

Empirical grasping datasets are typically sparsely labeled (i.e., a small number of successful grasp labels in each image).
We propose a maximum likelihood grasp sampling loss (MLGSL) for learning robotic grasping from sparsely labeled datasets.
MLGSL is 8× more data-efficient than SOTA with a 91.8% grasp success rate in real-world experiments.

ICLR 2024-2025, CVPR 2024-2025, ICML 2024, AAAI 2025, NeurIPS 2023, IROS 2022, TMLR, CAPL 2024,