Johan Sokrates Wind

Johan Sokrates Wind is a Research Fellow at the University of Oslo's Department of Mathematics, specializing in Differential Equations and Computational Mathematics. His primary affiliation is with the Faculty of Mathematics and Natural Sciences. He holds a Master's in Industrial Mathematics from the Norwegian University of Science and Technology (NTNU) and began his PhD in August 2021. Wind's research focuses on deep learning, neural networks, and overparameterized machine learning systems. He has explored topics such as the Neural Tangent Kernel, Deep Linear Networks, and implicit biases in optimization algorithms. His work bridges theoretical analysis with practical implementations, as evidenced by his blog The Good Minima, where he publishes technical insights on neural network behavior and training dynamics. Notably, he has contributed to projects like real-time visual odometry on smartphones during his part-time role at Arm Ltd. Wind is an active participant in competitive programming and Kaggle competitions, showcasing his problem-solving skills and algorithmic expertise.

His research emphasizes analytically tractable models and the mathematical foundations of modern AI systems. Recent investigations include the RWKV language model architecture, efficient CIFAR-10 classification, and the role of initialization and learning rates in SGD's implicit bias. Wind’s publications highlight interdisciplinary approaches, combining elements of optimization theory, computational mathematics, and applied machine learning.

He maintains an active blog with detailed technical posts, demonstrating a commitment to open science and knowledge-sharing. His academic journey reflects a balance between theoretical rigor and practical innovation, positioning him as a rising researcher in computational and mathematical aspects of deep learning.