My Favorite Publications (not necessarily yours) (till 2015)

• Rationality, Optimism and Guarantees in General Reinforcement Learning (with P. Sunehag)
  Journal of Machine Learning Research, 16 (2015) 1345-1390
  [asymptotic optimality of optimistic agents in very large environment classes]
  
  
• Probabilities on Sentences in an Expressive Logic (with J.W. Lloyd & K.S. Ng & W.T.B. Uther)
  Journal of Applied Logic, 11 (2013) 386-420
  [unifies probability and logic for learning]
  
  
• A Monte Carlo AIXI Approximation (with J. Veness & K. S. Ng & W. Uther & D. Silver)
  Journal of Artificial Intelligence Research, 40 (2011) 95-142
  [implementation of Universal AI. AIXI learns to play Poker,Pacman,TicTacToe,...]
  
  
• Feature Reinforcement Learning: Part I: Unstructured MDPs (& Part II: Structured MDPs)
  Journal of Artificial General Intelligence, 1 (2009) 3-24
  [my currently best bet towards practical universal AI]
  
  
• On Universal Prediction and Bayesian Confirmation
  Theoretical Computer Science, 384:1 (2007) 33-48
  [solves the induction problem; recommended]
  
  
• Universal Artificial Intelligence: Sequential Decisions based on Algorithmic Probability
  EATCS Book, Springer, Berlin (2005)
  [first complete and sound theory of AI]
  
  
• Optimality of Universal Bayesian Sequence Prediction for General Loss and Alphabet
  Journal of Machine Learning Research 4 (2003) 971-1000
  [consistency and bounds for non-i.i.d. time-series prediction; very readable]
  
  
• Self-Optimizing and Pareto-Optimal Policies in General Environments based on Bayes-Mixtures
  Proc. 15th Annual Conf. on Computational Learning Theory (COLT-2002) 364-379
  [the core proof of the optimality of the universal AIξ model]
  
  
• The Fastest and Shortest Algorithm for All Well-Defined Problems
  International Journal of Foundations of Computer Science, 13:3 (2002) 431-443
  [asymptotically true; my most popular paper]
  
  
• Family Structure from Periodic Solutions of an Improved Gap Equation (with A. Blumhofer)
  Nuclear Physics B484 (1997) 80-96; B494 (1997) 485
  [solves the mystery of quarks coming in families with exponential mass spectrum]
  
  

Selected Publications (besides my favorites) (till 2021)

• Learning Curve Theory (2021)
  [toy model for explaining/understanding the deep learning scaling laws observed by OpenAI et al.]
  
  
• Universal reinforcement learning algorithms: Survey and experiments (with J. Aslanides & J. Leike)
  Proc. 26th International Joint Conf. on Artificial Intelligence (IJCAI-2017) 1403-1410
  [survey and online demo of history-based Bayesian RL agents]
  
  
• Extreme State Aggregation beyond Markov Decision Processes
  Theoretical Computer Science, 650 (2016) 73-91
  [any reinforcement learning problem can be reduced to a small MDP]
  
  
• Offline to Online Conversion
  Proc. 25th International Conf. on Algorithmic Learning Theory (ALT-2013) 230-244
  [includes combinatorial derivations of famous estimators including Good-Turing's]
  
  
• General Time Consistent Discounting (with T. Lattimore)
  Theoretical Computer Science, 519 (2014) 140-154
  [complete characterization for general discount]
  
  
• Sparse Adaptive Dirichlet-Multinomial-like Processes
  Journal of Machine Learning Research 30, W&CP (COLT-2013) 432-459
  [optimal concentration parameter esp. for large base alphabet]
  
  
• Universal Knowledge-Seeking Agents for Stochastic Environments (with L. Orseau & T. Lattimore)
  Proc. 24th International Conf. on Algorithmic Learning Theory (ALT-2013) 158-172
  [makes AIXI a truly autonomous agent by coupling rewards to information gain]
  
  
• Adaptive Context Tree Weighting (with A. O'Neill & W. Shao & P. Sunehag)
  Proc. Data Compression Conference (DCC-2012) 317-326
  [extension of CTW for non-stationary sources]
  
  
• Discrete MDL Predicts in Total Variation
  Advances in Neural Information Processing Systems 22 (NIPS-2009) 817-825
  [completely general non-iid predictive consistency of countable MDL]
  
  
• Practical Robust Estimators under the Imprecise Dirichlet Model
  International Journal of Approximate Reasoning, 50:2 (2009) 231-242
  [simple, fast, and useful expressions]
  
  
• The Loss Rank Principle for Model Selection
  Proc. 20th Annual Conf. on Learning Theory (COLT-2007) 589-603
  [for (non)parametric non-stochastic regression and classification like kNN]
  
  
• Fitness Uniform Optimization (with S. Legg)
  IEEE Transactions on Evolutionary Computation, 10:5 (2006) 568-589
  [a simple effective selection scheme with no explicit selection pressure]
  
  
• Hybrid Rounding Techniques for Knapsack Problems (with M. Mastrolilli)
  Discrete Applied Mathematics, 154:4 (2006) 640--649
  [a linear!time algorithm for the knapsack problem]
  
  
• Asymptotics of Discrete MDL for Online Prediction (with J. Poland)
  IEEE Transactions on Information Theory, 51:11 (2005) 3780-3795
  [the Minimal Description Length principle for discrete model classes]
  
  
• Adaptive Online Prediction by Following the Perturbed Leader (with J. Poland)
  Journal of Machine Learning Research 6 (2005) 639-660
  [elegant Sqrt(Loss) regret for general loss and adaptive learning rate]
  
  
• Distribution of Mutual Information from Complete and Incomplete Data (with M. Zaffalon)
  Computational Statistics & Data Analysis 48:3 (2005) 633-657
  [simple, fast, and useful expressions]
  
  
• Towards a Universal Theory of Artificial Intelligence based on Algorithmic Probability and Sequential Decisions
  Proc. 12th European Conf. on Machine Learning (ECML-2001) 226-238
  [first publication of the Universal AIXI model]
  
  
• Instantons in QCD: Theory and Application of the Instanton Liquid Model
  Ph.D. Thesis, LMU (1996), hep-ph/0107098
  [nice introduction with some new results, but nothing remarkable]
  
  

Best Paper Awards/Nominations

• Self-modification of policy and utility function in rational agents (with T. Everitt & D. Filan & M. Daswani)
  Proc. 9th Conf. on Artificial General Intelligence (AGI-2016) 1-11
  [Kurzweil prize for best AGI paper]
  
  
• Thompson sampling is asymptotically optimal in general environments (with J. Leike & T. Lattimore & L. Orseau)
  Proc. 32nd International Conf. on Uncertainty in Artificial Intelligence (UAI-2016) 417-426
  [Best student paper]
  
  
• A Monte Carlo AIXI Approximation (with J. Veness & K. S. Ng & W. Uther & D. Silver)
  Journal of Artificial Intelligence Research, 40 (2011) 95-142
  [Honorable mention for the 2014 IJCAI-JAIR best paper prize]
  
  
• Feature Markov decision processes
  Proc. 2nd Conf. on Artificial General Intelligence (AGI-2009) 61-66
  [First runner-up for the Kurzweil best paper prize]
  
  
• Bayesian Regression of Piecewise Constant Functions
  Bayesian Statistics 8 (ISBA-2007) 607-612
  [Lindley prize for innovative research in Bayesian statistics]
  
  

More Publications (+ abstracts, BibTeX, citations, ...)

• Long hyper-linked list of publications in my CV (compact)
• BibTeX with abstracts and links to papers, slides, code, latex, publisher, etc. (recommended)
• ArXiv: Publications + LaTeX source (incomplete after 2012)
• DBLP Bibliography Server: list of publications + coauthor information
• Google Scholar: list of publications + citation information
• ORCID: list of publications
• Semantic Scholar: list of publications + citation information
• Microsoft Academic: list of publications + citation statistics
• Research Gate: list of publications + citation statistics
• Projects and Publications sorted w.r.t. topic
• Old publications in particle physics (QCD, Instantons)
• Outdated publications in computer graphics and others