Gr

Interpretation of published papers

Code, Nr	Citation
1	http://www.cwi.nl/~pdg/fotoduin2a.jpg Peter Grünwald Address CWI Kruislaan 413 1098 SJ Amsterdam The Netherlands Office M266 Telephone 020-5924078 (+31 20 5924124) Telefax 020-5924199 (+31 20 5924199) E-mail pdg@cwi.nl Eh oui, c'est encore lui!
2	http://www.cwi.nl/cwi/departments/AA1.html The name's Peter Grünwald. I am a Ph.D. Student at the Algorithms and Complexity group of the CWI (Dutch Centre for Mathematics and Computer Science), Amsterdam , The Netherlands . My supervisor is prof. Paul Vitányi ; my research project concerns Machine Learning and the Minimum Description Length Principle . Welcome to my home page! It contains some information on me, myself, and I, in relation to My research interests (and running projects) Curriculous Highlights of my Past My publications Somewhat more personal interests You may consider to have a look at my bookmarks for some pointers to pages that are relevant to my research. You can also e-mail me directly). New!!! I have co-organized a mini-conference called Methods for Model Selection . The main intention of the conference was to keep mathematical psychologists up-to-date about recent mathematical developments in the problem of choosing between competing models for the same data. Click here for the conference announcement. Research Interests and Running Projects I belong to the part of the department that is mainly involved in fundamental aspects of machine learning (more about this on CWI's Machine Learning and Genetic Algorithms Project Description Page ).
3	http://www.auai.org/ The projects I am currently involved in are not all about machine learning, but they do share a common theme: non-deductive inference . Also, all my work involvesthe same technical framework: the powerful formalism of Bayesian Networks and Conditional Independence ; for more about this formalism, you may consider the homepage of the Association for Uncertainty in Artificial Intelligence a community that focuses much of its research on Bayesian Networks.
4	#Logic More specifically, I now work on: Machine Learning and Statistics using the MDL Principle. Nonmonotonic Logic and Reasoning about Action (concerning the frame problem and Pearl's theory of causation ). I also used to work on Recurrent Neural Networks though this work has been completed by now. I now consider these three subjects in turn. Links to my papers are here . The Minimum Description Length Principle Most of my time is devoted to studying different aspects, both theoretical and practical, of the Minimimum Description Length (MDL) Principle. Theoretical Considerations On the Learning Side, I am interested in the relations between the MDL Paradigm and other theoretical frameworks of learning, such as the PAC and the statistical physics framework. On the Statistical Side, I am interested in the exact senses in which the MDL Principle is optimal; furthermore, in the connections between MDL, Bayesian Inference and Maximum Entropy methods. In general, I am fascinated by all information-theoretic approaches to non-deductive reasoning. From Theory to Practice If a learning methodology is theoretically optimal, then this should of course be reflected in practice. I am currently spending much of my time together with the CoSCo group of the University of Helsinki on a project in which we apply theoretically optimal methodologies for learning with real-world data. A large part of this work is still theoretical in nature in that it involves finding explicit closed-form formulas for, or well-founded approximations to, the involved quantities. Recently, we have implemented the Naive Bayes classification model and compared its performance on real-world data for prediction using the Bayesian Evidence, the (new definition of) Stochastic Complexity and the classical Maximum Likelihood method. Grammar Inference I have also applied MDL to the computational learning/inferring of grammars from large samples of input text. As for now, mostly samples of natural language are studied (i.e. `corpora'). There are some nice results on using the MDL Principle for the task of classifying words into related classes. This can be seen as a `degenerate' case of grammar learning. This work can be seen as a generalization of several statistical approaches to word classification. In the near future I hope to extend this work to more complex kinds of grammars and other applications (i.e. DNA sequences). Recurrent Neural Networks (this is joint work with psychologist Mark Steijvers of Indiana University ). In recent years, different types of recurrent neural networks have been used to model human cognitive phenomena. We think that if we want to take these networks as serious candidates for psychological models, then we should be able to compare their representational capabilities to other, more traditional, `symbolic' formalisms. Until very recently, the general belief seems to have been that recurrent networks can only represent regular and maybe some context-free languages in an effective way. Recently, we have come up with a very simple recurrent network having only one parameter that is capable of representing the essential properties of some small context-sensitive language that is not context-free. While this does not prove anything about learnability, it does give new insights on what is representable (and how it is represented) by recurrent networks. Nonmonotonic Temporal Reasoning I have developed a new theory of reasoning about action which covers a very broad range of problem classes. (consisting of well-known AI problems like the Frame Problem and the Yale Shooting Problem). Later, I have extended our theory and I have found that what it actually does is applying the Causal Network Theory of Causation (an extension of Bayesian Network theory that has its originis in econometrics and statistics; it is mainly associated with the writings of J.Pearl ). It turns out that our theory can be interpreted as a generalization and/or correction of most existing formalisms, including Chronological Minimization, Filter Preferential Entailment, Baker's solution to the frame problem, Lin's `embrace of causality', Morgenstern & Stein's Motivated Action Theory, McCain & Turner's theory of ramifications and qualifications, Lifschitz & Rabinov's causal minimization and Baral and Gelfond's L3-approach. For some of these formalisms, we have been able to actually prove this; for the other ones, we give a reinterpretation of what they do in terms of causal network theory and we give distinguishing examples for which our theory gives better results then they do). Curriculum Vitae 1970: born in Geldrop, the Netherlands. Austrian Parents; raised bilingually German-Dutch 1988: finishing secondary `Gymnasium Beta' school. Moving to Amsterdam; starting to study Computer Science at the Free University of Amsterdam. 1989: `propaedeutic exam' in Computer Science 1991: `propaedeutic exam' in Mathematics 1991-1993: `Student Assistant' supervising practical work of younger students (PROLOG, Expert Systems, Neural Networks) 1993-1994: 6 Months spent at the IRIT (Institut de Recherche en Informatique de Toulouse) / Universite Paul Sabatier in France 1994: `Doctorandus' (something like Master's Degree) in Computer Science at the Free University of Amsterdam. Specializations: Artificial Intelligence, Theoretical Computer Science. Minor in Cognitive Psychology. 1994: starting work as a Ph.D-student/research assistent at the CWI in Amsterdam. Publications Publications on Learning and MDL
5	ftp://ftp.cwi.nl/pub/pdg/uai98.ps.gz P. Grünwald, P.Kontkanen, P. Myllymäki, T. Silander and H.Tirri, Minimum Encoding Approaches for Predictive Modeling . To appear in Proceedings of the 14th International Conference on Uncertainty in Artificial Intelligence (UAI'98), Madison, WI, USA, July 1998.
6	ftp://ftp.cwi.nl/pub/pdg/jmp.ps P. Grünwald, Model Selection based on Minimum Description Length , to appear in Journal of Mathematical Psychology , special issue on Model Selection, 1998.
7	http://www.cs.Helsinki.FI/research/cosco/Articles/ecml98priors.ps.gz P.Kontkanen, P. Myllymäki, T. Silander, H.Tirri, and P. Grünwald, Bayesian and Information-Theoretic Priors for Bayesian Network Parameters> , Pp. 89-94 in Machine Learning: ECML-98, Proceedings of the 10th European Conference, edited by C.Nédellec and C.Rouveirol. Vol. 1398 in Lecture Notes in Artificial Intelligence, Springer-Verlag, 1998.
8	ftp://ftp.cwi.nl/pub/pdg/benelearn97.ps.Z P.Kontkanen, P. Myllymäki, T. Silander, H.Tirri, and P. Grünwald, On Predictive Distributions and Bayesian Networks , proceedings BeNeLearn '97 (Belgium-Netherlands conference on Machine Learning), Tilburg 1997.
9	http://www.cs.Helsinki.FI/research/cosco/Articles/aistat97.ps.gz P.Kontkanen, P. Myllymäki, T. Silander, H.Tirri, and P. Grünwald, Comparing Predictive Inference Methods for Discrete Domains . Pp. 311-318 in Proceedings of the Sixth International Workshop on Artificial Intelligence and Statistics (Ft. Lauderdale, USA, January 1997).
10	ftp://ftp.cwi.nl/pub/pdg/pdbn.ps.Z P.Kontkanen, P. Myllymäki, T. Silander, H.Tirri, and P. Grünwald, Predictive Distributions and Bayesian Networks . Neurocolt Technical Report NC-TR-97-032, University of Helsinki, Department of Computer Science.
11	ftp://ftp.cwi.nl/pub/pdg/mdlagi.ps.Z P. Grünwald, A Minimum Description Length Approach to Grammar Inference In `Symbolic, Connectionist and Statistical Approaches to Learning for Natural Language Processing' (editors S. Wermter, E. Riloff, G. Scheler),pp 203-216; Lecture Notes in Artificial Intelligence no. 1040. Springer Verlag, Berlin, Germany, 1996.
12	ftp://ftp.cwi.nl/pub/pdg/nonded.ps.Z P. Grünwald, The Minimum Description Length Principle and Non-Deductive Inference Proceedings IJCAI Workshop on Abduction and Induction in AI (editor P. Flach), Nagoya, Japan 1997. The next two papers are a little obsolete (the MDL Principle is not applied in a completely correct manner there) but may still be interesting to read.
13	ftp://ftp.cwi.nl/pub/pdg/old/masters.ps.Z P. Grünwald, Automatic Grammar Induction using the MDL Principle 110 pages, 1994. The author's Master's Thesis, containing a general introduction to language learning by humans and machines, a very detailed description of a new model of grammar inference based on the MDL Principle and a (partially formal) demonstration that the new method is a generalization of several existing grammar inference and word classification algorithms. P. Grünwald, Grammar Inference and the MDL Principle 12 pages, 1994. Paper presented at the BeNeLearn-94 conference. A short summary of the author's Master's Thesis, intended for a rather general audience. Publications on Nonmonotonic Temporal Reasoning
14	http://www.dcs.qmw.ac.uk/conferences/CS98/CS22.ps P. Grünwald, Ramifications and Sufficient Causes . Accepted for presentation at the Fourth Symposium on Logical Formalizations of Common Sense Reasoning (Common Sense '98), London 1998.
15	ftp://ftp.cwi.nl/pub/pdg/ki97.ps.Z P. Grünwald, Causation and Nonmonotonic Temporal Reasoning . In `KI-97: Advances in Artificial Intelligence' (editors G. Brewka, C. Habel and B. Nebel),pp 159-170; Lecture Notes in Artificial Intelligence no. 1303. Springer Verlag, Berlin, Germany, 1997.
16	ftp://ftp.cwi.nl/pub/pdg/nrac97.ps.Z P. Grünwald, Nonmonotonic Temporal Reasoning as a Search for Explanations . NRAC '97 (Second IJCAI Workshop on Nonmonotonic Reasoning, Action and Change), Nagoya, Japan, 1997.
17	ftp://ftp.cwi.nl/pub/pdg/DGWNMR.ps.Z P. Grünwald, Causation, Explanation and Persistence Proceedings 1997 Dutch German Workshop on Nonmonotonic Reasoning, Pp. 149-158, Saarbrücken 1997.
18	ftp://ftp.cwi.nl/pub/pdg/R9709.ps.Z P. Grünwald, The Sufficient Cause Principle and Reasoning About Action CWI Technical Report INS-R9709, november 1997.
19	ftp://ftp.cwi.nl/pub/pdg/naic96.ps.Z P. Grünwald, Causal Networks and Nonmonotonic Temporal Reasoning Proceedings 1996 Dutch Conference on Artificial Intelligence (NAIC-96), Pp. 157-166, nominated for Best Paper Award. Utrecht 1996. The next paper is again rather obsolete:
20	ftp://ftp.cwi.nl/pub/pdg/old/naic95.ps.Z P. Grünwald, B. Gaume and M. Bouajjani A New Causal Theory of Action Proceedings 1995 Dutch Conference on Artificial Intelligence (NAIC-95), Rotterdam 1995. Publications on Recurrent Neural Networks
21	ftp://ftp.cwi.nl/pub/pdg/mark.ps.Z M. Steijvers, P. Grünwald, A Recurrent Network that performs a context-sensitive prediction task . Proceedings Eighteenth Annual Conference of the Cognitive Science Society, Morgan Kauffman, 1996. A sligthly modified version of the Cognitive Science-1996 paper is available as technical report no. NC-TR-94-015 at the Neurocolt ftp site . Personal Interests Just some of the more important ones, not in any particular order: F.E.R.Dy.D.U.R.K.E. and also here . Movies (A Clockwork Orange, The Belly of an Architect, Giullietta degli Spiriti, Amarcord, The Third Man, Apocalypse Now, Once Upon A Time In America, just to name a few) Paintings ( Schiele , Vermeer , Monet , Matisse , Picasso) Good food (& drink!!!) The Mind-Body Problem (philosophy of mind in general, functionalism vs. dualism vs. physicalism vs. many other things. My own point of view, of which I am not sure at all, I baptized 'informationalism'. I think that Dennett, Searle, Penrose, Popper & Eccles, Ryle, Harnad, Minsky, Lucas and all their enemies are all wrong, though they all have very interesting things to say.) Power Politics and the Changing of Societies: though not necessarily supportive of, I am tremendously fascinated by people who have been able to change the system `from within'. So I do not mean the revolutionaries; I mean those who grew up in the traditions of the old system and managed to reach its top, but then started to change the very system that raised them. People like De Klerk, Gorbatshev, De Gaulle (the French nationalist who installed a solid peace with Germany) and to a somewhat lesser extent Deng. Some other keywords are humour, Venice, Tuscany, Mahler's 4th Symphony , Bach's Goldberg Variations, Jacques Brel , Vladimir Nabokov, James Bond, learning French, having a beer outside in front of a canal on a warm summer evening in beloved Amsterdam and many other things... This page has been accessed times since May first 1997 (counted by Siteflow's free counter service ). Last update of this page: december 12th 1997 pdg@cwi.nl . CWI DISCLAIMER