This week in The History of AI at AIWS.net – David Rumelhart, Geoffrey Hinton, and Ronald Williams published “Learning representations by back-propagating errors”

by Admin | Oct 16, 2020 | Chronicles, News

This week in The History of AI at AIWS.net – David Rumelhart, Geoffrey Hinton, and Ronald Williams published “Learning representations by back-propagating errors” in October 1986. In this paper, they describe “a new learning procedure, back-propagation, for networks of neurone-like units.” The term backpropagation was introduced in this paper, and the concept of it was also introduced to neural networks. The paper can be found here.

David E. Rumelhart was an American psychologist. He is notable for his contributions to the study of human cognition, in terms of mathematical psychology, symbolic artificial intelligence, and connectionism. At the time of publication of the paper (1986), he was a Professor at the Department of Psychology at University of California, San Diego. In 1987, he then moved to Stanford, becoming Professor there until 1998. Rumelhart also received the MacArthur Fellowship in 1987, and was elected to the National Academy of Sciences in 1991.

Geoffrey Hinton is an English-Canadian cognitive psychologist and computer scientist. He is most notable for his work on neural networks. He is also known for his work into Deep Learning. Hinton, along with Yoshua Bengio and Yann LeCun (who was a postdoctorate student of Hinton), are considered the “Fathers of Deep Learning”. They were awarded the 2018 ACM Turing Award, considered the Nobel Prize of Computer Science, for their work on deep learning. 

Ronald Williams is a computer scientist and a pioneer into neural networks. He is a Professor of Computer Science at Northeastern University. He was an author on the paper “Learning representations by back-propagating errors”, and he also made contributions to recurrent neural networks and reinforcement learning. 

The History of AI Initiative considers this paper important because it introduces backpropagandation. Furthermore, the paper created a boom in research into neural network, a component of AI. Geoffrey Hinton, one of the authors of the paper, would also go on and play an important role in Deep Learning, which is a field of Machine Learning, part of Artificial Intelligence.

This week in The History of AI at AIWS.net – “Computing Machinery and Intelligence” by Alan Turing was published

by Admin | Oct 9, 2020 | Chronicles, News

This week in The History of AI at AIWS.net – “Computing Machinery and Intelligence” by Alan Turing was published in the Mind quarterly academic journal in October 1950. It was the first instance that the “Turing test” was introduced to the public. The paper takes the question “Can machines think” and breaks it down. The paper also addresses 9 objections and arguments against Artificial Intelligence – Religious, “Heads in the Sands”, Mathematical, etc. Turing wrote about a potential “Learning Machine” that could successfully bypass the Turing test.

The Turing test, also known as the Imitation game, can be used to tell machines from humans. It poses a hypothetical, where a human evaluator would judge conversations between a machine designed for human-like responses and a human; if the evaluator cannot identify the machine from the human, then the machine passed the test. The test has proven to be both influential and controversial.

Alan Turing was a British computer scientist and cryptanalyst. He developed the Turing machine, a model of a general-purpose computer, in 1936. During the Second World War, he worked at Bletchley Park (Government Code and Cyper School) as a codebreaker for the United Kingdom. At his time here, he would play a critical role in solving Enigma, Germany’s wartime infamous encryption system. Solving Enigma helped turning the tide of the war in favour of the Allies. After the war, he would go on to develop the Turing test in 1950. Alan Turing is widely considered the father of modern Artificial Intelligence, as well as being highly influential in theoretical computer science. The “Nobel Prize of Computing”, the ACM Turing Award, is named after him.

The History of AI initiative considers this event to be important due to “Computing Machinery and Intelligence” being a seminal paper in regards to both Computer Science and AI. The paper introduces many new concepts in CS and AI to the general public. Alan Turing is a pivotal figure in the development of Artificial Intelligence, computing, and machine learning as well. Thus, the publication of this paper is a critical moment in the History of AI.

This week in The History of AI at AIWS.net – Judea Pearl published “Probabilistic Reasoning in Intelligent Systems”

by Admin | Oct 2, 2020 | Chronicles, News

This week in The History of AI at AIWS.net – computer scientist Judea Pearl published Probabilistic Reasoning in Intelligent Systems in 1988. The book is, according to the publisher, about “the theoretical foundations and computational methods that underlie plausible reasoning under uncertainty.” The book covers topics such as AI systems, Markov and Baynesian networks, network propagation, and more. The Association for Computing Machinery (ACM) hailed the book as “[o]ne of the most cited works in the history of computer science” and that it “initiated the modern era in AI and converted many researchers who had previously worked in the logical and neural-network communities.”

1988 falls under the end of the second boom of AI, with the promotion of the Strategic Computing Initiative, Japan’s Fifth Generation, and other counterparts from other countries. The bust that followed it, known as the Second AI winter, lasted from the late 1980s to 1993. This bust was due to the perceptions of governments and investors, who believed that the field was failing, despite the fact that advances were still made.

Judea Pearl is a renowned Israeli-American computer scientist. He is a pioneer into Baynesian networks, probabilistic approaches to AI, and causal inference. He is also known for his other books, Causality: Models, Reasoning, and Inference (2000) and The Book of Why: The New Science of Cause and Effect (2018). Professor Pearl won the Turing Award, one of the highest honours in the field of computer science, in 2011, for his works into AI through probabilistic and causal reasoning. He is a Chancellor’s Professor at UCLA.

 Probabilistic Reasoning in Intelligent Systems (1988) can be accessed through the ACM digital library, which also has other resources on computer science and AI.

Due to the impact that the book has, the History of AI initiative considers it an important marker in AI history. Professor Judea Pearl is one of the most influential computer scientists around the world. He is a Mentor of AI World Society Innovation Network (AIWS.net). Professor Pearl resides on the History of AI Board. He was honored as 2020 World Leader in AI World Society by Michael Dukakis Institute and the Boston Global Forum. 

This week in The History of AI at AIWS.net – computer scientist James Robert Slagle developed SAINT

by Admin | Sep 25, 2020 | Chronicles, News

This week in The History of AI at AIWS.net – computer scientist James Robert Slagle developed SAINT in 1961. SAINT stands for Symbolic Automatic INTegrator. It was a heuristic program that could solve symbolic integration problems in freshman calculus. The machine was developed as a part of Slagle’s dissertation at MIT, with Marvin Minsky’s help. SAINT is considered the the first expert system. An expert system is a system that performs at the level of a human expert. SAINT was also one of the first projects that tried to produce a program that can come close to surpassing the Turing test as well.

James Robert Slagle is an American computer scientist. He worked on SAINT for his dissertation at MIT with Marvin Minsky. Slagle would receive his PhD in Mathematics from MIT later on in 1961. He’s a Professor in Computer Science with appointments in universities such as MIT, Johns Hopkins, UC Berkeley, and University of Minnesota.

Marvin Minsky also played a role in this project, as Slagle worked with him for this section of his dissertation. He would go on to be an important pioneer in the field of AI. He penned the research proposal for the Dartmouth Conference, which coined the term “Artificial Intelligence”, and he was a participant in it when it was hosted the next summer. Minsky would also co-founded the MIT AI labs, which went through different names, and the MIT Media Laboratory. In terms of popular culture, he was an adviser to Stanley Kubrick’s acclaimed movie 2001: A Space Odyssey. Minsky won the Turing Award in 1969.

The full dissertation can be found here. This project and dissertation is special in regard to AI, due to it being another step in its development, most notably for being the first expert system. Albeit it was only a minor project, the HAI initiative regards it as another pioneering attempt in the History of AI.

This week in The History of AI at AIWS.net – Marvin Minsky and Dean Edmonds built SNARC, the first artificial neural network

by Admin | Sep 18, 2020 | Chronicles, News

This week in The History of AI at AIWS.net – Marvin Minsky and Dean Edmonds built SNARC, the first artificial neural network, in 1951. SNARC stands for the Stochastic Neural Analog Reinforcement Calculator. It is a neural net machine, which itself is a randomly connected network of 40 Hebbs synapses. The idea to develop the machine was from a letter by Minsky to George Armitage Miller, who then got funding from the Air Force Office of Scientific Research. Dean Edmonds volunteered to help, as he was good with electronics. 

Marvin Minsky was a graduate student in mathematics at Princeton at the time of this project. He would go on to be an important pioneer in the field of AI. He penned the research proposal for the Dartmouth Conference, which coined the term “Artificial Intelligence”, and he was a participant in it when it was hosted the next summer. Minsky would also co-founded the MIT AI labs, which went through different names, and the MIT Media Laboratory. In terms of popular culture, he was an adviser to Stanley Kubrick’s acclaimed movie 2001: A Space Odyssey. He won the Turing Award in 1969.

Dean Edmonds was a graduate student in physics at Princeton at the time of the project. Although there is not many information about him, he wrote a short piece about his time around academia, which also details his contacts with Minsky and SNARC. George Armitage Miller, who helped get funding for the project from the Air Force, was an American psychologist. He was a pioneer in the field of cognitive psychology and cognitive science. 

This project is important to the History of AI as SNARC machine was one of the first experiments in the Artifical Intelligence. Furthermore, Marvin Minsky would become one of the founders of AI. HAI considers this an eventful project in the early development of AI.

This week in The History of AI at AIWS.net – the Strategic Computing Initiative founded by DARPA

by Admin | Sep 11, 2020 | Chronicles, News

This week in The History of AI at AIWS.net – DARPA founded the Strategic Computing Initiative to fund research of advanced computer hardware and artificial intelligence in 1983. DARPA stands for the Defense Advanced Research Projects Agency, a research and development agency founded by the US Department of Defense in 1958 as the ARPA. Although its aim was for usage in the military, many of the innovations the agency funded were beyond the requirements for the US military. Some technologies that emerged from the backing of DARPA are computer networking and graphical user interfaces. DARPA works with academics and industry and report directly to senior DoD officials. 

The Strategic Computing Initiative was founded in 1983, after the first AI winter in the 70s. The initiative supported projects that helped develop machine intelligence, from chip design to AI software. The DoD spent a total of 1 billion USD (not adjusted for inflation) before the program’s shutdown in 1993. Although the initiative failed to reach its overarching goals, specific targets were still met.

This project was created in response to Japan’s Fifth Generation Computer program, funded by the Japanese Ministry of Trade and Industry in 1982. The goal of this program was to create computers with massively parallel computing and logic programming and to propel Japan to the top spots in advanced technology. This will then create a platform for future developments in AI. By the time of the program’s end, the opinion of it was mixed, divided between considering it a failure or ahead of its time. 

Although the results of the SCI and other computer/AI projects in the 80s were mixed, they helped brought funding back to AI development after the first AI winter in the 70s. The History of AI marks the Strategic Computer Initative as an important event in AI due to its revival of AI in the US.

This week in the History of AI at AIWS.net – the Dartmouth Summer Research Project proposal

by Admin | Sep 4, 2020 | Chronicles, News

This week in The History of AI at AIWS.net – the Dartmouth Summer Research Project on Artificial Intelligence was proposed. The proposal was submitted on September 2, 1955, but written on August 31, 1955. It was the collaboration of John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon, who would all go on to become important AI pioneers. John McCarthy and Marvin Minsky came from academic backgrounds (Dartmouth and Harvard respectively). On the other hand, Nathaniel Rochester (IBM) and Claude Shannon (Bell Telephones) were tied to corporations.

This proposal was the first instance of the phrase Artificial Intelligenc being used officially. The document names direct some aspects of AI – automatic computer, how can a computer can be programmed to use a language, neuron nets, theory of the size of calculation, self improvements, and randomness and creativity. The research proposal asked for funding from sources such as the Rockefeller Foundation. Their estimated cost was $13,500 (not calculated for inflation)

The document called for a “2 month, 10-man study of Artificial Intelligence” in the summer of 1956 (the year following this document’s publication) at Dartmouth College. During this study, researchers will try to connect computer and information sciences with the brain. Each originator of the document wrote their own research proposal.

This event marks one of the beginnings of AI – the conception of what AI is and AI could be. It is the prelude to the big event of AI, the Dartmouth Conference. Without a seminal source like this, AI would not exist or may have taken a different direction entirely. The program, History of AI, owes a debt to the document.

A PDF of this proposal can be found here.