What is Artificial intelligence

what is artificial intelligence characteristics,what is frames in artificial intelligence with example,what is artificial intelligence and natural intelligence and what is artificial intelligence in computer science pdf
HalfoedGibbs Profile Pic
HalfoedGibbs,United Kingdom,Professional
Published Date:02-08-2017
Your Website URL(Optional)
In which we try to explain why we consider artijficial intelligence to be a subject most worthy of study, and in which we try to decide what exactly it is, this being a good thing to decide before embarking. We call ourselves Homo sapiens-man the wise-because our mental capacities are so im- portant to us. For thousands of years, we have tried to understand how we think; that is, how a mere handful of stuff can perceive, understand, predict, arid manipulate a world far larger ARTIFICIAL and more complicated than itself. The field of artificial intelligence, or AI, goes further still: INTELLIGENCE it attempts not just to understand but also to build intelligent entities. A1 is one of the newest sciences. Work started in earnest soon after World War 11, and the name itself was coined in 1956. Along with molecular biology, A1 is regularly cited as the "field I would most like to be in" by scientists in other disciplines. A student in physics might reasonably feel that all the good ideas have already been taken by Galileo, Newton, Einstein, and the rest. AI, on the other hand, still has openings for several full-time Einsteins. A1 currently encompasses a huge variety of subfields, ranging from general-purpose areas, such as learning and perception to such specific tasks as playing chess, proving math- ematical theorems, writing poetry, and diagnosing diseases. A1 systematizes and automates intellectual tasks and is therefore potentially relevant to any sphere of human intellectual activity. In this sense, it is truly a universal field. 1.1 WHAT IS AI? We have claimed that A1 is exciting, but we have not said what it is. Definitions of artificial intelligence according to eight textbooks are shown in Figure 11.1. These definitions vary along two main dimensions. Roughly, the ones on top are concerned with thought processes and reasoning, whereas the ones on the bottom address behavior. The definitions on the left measure success in terms of fidelity to human performance, whereas the ones on the right RATIONALITY measure against an ideal concept of intelligence, which we will call rationality. A system is rational if it does the "right thing," given what it knows. 2 Chapter 1. Introduction Systems that think like humans Systems that think rationally "The exciting new effort to make comput- "The study of mental faculties through the ers think . . . machines with minds, in the use of computational models." full and literal sense." (Haugeland, 1985) (Chamiak and McDermott, 1985) "The automation of activities that we "The study of the computations that make associate with human thinking, activities it possible to perceive, reason, and act." such as decision-making, problem solv- (Winston, 1992) ing, learning . . ." (Bellman, 1978) Systems that act like humans Systems that act rationally "The art of creating machines that per- "Computational Intelligence is the study form functions that require intelligence of the design of intelligent agents." (Poole when performed by people." (Kurzweil, et al., 1998) 1990) "The study of how to make computers do "A1 . . .is concerned with intelligent be- havior in artifacts." (Nilsson, 1998) things at which, at the moment, people are better." (Rich and Knight, 1991) Figure 1.1 Some definitions of artificial intelligence, organized into four categories. Historically, all four approaches to A1 have been followed. As one might expect, a tension exists between approaches centered around humans and approaches centered around rationality.' A human-centered approach must be an empirical science, involving hypothesis and experimental confirmation. A rationalist approach involves a combination of mathemat- ics and engineering. Each group has both disparaged and helped the other. Let us look at the four approaches in more detail. Acting humanly: The Turing Test approach The %ring Test, proposed by Alan Turing (195O), was designed to provide a satisfactory TURING TEST operational definition of intelligence. Rather than proposing a long and perhaps controversial list of qualifications required for intelligence, he suggested a test based on indistinguishability -human beings. The computer passes the test if a human from undeniably intelligent entities interrogator, after posing some written questions, cannot tell whether the written responses come from a person or not. Chapter 26 discusses the details of the test and whether a computer is really intelligent if it passes. For now, we note that programming a computer to pass the test provides plenty to work on. The computer would need to possess the following capabilities: NATURALLANGUAGE 0 natural language processing to enable it to communicate successfully in English. PROCESSING - We should point out that, by distinguishing between human and rational behavior, we are not suggesting that "irrational" in the sense of "emotionally unstable" or "insane." One merely need note humans are necessarily that we are not perfect: we are not all chess grandmasters, even those of us who know all the rules of chess; and, unfortunately, not everyone gets an A on the exam. Some systematic errors in human reasoning are cataloged by Kahneman et al. (1982). Section 1.1. What is AI? 3 KNOWLEDGE 0 knowledge representation to store what it knows or liears; REPRESENTATION AUTOMATED 0 automated reasoning to use the stored inforrnation to answer questions and to draw REASONING new conclusions; MACHINE LEARNING 0 machine learning to adapt to new circumstances and to detect and extrapolate patterns. Turing's test deliberately avoided direct physical interaction between the interrogator and the computer, because physical simulation of a person is unnecessary for intelligence. However, TOTAL TURING TEST the so-called total Turing Test includes a video signal so that the interrogator can test the subject's perceptual abilities, as well as the opportunity for the interrogator to pass physical objects "through the hatch." To pass the total Turing Test, the computer will need COMPUTER VISION computer vision to perceive objects, and ROBOTICS 0 robotics to manipulate objects and move about. These six disciplines compose most of AI, and Turing deserves credit for designing a test that remains relevant 50 years later. Yet A1 researchers have devoted little effort to passing the Turing test, believing that it is more important to study the underlying principles of in- telligence than to duplicate an exemplar. The quest for "arti.ficia1 flight" succeeded when the Wright brothers and others stopped imitating birds and learned about aerodynamics. Aero- nautical engineering texts do not define the goal of their field as making "machines that fly so exactly like pigeons that they can fool even other pigeons." Thinking humanly: The cognitive modeling approach If we are going to say that a given program thinks like a human, we must have some way of determining how humans think. We need to get inside the actual workings of human minds. There are two ways to do this: through introspectior-trying to catch our own thoughts as they go by- and through psychological experiments. Once we have a sufficiently precise theory of the mind, it becomes possible to express the theor,y as a computer program. If the program's input/output and timing behaviors match corresponding human behaviors, that is evidence that some of the program's mechanisms could also be operating in humans. For ex- ample, Allen Newel1 and Herbert Simon, who developed GPS, the "General Problem Solver" (Newel1 and Simon, 1961), were not content to have their program solve problems correctly. They were more concerned with comparing the trace of its reasoning steps to traces of human COGNITIVE SCIENCE subjects solving the same problems. The interdisciplinary field of cognitive science brings together computer models from A1 and experimental techniques from psychology to try to construct precise and testable theories of the workings of the human mind. Cognitive science is a fascinating field, worthy of an encyclopedia in itself (Wilson and Keil, 1999). We will not attempt to describe what is known of human cognition in this book. We will occasionally comment on similarities or difierences between AI techniques and human cognition. Real cognitive science, however, is necessarily based on experimental investigation of actual humans or animals, and we assume that the reader has access only to a computer for experimentation. In the early days of A1 there was often confusion between the approaches: an author would argue that an algorithm performs well on a task and that it is therefore a good model 4 Chapter 1. Introduction of human performance, or vice versa. Modern authors separate the two kinds of claims; this distinction has allowed both A1 and cognitive science to develop more rapidly. The two fields continue to fertilize each other, especially in the areas of vision and natural language. Vision in particular has recently made advances via an integrated approach that considers neurophysiological evidence and computational models. Thinking rationally: The "laws of thought" approach The Greek philosopher Aristotle was one of the first to attempt to codify "right thinking," that SYLLOGISMS is, irrefutable reasoning processes. His syllogisms provided patterns for argument structures that always yielded correct conclusions when given correct premises-for example, "Socrates is a man; all men are mortal; therefore, Socrates is mortal." These laws of thought were LOGIC supposed to govern the operation of the mind; their study initiated the field called logic. Logicians in the 19th century developed a precise notation for statements about all kinds of things in the world and about the relations among them. (Contrast this with ordinary arith- metic notation, which provides mainly for equality and inequality statements about numbers.) any solvable problem described in By 1965, programs existed that could, in principle, solve LOGICIST logical ntation. The so-called logicist tradition within artificial intelligence hopes to build on such programs to create intelligent systems. There are two main obstacles to this approach. First, it is not easy to take informal knowledge and state it in the formal terms required by logical notation, particularly when the knowledge is less than 100% certain. Second, there is a big difference between being able to solve a problem "in principle" and doing so in practice. Even problems with just a few dozen facts can exhaust the computational resources of any computer unless it has some guidance as to which reasoning steps to try first. Although both of these obstacles apply to any attempt to build computational reasoning systems, they appeared first in the logicist tradition. Acting rationally: The rational agent approach AGENT An agent is just something that acts (agent comes from the Latin agere, to do). But computer agents are expected to have other attributes that distinguish them from mere "programs," such as operating under autonomous control, perceiving their environment, persisting over a prolonged time period, adapting to change, and being capable of taking on another's goals. A RATIONALAGENT rational agent is one that acts so as to achieve the best outcome or, when there is uncertainty, the best expected outcome. In the "laws of thought" approach to AI, the emphasis was on correct inferences. Mak- ing correct inferences is sometimes part of being a rational agent, because one way to act rationally is to reason logically to the conclusion that a given action will achieve one's goals and then to act on that conclusion. On the other hand, correct inference is not all of ratio- nality, because there are often situations where there is no provably correct thing to do, yet something must still be done. There are also ways of acting rationally that cannot be said to involve inference. For example, recoiling from a hot stove is a reflex action that is usually more successful than a slower action taken after careful deliberation. If there is no solution, the program might never stop looking for one. Section 1.2. The Foundations of Artificial Intelligence 5 All the skills needed for the Turing Test are there to allow rational actions. Thus, we need the ability to represent knowledge and reason \with it because this enables us to reach good decisions in a wide variety of situations. We need to be able to generate comprehensible sentences in natural language because saying those sentences helps us get by in a complex society. We need learning not just for erudition, but because having a better idea of how the world works enables us to generate more effective strategies for dealing with it. We need visual perception not just because seeing is fun, but 1.0 get a better idea of what an action might achieve-for example, being able to see a tasty morsel helps one to move toward it. For these reasons, the study of A1 as rational-agent design has at least two advantages. First, it is more general than the "laws of thought" approach, because correct inference is just one of several possible mechanisms for achieving ratiornality. Second, it is more amenable to scientific development than are approaches based on human behavior or human thought be- cause the standard of rationality is clearly defined and conipletely general. Human behavior, on the other hand, is well-adapted for one specific eilvironnent and is the product, in part, of a complicated and largely unknown evolutionary pirocess that still is far from producing perfection. This book will therefore concentrate on general principles of rational agents and on components for constructing them. We will see that despite the apparent simplicity with which the problem can be stated, an enormous variety of issues come up when we try to solve it. Chapter 2 outlines some of these issues in more detail. One important point to keep in mind: We will see before too long that achieving perfect rationality-always doing tlle right thing-is not feasible in complicated environments. The computational demands are just too high. For most of the book, however, we will adopt the working hypothesis that perfect rationality is a good starting point for analysis. It simplifies the problem and provides the appropriate setting for most of the foundational material in LIMITED the field. Chapters 6 and 17 deal explicitly with the issue of limited rationality-acting RATIONALITY appropriately when there is not enough time to do all the comiputations one might like. In this section, we provide a brief history of the disciplines that contributed ideas, viewpoints, and techniques to AI. Like any history, this one is forced to (concentrate on a small number of people, events, and ideas and to ignore others that (also were important. We organize the history around a series of questions. We certainly would not vvish to give the impression that these questions are the only ones the disciplines address or that the disciplines have all been working toward A1 as their ultimate fruition. Philosophy (428 B . c .-present) Can formal rules be used to draw valid conclusions? How does the mental mind arise from a physical brain? Where does knowledge come from? How does knowledge lead to action? Chapter I. Introduction Aristotle (384-322 B.C.) was the first to formulate a precise set of laws governing the ratio- nal part of the mind. He developed an informal system of syllogisms for proper reasoning, which in principle allowed one to generate conclusions mechanically, given initial premises. Much later, Ramon Lull (d. 13 15) had the idea that useful reasoning could actually be carried out by a mechanical artifact. His "concept wheels" are on the cover of this book. Thomas Hobbes (1588-1679) proposed that reasoning was like numerical computation, that "we add " The automation of computation itself was already well and subtract in our silent thoughts. under way; around 1500, Leonardo da Vinci (1452-1519) designed but did not build a me- chanical calculator; recent reconstructions have shown the design to be functional. The first known calculating machine was constructed around 1623 by the German scientist Wilhelm Schickard (1592-1635), although the Pascaline, built in 1642 by Blaise Pascal (1623-1662), is more famous. Pascal wrote that "the arithmetical machine produces effects which appear nearer to thought than all the actions of animals." Gottfried Wilhelm Leibniz (1646-1716) built a mechanical device intended to carry out operations on concepts rather than numbers, but its scope was rather limited. Now that we have the idea of a set of rules that can describe the formal, rational part of the mind, the next step is to consider the mind as a physical system. RenC Descartes (1596-1650) gave the first clear discussion of the distinction between mind and matter and of the problems that arise. One problem with a purely physical conception of the mind is that it seems to leave little room for free will: if the mind is governed entirely by physical laws, then it has no more free will than a rock "deciding" to fall toward the center of the earth. Although DUALISM a strong advocate of the power of reasoning, Descartes was also a proponent of dualism. He held that there is a part of the human mind (or soul or spirit) that is outside of nature, exempt from physical laws. Animals, on the other hand, did not possess this dual quality; they could MATERIALISM be treated as machines. An alternative to dualism is materialism, which holds that the brain's operation according to the laws of physics constitutes the mind. Free will is simply the way that the perception of available choices appears to the choice process. Given a physical mind that manipulates knowledge, the next problem is to establish the EMPIRICISM source of knowledge. The empiricism movement, starting with Francis Bacon's (1561-1626) Novum is characterized by a dictum of John Locke (1632-1704): "Nothing is in the understanding, which was not first in the senses." David Hume's (171 1-1776) A Treatise INDUCTION of Human Nature (Hume, 1739) proposed what is now known as the principle of induction: that general rules are acquired by exposure to repeated associations between their elements. Building on the work of Ludwig Wittgenstein (1889-1951) and Bertrand Russell (1872- 1970), the famous Vienna Circle, led by Rudolf Carnap (1891-1970), developed the doctrine of logical positivism. This doctrine holds that all knowledge can be characterized by logical LOGICAL POSITI\ mM OBSERVATION theories connected, ultimately, to observation sentences that correspond to sensory inputs.4 SENTENCES CONFIRMATION The confirmation theory of Carnap and Carl Hempel (1905-1997) attempted to understand THEORY how knowledge can be acquired from experience. Carnap's book The Logical Structure of An update of Aristotle's Organon, or instrument of thought. In this picture, all meaningful statements can be verified or falsified either by analyzing the meaning of the words or by carrying out experiments. Because this rules out most of metaphysics, as was the intention, logical positivism was unpopular in some circles. Section 1.2. 'The Foundations of Artificial Intelligence 7 the World (1928) defined an explicit computational procedure for extracting knowledge from elementary experiences. It was probably the first theory of mind as a computational process. The final element in the philosophical picture of the mind is the connection between knowledge and action. This question is vital to AI, because intelligence requires action as well as reasoning. Moreover, only by understanding how actions are justified can we understand how to build an agent whose actions are justifiable (or rational). Aristotle argued that actions are justified by a logical connection between goals and knowledge of the action's outcome (the last part of this extract also appears on the front cover of this book): But how does it happen that thinking is sometimes accompanied by action and sometimes not, sometimes by motion, and sometimes not? It looks as if almost the same thing happens as in the case of reasoning and making inferences about unchanging objects. But in that case the end is a speculative proposition . . . whereas here the conclusion which results from the two premises is an action. . . . I neeld covering; a cloak is a covering. I need a cloak. What I need, I have to make; I need a cloak. I have to make a cloak. And the conclusion, the "I have to make a cloak:' is an action. (I'Jussbaum, 1978, p. 40) In the Nicomachean Ethics (Book 111. 3, 11 12b), Aristotle further elaborates on this topic, suggesting an algorithm: We deliberate not about ends, but about means. For a doctor does not deliberate whether he shall heal, nor an orator whether he shall persuade, . . . They assume the end and consider how and by what means it is attained, and if it seems easily and best produced thereby; while if it is achieved by one means only they consider how it will be achieved by this and by what means this will be achieved, till they come to the first cause, . . . and what is last in the order of analysis seems to be first in the order of becoming. And if we come on an impossibility, we give up the search, e.g. if we need money and this cannot be got; but if a thing appears possible we try to do it. Aristotle's algorithm was implemented 2300 years later by Newel1 and Simon in their GPS program. We would now call it a regression planning system. (See Chapter 1 1 .) Goal-based analysis is useful, but does not say what 110 do when several actions will achieve the goal, or when no action will achieve it completely. Antoine Arnauld (1612-1694) correctly described a quantitative formula for deciding what action to take in cases like this -1873) book Utilitarianism (Mill, 1863) promoted (see Chapter 16). John Stuart Mill's (1806 the idea of rational decision criteria in all spheres of human activity. The more formal theory of decisions is discussed in the following section. Mathematics (c. 800-present) o What are the formal rules to draw valid conclusions? What can be computed? e How do we reason with uncertain information? Philosophers staked out most of the important ideas of k1, but the leap to a formal science re- quired a level of mathematical formalization in three fundamt:ntal areas: logic, computation, and probability. The idea of formal logic can be traced back to the philosophers of ancient Greece (see Chapter 7), but its mathematical development really began with the work of George Boole 8 Chapter 1. Introduction (1 8 15-1 864), who worked out the details of propositional, or Boolean, logic (Boole, 1847). In 1879, Gottlob Frege (1848-1925) extended Boole's logic to include objects and relations, creating the first-order logic that is used today as the most basic knowledge representation system.5 Alfred Tarski (1902-1983) introduced a theory of reference that shows how to relate the objects in a logic to objects in the real world. The next step was to determine the limits of what could be done with logic and computation. ALGORITHM The first nontrivial algorithm is thought to be Euclid's algorithm for computing great- est common denominators. The study of algorithms as objects in themselves goes back to al-Khowarazmi, a Persian mathematician of the 9th century, whose writings also introduced Arabic numerals and algebra to Europe. Boole and others discussed algorithms for logical deduction, and, by the late 19th century, efforts were under way to formalize general math- ematical reasoning as logical deduction. In 1900, David Hilbert (1862-1943) presented a list of 23 problems that he correctly predicted would occupy mathematicians for the bulk of the century. The final problem asks whether there is an algorithm for deciding the truth of any logical proposition involving the natural numbers-the famous Entscheidungsproblem, or decision problem. Essentially, Hilbert was asking whether there were fundamental limits to the power of effective proof procedures. In 1930, Kurt Godel (1906-1978) showed that there exists an effective procedure to prove any true statement in the first-order logic of Frege and Russell, but that first-order logic could not capture the principle of mathematical induc- tion needed to characterize the natural numbers. In 1931, he showed that real limits do exist. lNCoMPLETENEss His incompleteness theorem showed that in any language expressive enough to describe the THEOREM properties of the natural numbers, there are true statements that are undecidable in the sense that their truth cannot be established by any algorithm. This fundamental result can also be interpreted as showing that there are some functions on the integers that cannot be represented by an algorithm-that is, they cannot be computed. This motivated Alan Turing (1912-1954) to try to characterize exactly which functions are capable of being computed. This notion is actually slightly problematic, because the notion of a computation or effective procedure really cannot be given a formal definition. However, the Church-Turing thesis, which states that the Turing machine (Turing, 1936) is capable of computing any computable function, is generally accepted as providing a sufficient definition. Turing also showed that there were some functions that no Turing machine can compute. For example, no machine can tell in general whether a given program will return an answer on a given input or run forever. Although undecidability and noncomputability are important to an understanding of computation, the notion of intractability has had a much greater impact. Roughly speak- INTRACTABILITY ing, a problem is called intractable if the time required to solve instances of the problem grows exponentially with the size of the instances. The distinction between polynomial and exponential growth in complexity was first emphasized in the mid-1960s (Cobham, 1964; Ed- monds, 1965). It is important because exponential growth means that even moderately large instances cannot be solved in any reasonable time. Therefore, one should strive to divide Frege's proposed notation for first-order logic never became popular, for reasons that are apparent immediately from the example on the front cover. Section 1.2. The Foundations of Artificial Intelligence 9 the overall problem of generating intelligent behavior into tractable subproblems rather than intractable ones. NP-COMPLETENESS How can one recognize an intractable problem? The theory of NP-completeness, pio- neered by Steven Cook (1971) and Richard Karp (1972), provides a method. Cook and Karp showed the existence of large classes of canonical cornbinat.oria1 search and reasoning prob- lems that are NP-complete. Any problem class to which the: class of NP-complete problems can be reduced is likely to be intractable. (Although it has not been proved that NP-complete These results contrast problems are necessarily intractable, most theoreticians believe it.) with the optimism with which the popular press greeted the first computers-"Electronic Super-Brains" that were "Faster than Einstein" Despite the increasing speed of computers, careful use of resources will characterize intelligent systems. Put crudely, the world is an extremely large problem instance In recent years, A1 has helped explain why some instances of NP-complete problems are hard, yet others are easy (Cheeseman et al., 1991). Besides logic and computation, the third gretit contribution of mathematics to A1 is PROBABILITY the theory of probability. The Italian Gerolamo Cardano (1501-1576) first framed the idea of probability, describing it in terms of the possible outcomes of gambling events. Prob- ability quickly became an invaluable part of all the quantitative sciences, helping to deal with uncertain measurements and incomplete theories. Pierre Fermat (1 60 1-1 665), Blaise Pascal (1623-1662), James Bernoulli (1654-1705), F'ierre Laplace (1749-1827), and oth- ers advanced the theory and introduced new statistical methods. Thomas Bayes (1702-1 761) proposed a rule for updating probabilities in the light of new evidence. Bayes' rule and the re- sulting field called Bayesian analysis form the basis of most modern approaches to uncertain reasoning in A1 systems. Economics (1776-present) a How should we make decisions so as to maximize payoff? o How should we do this when others may not go along? a How should we do this when the payoff may be f,x in the future? The science of economics got its start in 1776, when Scottish philosopher Adam Smith (1723-1790) published An Inquiry into the Nature and Causes of the Wealth of Nations. While the ancient Greeks and others had made contributions to economic thought, Smith was the first to treat it as a science, using the idea that economies can be thought of as consist- ing of individual agents maximizing their own economic well-being. Most people think of economics as being about money, but economists will say that they are really studying how people make choices that lead to preferred outcomes. The mathematical treatment of "pre- 7 ferred outcomes ' or utility was first formalized by Lkon Walras (pronounced "Valrasse") (1834 - 1910) and was improved by Frank Ramsey (193 1) and later by John von Neumann and Oskar Morgenstern in their book The Theory of Games and Economic Behavior (1944). DECISION THEORY Decision theory, which combines probability theory with utility theory, provides a for- mal and complete framework for decisions (economic or otherwise) made under uncertainty- that is, in cases where probabilistic descriptions appropriately capture the decision-maker's environment. This is suitable for "large" economies where each agent need pay no attention 10 Chapter 1. Introduction to the actions of other agents as individuals. For "small" economies, the situation is much more like a game: the actions of one player can significantly affect the utility of another (either positively or negatively). Von Neumann and Morgenstern's development of game GAMETHEORY theory (see also Luce and Raiffa, 1957) included the surprising result that, for some games, a rational agent should act in a random fashion, or at least in a way that appears random to the adversaries. For the most part, economists did not address the third question listed above, namely, how to make rational decisions when payoffs from actions are not immediate but instead re- sult from several actions taken in sequence. This topic was pursued in the field of operations OPERATIONS research, which emerged in World War I1 from efforts in Britain to optimize radar installa- RESEARCH tions, and later found civilian applications in complex management decisions. The work of Richard Bellman (1957) formalized a class of sequential decision problems called Markov decision processes, which we study in Chapters 17 and 2 1. Work in economics and operations research has contributed much to our notion of ra- tional agents, yet for many years A1 research developed along entirely separate paths. One reason was the apparent complexity of making rational decisions. Herbert Simon (1 91 6- 2001), the pioneering A1 researcher, won the Nobel prize in economics in 1978 for his early SATISFICING work showing that models based on satisficing-making decisions that are "good enough," rather than laboriously calculating an optimal decision-gave a better description of actual human behavior (Simon, 1947). In the 1990s, there has been a resurgence of interest in decision-theoretic techniques for agent systems (Wellman, 1995). Neuroscience (1861-present) How do brains process information? NEUROSCIENCE Neuroscience is the study of the nervous system, particularly the brain. The exact way in which the brain enables thought is one of the great mysteries of science. It has been appre- ciated for thousands of years that the brain is somehow involved in thought, because of the evidence that strong blows to the head can lead to mental incapacitation. It has also long been known that human brains are somehow different; in about 335 B.C. Aristotle wrote, "Of all the animals, man has the largest brain in proportion to his size." Still, it was not until the middle of the 18th century that the brain was widely recognized as the seat of consciousness. Before then, candidate locations included the heart, the spleen, and the pineal gland. Paul Broca's (1824-1880) study of aphasia (speech deficit) in brain-damaged patients in 1861 reinvigorated the field and persuaded the medical establishment of the existence of localized areas of the brain responsible for specific cognitive functions. In particular, he showed that speech production was localized to a portion of the left hemisphere now called Broca's area7 By that time, it was known that the brain consisted of nerve cells or neurons, NEURONS but it was not until 1873 that Carnillo Golgi (1843-1926) developed a staining technique allowing the observation of individual neurons in the brain (see Figure 1.2). This technique Since then, it has been discovered that some species of dolphins and whales have relatively larger brains. The large size of human brains is now thought to be enabled in part by recent improvements in its cooling system. Many cite Alexander Hood (1824) as a possible prior source. Section 1.2. The Foundations of Artificial Intelligence 11 Axon from another cell I Synapses Cell body or Soma The parts of a nerve cell or neuron. Each neuron consists of a cell body, Figure 1.2 or soma, that contains a cell nucleus. Branching out from the cell body are a number of fibers called dendrites and a single long fiber called the axon. The axon stretches out for a long distance, much longer than the scale in this diagram indicates. Typically they are 1 cm long (100 times the diameter of the cell body), but can reach up to 1 meter. A neuron makes connections with 10 to 100,000 other neurons at junctions called synapses. Signals are propagated from neuron to neuron by a complicated electrochemical reaction. The signals control brain activity in the short term, and also enable long-term changes in the position and connectivity of neurons. These mechanisms are thought to form the basis for learning in the brain. Most information processing goes on in Ihe cerebral cortex, the outer layer of the brain. The basic organizational unit appears to be a column of tissue about 0.5 mm in diameter, extending the full depth of the cortex, which is about 4 mm in humans. A column contains about 20,000 neurons. was used by Santiago Ramon y Cajal (1852-1934) in his pioneering studies of the brain's neuronal structures.' We now have some data on the mapping between areas of the brain and the parts of the body that they control or from which they receive senstory input. Such mappings are able to change radically over the course of a few weeks, and some animals seem to have multiple maps. Moreover, we do not fully understand how other areas can take over functions when one area is damaged. There is almost no theory on how an individual memory is stored. The measurement of intact brain activity began1 in 1929 with the invention by Hans Berger of the electroencephalograph (EEG). The recent development of functional magnetic resonance imaging (fMRI) (Ogawa et al., 1990) is giving neuroscientists unprecedentedly detailed images of brain activity, enabling measurements that correspond in interesting ways to ongoing cognitive processes. These are augmented by advances in single-cell recording of Golgi persisted in his belief that the brain's functions were carried out primarily in a continuous medium in which neurons were embedded, whereas Cajal propounded the "neuronal doctrine." The two shared the Nobel prize in 1906 but gave rather antagonistic acceptance speeches. 12 Chapter 1. Introduction I / Computer 1 Human Brain Computational units 1 CPU, lo8 gates 10" neurons Storage units 101° bits RAM 10" neurons 10" bits disk 1014 synapses Cycle time lo-' sec sec Bandwidth 10" bitslsec 1014 bitslsec Memory updateslsec 109 1014 Figure 1.3 A crude comparison of the raw computational resources available to computers (circa 2003) and brains. The computer's numbers have all increased by at least a factor of 10 since the first edition of this book, and are expected to do so again this decade. The brain's numbers have not changed in the last 10,000 years. neuron activity. Despite these advances, we are still a long way from understanding how any of these cognitive processes actually work. The truly amazing conclusion is that a collection of simple cells can lead to thought, action, and consciousness or, in other words, that brains cause minds (Searle, 1992). The only real alternative theory is mysticism: that there is some mystical realm in which minds operate that is beyond physical science. Brains and digital computers perform quite different tasks and have different properties. Figure 1.3 shows that there are 1000 times more neurons in the typical human brain than there 9 are gates in the CPU of a typical high-end computer. Moore's Law predicts that the CPU's gate count will equal the brain's neuron count around 2020. Of course, little can be inferred from such predictions; moreover, the difference in storage capacity is minor compared to the difference in switching speed and in parallelism. Computer chips can execute an instruction in a nanosecond, whereas neurons are millions of times slower. Brains more than make up for this, however, because all the neurons and synapses are active simultaneously, whereas most current computers have only one or at most a few CPUs. Thus, even though a computer is a million times faster in raw switching speed, the brain ends up being 100,000 times faster at what it does. Psychology (1879-present) a How do humans and animals think and act? The origins of scientific psychology are usually traced to the work of the German physi- cist Hermann von Helmholtz (1 82 1-1 894) and his student Wilhelm Wundt (1 832-1920). Helmholtz applied the scientific method to the study of human vision, and his Handbook of Physiological Optics is even now described as "the single most important treatise on the physics and physiology of human vision" (Nalwa, 1993, p.15). In 1879, Wundt opened the first laboratory of experimental psychology at the University of Leipzig. Wundt insisted on carefully controlled experiments in which his workers would perform a perceptual or associa- Moore's Law says that the number of transistors per square inch doubles every 1 to 1.5 years. Human brain capacity doubles roughly every 2 to 4 million years. Section 1.2. The Foundations of Artificial Intelligence 13 tive task while introspecting on their thought processes. The careful controls went a long way toward making psychology a science, but the subjective nature of the data made it unlikely that an experimenter would ever disconfirm his or her own theories. Biologists studying animal behavior, on the other hand, lacked introspective data and developed an objective methodology, as described by H. S. Jennings (1906)l in his influential work Behavior of the BEHAVIORISM Lower Organisms. Applying this viewpoint to humans, the belaviorism movement, led by John Watson (1878-1958), rejected any theory involving rnental processes on the grounds that introspection could not provide reliable evidence. Behaviorists insisted on studying only objective measures of the percepts (or stiwzulus) given to an1 animal and its resulting actions (or response). Mental constructs such as knowledge, beliefs, goals, and reasoning steps were dismissed as unscientific "folk psychology." Behaviorism discovered a lot about rats and pi- geons, but had less success at understanding humans. Nevertheless, it exerted a strong hold on psychology (especially in the United States) from about I1920 to 1960. The view of the brain as an information-processing device, which is a principal charac- COGNITIVE teristic of cognitive psychology, can be traced back at least to the works of William ames" PSYCHOLOGY (1 842-19 10). Helmholtz also insisted that perception involved a form of unconscious log- ical inference. The cognitive viewpoint was largely eclipsed by behaviorism in the United States, but at Cambridge's Applied Psychology Unit, directed by Frederic Bartlett (1886- 1969), cognitive modeling was able to flourish. The Nature of Explanation, by Bartlett's student and successor Kenneth Craik (1943), forcefully reestablished the legitimacy of such "mental" terms as beliefs and goals, arguing that they are just as scientific as, say, using pressure and temperature to talk about gases, despite their being made of molecules that have neither. Craik specified the three key steps of a knowledge-based agent: (1) the stimulus must be translated into an internal representation, (2) the representation is manipulated by cogni- tive processes to derive new internal representations, and (3) these are in turn retranslated back into action. He clearly explained why this was a good design for an agent: If the organism carries a "small-scale model" of external reality and of its own possible actions within its head, it is able to try out various alternatives, conclude which is the best of them, react to future situations before they arise, utilize the knowledge of past events in dealing with the present and future, and in every way to react in a much fuller, safer, and more competent manner to the emergencies which face it. (Craik, 1943) After Craik's death in a bicycle accident in 194.5, his work was continued by Don- ald Broadbent, whose book Perception and Communication (1958) included some of the first information-processing models of psychological phenomena. Meanwhile, in the United States, the development of computer modeling led to the creation of the field of cognitive COGNITIVESCIENCE science. The field can be said to have started at a workshop in September 1956 at MIT. (We shall see that this is just two months after the conferelnce at which A1 itself was "born.") At the workshop, George Miller presented The Magic Number Seven, Noam Chomsky presented Three Models of Language, and Allen Newel1 and Herbert Simon presented The Logic The- ory Machine. These three influential papers showed how coniputer models could be used to lo William James was the brother of novelist Henry James. It is said that Henry wrote fiction as if it were psychology and William wrote psychology as if it were fiction. 14 Chapter 1. Introduction address the psychology of memory, language, and logical thinlung, respectively. It is now a common view among psychologists that "a cognitive theory should be like a computer pro- gram" (Anderson, 1980), that is, it should describe a detailed information-processing mecha- nism whereby some cognitive function might be implemented. Computer engineering (1940-present) How can we build an efficient computer? For artificial intelligence to succeed, we need two things: intelligence and an artifact. The computer has been the artifact of choice. The modern digital electronic computer was in- vented independently and almost simultaneously by scientists in three countries embattled in World War 11. The first operational computer was the electromechanical Heath obinson," built in 1940 by Alan Turing's team for a single purpose: deciphering German messages. In 1943, the same group developed the Colossus, a powerful general-purpose machine based on vacuum tubes.12 The first operational programmable computer was the 2-3, the inven- tion of Konrad Zuse in Germany in 1941. Zuse also invented floating-point numbers and the first high-level programming language, Plankalkiil. The first electronic computer, the ABC, was assembled by John Atanasoff and his student Clifford Berry between 1940 and 1942 at Iowa State University. Atanasoff's research received little support or recognition; it was the ENIAC, developed as part of a secret military project at the University of Pennsylvania by a team including John Mauchly and John Eckert, that proved to be the most influential forerunner of modern computers. In the half-century since then, each generation of computer hardware has brought an increase in speed and capacity and a decrease in price. Performance doubles every 18 months or so, with a decade or two to go at this rate of increase. After that, we will need molecular engineering or some other new technology. Of course, there were calculating devices before the electronic computer. The earliest automated machines, dating from the 17th century, were discussed on page 6. The first pro- grammable machine was a loom devised in 1805 by Joseph Marie Jacquard (1752-1834) that used punched cards to store instructions for the pattern to be woven. In the mid-19th century, Charles Babbage (1792-1871) designed two machines, neither of which he completed. The "Difference Engine," which appears on the cover of this book, was intended to compute math- ematical tables for engineering and scientific projects. It was finally built and shown to work in 1991 at the Science Museum in London (Swade, 1993). Babbage's "Analytical Engine" was far more ambitious: it included addressable memory, stored programs, and conditional jumps and was the first artifact capable of universal computation. Babbage's colleague Ada Lovelace, daughter of the poet Lord Byron, was perhaps the world's first programmer. (The programming language Ada is named after her.) She wrote programs for the unfinished Ana- lytical Engine and even speculated that the machine could play chess or compose music. l1 Heath Robinson was a cartoonist famous for his depictions of whimsical and absurdly complcated conbap- tions for everyday tasks such as buttering toast. l2 In the postwar period, Turing wanted to use these computers for A1 research-for example, one of the first chess programs (Turing et al., 1953). His efforts were blocked by the British government. Section 1.2. The Foundations of Artificial Intelligence 15 A1 also owes a debt to the software side of computer science, which has supplied the operating systems, programming languages, and tools needed to write modern programs (and papers about them). But this is one area where the debt has been repaid: work in A.1 has pio- neered many ideas that have made their way back to mainstream computer science, including time sharing, interactive interpreters, personal computers with windows and mice, rapid de- velopment environments, the linked list data type, automatic storage management, and key concepts of symbolic, functional, dynamic, and object-oriented programming. Control theory and Cybernetics (1948-present) 0 How can artifacts operate under their own control? Ktesibios of Alexandria (c. 250 B.c.) built the first self-controlling machine: a water clock with a regulator that kept the flow of water running through it at a constant, predictable pace. This invention changed the definition of what an artifact could do. Previously, only living things could modify their behavior in response to changes in the environment. Other examples of self-regulating feedback control systems include the steam engine governor, created by James Watt (1736-1 8 19), and the thermostat, invented by Colnelis Drebbel (1 572-1633), who also invented the submarine. The mathematical theory of stable feedback systems was developed in the 19th century. CONTROL THEORY The central figure in the creation of what is now called control theory was Norbert Wiener (1894-1964). Wiener was a brilliant mathematician who worked with Bertrand Rus- sell, among others, before developing an interest in biological and mechanical controll systems and their connection to cognition. Like Craik (who also used control systems as psycholog- ical models), Wiener and his colleagues Arturo Rosenblueth and Julian Bigelow challenged the behaviorist orthodoxy (Rosenblueth et al., 1943). They viewed purposive behavior as arising from a regulatory mechanism trying to minimize "error"-the difference between current state and goal state. In the late 1940s, Wiener, along with Warren McCulloch, Walter 13itts, and John von Neumann, organized a series of conferences that explored the nevv mathe- matical and computationall models of cognition and influenced many other researchers in the CYBERNE ICS behavioral sciences. Wiener's book Cybernetics (1948) became a bestseller and avvoke the public to the possibility of artificially intelligent machines. Modern control theory, especially the branch known as stochastic optimal control, has OBJECTIVE as its goal the design of systems that maximize an objective function over time. This roughly FUNCTION matches our view of AH: designing systems that behave optimally. Why, then, are A1 and con- trol theory two different fields, especially given the close connections among their founders? The answer lies in the close coupling between the mathematical techniques that were familiar to the participants and the corresponding sets of problems that were encompassed in each world view. Calculus and matrix algebra, the tools of control theory, lend themselves to sys- tems that are describable b,y fixed sets of continuous variables; furthermore, exact analysis is typically feasible only for linear systems. A1 was founded in part as a way to escape from the limitations of the mathematics of control theory in the 1950s. The tools of logical inference and computation allowed A1 researchers to consider some problems such as language, vision, and planning, that fell completely outside the control theorist's purview. 16 Chapter 1. Introduction Linguistics (1957-present) How does language relate to thought? In 1957, B. F. Skinner published Verbal Behavior. This was a comprehensive, detailed ac- count of the behaviorist approach to language learning, written by the foremost expert in the field. But curiously, a review of the book became as well known as the book itself, and served to almost kill off interest in behaviorism. The author of the review was Noam Chomsky, who had just published a book on his own theory, Syntactic Structures. Chomsky showed how the behaviorist theory did not address the notion of creativity in language-it did not explain how a child could understand and make up sentences that he or she had never heard before. Chomsky's theory-based on syntactic models going back to the Indian linguist Panini (c. 350 .c.)-could explain this, and unlike previous theories, it was formal enough that it could in principle be programmed. Modem linguistics and AI, then, were "born" at about the same time, and grew up together, intersecting in a hybrid field called computational linguistics or natural language processing. The problem of understanding language soon turned out to be considerably more complex than it seemed in 1957. Understanding language requires an understanding of the subject matter and context, not just an understanding of the structure of sentences. This might seem obvious, but it was not widely appreciated until the 1960s. Much of the early work in knowledge representation (the study of how to put knowledge into a form that a computer can reason with) was tied to language and informed by research in Linguistics, which was connected in turn to decades of work on the philosophical analysis of language. With the background material behind us, we are ready to cover the development of A1 itself. The gestation of artificial intelligence (1943-1955) The first work that is now generally recognized as A1 was done by Warren McCulloch and Walter Pitts (1943). They drew on three sources: knowledge of the basic physiology and function of neurons in the brain; a formal analysis of propositional logic due to Russell and Whitehead; and Turing's theory of computation. They proposed a model of artificial neurons in which each neuron is characterized as being "on" or "off," with a switch to "on" occurring in response to stimulation by a sufficient number of neighboring neurons. The state of a neuron was conceived of as "factually equivalent to a proposition which proposed its adequate stimulus." They showed, for example, that any computable function could be computed by some network of connected neurons, and that all the logical connectives (and, or, not, etc.) could be implemented by simple net structures. McCulloch and Pitts also suggested that suitably defined networks could learn. Donald Hebb (1949) demonstrated a simple updating rule for modifying the connection strengths between neurons. His rule, now called Hebbian learning, remains an influential model to this day. Section 1.3. The History of Artificial Intelligence 17 Two undergraduate students at Harvard, Marvin Minsky and Dean Edmonds, built the first neural network computer in 1950. The SNARC, as it was called, used 3000 vacuum tubes and a surplus automatic pilot mechanism from a B-24 bomber to simulate a network of 40 neurons. Later, at Princeton, Minsky studied universal computation in neural networks. His Ph.D. committee was skeptical about whether this kind of work should be considered mathematics, but von Neumann reportedly said, "If it isn't now, it will be someday." Minsky was later to prove influential theorems showing the limitations of neural network research. There were a number of early examples of work that can be characterized as AI, but it was Alan Turing who first articulated a complete vision of A1 in his 1950 article "Comput- ing Machinery and Intelligence." Therein, he introduced the Turing test, machine learning, genetic algorithms, and reinforcement learning. The birth of artificial intelligence (1956) Princeton was home to another influential figure in AI, John McCarthy. After graduation, McCarthy moved to Dartmouth College, which was to become the official birthplace of the field. McCarthy convinced Minsky, Claude Shannon, and Nathaniel Rochester to help him bring together U.S. researchers interested in automata theory, neural nets, and the study of intelligence. They organized a two-month workshop at Dartmouth in the summer of 1956. There were 10 attendees in all, including Trenchard More from Princeton, Arthur Samuel from IBM, and Ray Solomonoff and Oliver Selfridge from MIT. Two researchers from Carnegie Tech,13 Allen Newell and Herbert Simon, rather stole the show. Although the others had ideas and in some cases programs for particular appli- cations such as checkers, Newel1 and Simon already had a reasoning program, the Logic Theorist (LT), about which Simon claimed, "We have invented a computer program capable of thinking non-numerically, and thereby solved the venerable mind-body problem."14 Soon after the workshop, the program was able to prove most of the theorems in Chapter 2 of Rus- sell and Whitehead's Principia Mathernatica. Russell was reportedly delighted when Simon showed him that the program had come up with a proof for one theorem that was shorter than the one in Principia. The editors of the Journal of Symbolic Logic were less impressed; they rejected a paper coauthored by Newell, Simon, and Logic Theorist. The Dartmouth workshop did not lead to any new breakthroughs, but it did introduce all the major figures to each other. For the next 20 years, the field would be dominated by these people and their students and colleagues at MIT, CMU, Stanford, and IBM. Perhaps the longest-lasting thing to come out of the workshop was an agreement to adopt McCarthy's new name for the field: artificial intelligence. Perhaps "computational rationality" would have been better, but "AI" has stuck. Looking at the proposal for the Dartmouth workshop (McCarthy et al., 1955), we can A1 to become a separate field. Why couldn't all the work done see why it was necessary for l3 NOW Carnegie Mellon University (CMU). l4 Newel1 and Simon also invented a list-processing language, IPL, to write LT. They had no compiler, and translated it into machine code by hand. To avoid errors, they worked in parallel, calling out binary numbers to each other as they wrote each instruction to make sure they agreed. 18 Chawter 1. Introduction in A1 have taken place under the name of control theory, or operations research, or decision theory, which, after all, have objectives similar to those of AI? Or why isn't A1 a branch of mathematics? The first answer is that A1 from the start embraced the idea of duplicating human faculties like creativity, self-improvement, and language use. None of the other fields were addressing these issues. The second answer is methodology. A1 is the only one of these fields that is clearly a branch of computer science (although operations research does share an emphasis on computer simulations), and A1 is the only field to attempt to build machines that will function autonomously in complex, changing environments. Early enthusiasm, great expectations (1952-1969) The early years of A1 were full of successes-in a limited way. Given the primitive computers and programming tools of the time, and the fact that only a few years earlier computers were seen as things that could do arithmetic and no more, it was astonishing whenever a computer did anything remotely clever. The intellectual establishment, by and large, preferred to believe that "a machine can never do X." (See Chapter 26 for a long list of X's gathered by Turing.) A1 researchers naturally responded by demonstrating one X after another. John McCarthy referred to this period as the "Look, Ma, no hands" era. Newel1 and Simon's early success was followed up with the General Problem Solver, or GPS. Unlike Logic Theorist, this program was designed from the start to imitate human problem-solving protocols. Within the limited class of puzzles it could handle, it turned out that the order in which the program considered subgoals and possible actions was similar to that in which humans approached the same problems. Thus, GPS was probably the first pro- gram to embody the "thinking humanly" approach. The success of GPS and subsequent pro- grams as models of cognition led Newel1 and Simon (1976) to formulate the famous physical PHyslCALSYMBOL symbol system hypothesis, which states that "a physical symbol system has the necessary and SYSTEM sufficient means for general intelligent action." What they meant is that any system (human or machine) exhibiting intelligence must operate by manipulating data structures composed of symbols. We will see later that this hypothesis has been challenged from many directions. At IBM, Nathaniel Rochester and his colleagues produced some of the first A1 pro- grams. Herbert Gelernter (1959) constructed the Geometry Theorem Prover, which was able to prove theorems that many students of mathematics would find quite tricky. Starting in 1952, Arthur Samuel wrote a series of programs for checkers (draughts) that eventually learned to play at a strong amateur level. Along the way, he disproved the idea that comput- ers can do only what they are told to: his program quickly learned to play a better game than its creator. The program was demonstrated on television in February 1956, creating a very strong impression. Like Turing, Samuel had trouble finding computer time. Working at night, he used machines that were still on the testing floor at IBM's manufacturing plant. Chapter 6 covers game playing, and Chapter 21 describes and expands on the learning techniques used by Samuel. John McCarthy moved from Dartmouth to MIT and there made three crucial contribu- tions in one historic year: 1958. In MIT A1 Lab Memo No. 1, McCarthy defined the high-level language Lisp, which was to become the dominant A1 programming language. Lisp is the Section 1.3. The History of Artificial Intelligence 19 second-oldest major high-level language in current use, one year younger than FORTRAN. With Lisp, McCarthy had the tool he needed, but access to scarce and expensive computing resources was also a serious problem. In response, he and others at MIT invented time shar- ing. Also in 1958, McCarthy published a paper entitled Programs with Common Sense, in which he described the Advice Taker, a hypothetical program that can be seen as the first complete A1 system. Like the Logic Theorist and Geometry Theorem Prover, McCarthy's program was designed to use knowledge to search for solutions to problems. But unlike the others, it was to embody general knowledge of the world. For example, he showed how some simple axioms would enable the program to generate a plan to drive to the airport to catch a plane. The program was also designed so that it could accept new axioms in the normal course of operation, thereby allowing it to achieve competence in new areas without being reprogrammed. The Advice Taker thus embodied the central principles of knowledge repre- sentation and reasoning: that it is useful to have a formal, explicit representation of the world and of the way an agent's actions affect the world and to be able to manipulate these repre- sentations with deductive processes. It is remarkable how much of the 1958 paper remains relevant even today. 1958 also marked the year that Marvin Minsky moved to MIT. His initial collabora- tion with McCarthy did not last, however. McCarthy stressed representation and reasoning in formal logic, whereas Minsky was more interested in getting programs to work and even- tually developed an anti-logical outlook. In 1963, McCarthy started the A1 lab at Stanford. His plan to use logic to build the ultimate Advice Taker was advanced by J. A. Robinson's discovery of the resolution method (a complete theorem-proving algorithm for first-order logic; see Chapter 9). Work at Stanford emphasized general-purpose methods for logical reasoning. Applications of logic included Cordell Green's question-answering and planning systems (Green, 1969b) and the Shakey robotics project at the new Stanford Research Insti- tute (SRI). The latter project, discussed further in Chapter 25, was the first to demonstrate the complete integration of logical reasoning and physical activity. Minsky supervised a series of students who chose limited problems that appeared to MICROWORLDS require intelligence to solve. These limited domains became known as microworlds. James Slagle's SAINT program (1963a) was able to solve closed-form calculus integration problems typical of first-year college courses. Tom Evans's ANALOGY program (1968) solved geomet- ric analogy problems that appear in IQ tests, such as the one in Figure 1.4. Daniel Bobrow's STUDENT program (1967) solved algebra story problems, such as the following: If the number of customers Tom gets is twice the square of 20 percent of the number of advertisements he runs, and the number of advertisements he runs is 45, what is the number of customers Tom gets? The most famous microvvorld was the blocks world, which consists of a set of solid blocks placed on a tabletop (or more often, a simulation of a tabletop), as shown in Figure 1.5. A typical task in this world is to rearrange the blocks in a certain way, using a robot hand that can pick up one block at a time. The blocks world was home to the vision project of David Huffman (1971), the vision and constraint-propagation work of David Waltz (1975), the learning theory of Patrick Winston (1970), the natural language understanding program 20 Chapter 1. Introduction Figure 1.4 An example problem solved by Evans's ANALOGY program. Figure 1.5 A scene from the blocks world. SHRDLU (Winograd, 1972) has just completed the command, "Find a block which is taller than the one you are holding and put it in the box." of Terry Winograd (1972), and the planner of Scott Fahlman (1974). Early work building on the neural networks of McCulloch and Pitts also flourished. The work of Winograd and Cowan (1963) showed how a large number of elements could collectively represent an individual concept, with a corresponding increase in robustness and parallelism. Hebb's learning methods were enhanced by Bernie Widrow (Widrow and Hoff,

Advise: Why You Wasting Money in Costly SEO Tools, Use World's Best Free SEO Tool Ubersuggest.