Thinking About Thinking: The Discovery of the LMS Algorithm IEEE SIGNAL PROCESSING MAGAZINE [100] JANUARY 2005 [ dsp HISTORY ] Bernard Widrow I t was the summer of 1956. I was at the Massachusetts Institute of Technology (MIT) and had just fin- ished my doctoral thesis on the theo- ry of quantization noise, in the field of digital signal processing. During the past academic year, I had been working very hard on my thesis and teaching courses in the fields of radar, digital signal processing, and digital controls. The sum- mer was a pleasant time, much more relaxed. I was looking forward to the fall semester, when I would join the MIT fac- ulty as an assistant professor of electrical engineering. A colleague in our laborato- ry, Ken Shoulders, told me about an ongoing seminar that summer at Dartmouth College on the subject of arti- ficial intelligence (AI). He planned to go to Dartmouth to learn about AI and about the progress that had been made in the field. I agreed to go with him. The Dartmouth seminar was really the begin- ning of the field of AI. The founders and pioneers were there. The discussions were highly stimulating and inspirational. We joined the seminar for one week and then tore ourselves away and returned to MIT. Some people at the Dartmouth AI meet- ing were seriously considering building an artificial brain. I was so taken by this that I never got over it. I spent the next six months thinking about thinking. I forgot all about digital signal processing and the theory of quantization noise. I began to see a connection between problem solving and game playing, and I began to contemplate building a prob- lem-solving machine that could perform simple reasoning. I concluded, however, that it would take about 25 years to do this, given the state of electronics at that time. Ken Shoulders was working on some basic ideas for integrated circuits, but they were years away. Being interest- ed in teaching and academic research, I realized that a 25-year time horizon for practical realization was too far out, and with the “publish or perish” system, I couldn’t work on this subject and suc- ceed as an academic. I was lucky to have realized this at an early stage. I dropped out of AI, but I never lost my interest in it. Almost 50 years have gone by since then, and we are not even close to build- ing an artificial brain. Maybe we will be able to do it during the next 50 years. After working on AI for six months, I was very anxious to get back to some- thing with a more near-term payoff. I returned to the field of digital signal processing. I was familiar with Wiener filter theory in both its continuous and discrete forms. To design a Wiener filter, you need to know the autocorrelation IEEE SIGNAL PROCESSING MAGAZINE [100] JANUARY 2005 In this issue, our guest is Dr. Bernard Widrow. Born during the winter holiday season of 1929 in a small town in Connecticut, Dr. Widrow gladly remembers the advice received in his youth to have the courage to apply to the Massachusetts Institute of Technology (MIT), even though he didn’t know a soul. Bernard Widrow applied to MIT, was admitted, and then completed his S.B. (1951), S.M. (1953), and Sc.D. (1956) degrees, all in electrical engineering. After spending a few more years at MIT as a fac- ulty member, he joined Stanford University in 1959 and later became a professor of electrical engineering there. Over the past half century, Dr. Widrow’s work has focused on numerous aspects of adaptive digital signal processing: noise canceling, antennas, inverse control, and non- linear filtering. He coauthored the books Adaptive Signal Processing (1985), Adaptive Control (1996), and Quantization Noise (to appear). Bernard Widrow has been award- ed prestigious distinctions, including the IEEE Centennial Medal (1984), the IEEE Alexander Graham Bell Medal (1986), the IEEE Neural Networks Pioneer Medal (1991), and the IEEE Millennium Medal (2000). He was also inducted into the National Academy of Engineering (1995) and the Silicon Valley Engineering Council Hall of Fame (1999). Nicknamed “Doc” by his students, Bernie Widrow values a “can do” attitude in his collaborators and appreciates their faith in him. He confesses that getting a major new idea approximately every five years stimulates him greatly, whereas the times when a new idea doesn’t work keep him grounded. Between research, decade-long collaborations (such as that with John McCool on adaptive filtering and applications), and teaching, he finds balance by enjoying opera, symphony, and ballet; collecting art; going to museums; and watching movies. For the curious journalist, “Doc” admits that his artistic interests may be partly inherited, as he is related to the famous painter Marc Chagall, who was his grandmother’s cousin. Needless to say, sharing impressions with Bernard Widrow on the daunting Chagall exhibition held at the San Francisco Museum of Modern Art (2003) is a delight. “Doc” also likes traveling, visiting his chil- dren and grandchildren, and going out for walks. We invite you to join him as he is “thinking about thinking” and recalling the events related to the discovery of the LMS algorithm. —Adriana Dumitras and George Moschytz “DSP History” column editors adrianad@ieee.org, moschytz@isi.ee.ethz.ch 1053-5888/05/$20.00©2005IEEE