The Art of Doing Science and Engineering: Learning to Learn

by Richard Hamming

The joke around Princeton was that John von Neumann “was not human, but a demigod who had made a detailed study of humans and could imitate them perfectly.” Bad news for any aspiring von Neumanns of more terrestrial stock. Hamming’s conviction—indeed, obsession—was the opposite: that this greatness was less a matter of genius (or divinity), and more a kind of virtuosity. He saw these undeniably great figures as human beings that had learned how to do something, and by studying them, he could learn it too.

Teachers should prepare the student for the student’s future, not for the teacher’s past.

The belief anything can be “talked about” in words was certainly held by the early Greek philosophers Socrates (469–399 bc), Plato (427–347 bc), and Aristotle (384–322 bc). This attitude ignored the current mystery cults of the time which asserted you had to “experience” some things which could not be communicated in words. Examples might be the gods, truth, justice, the arts, beauty, and love.

As I will several times say, there are so many ways of being wrong and so few of being right, studying successes is more efficient, and furthermore when your turn comes you will know how to succeed rather than how to fail!

Education is what, when, and why to do things. Training is how to do it. Either one without the other is not of much use. You need to know both what to do and how to do it.

The reason back-of-the-envelope calculations are widely used by great scientists is clearly revealed—you get a good feeling for the truth or falsity of what was claimed, as well as realize which factors you were inclined not to think about, such as exactly what was meant by the lifetime of a scientist. Having done the calculation you are much more likely to retain the results in your mind. Furthermore, such calculations keep the ability to model situations fresh and ready for more important applications as they arise. Thus I recommend when you hear quantitative remarks such as the above you turn to a quick modeling to see if you believe what is being said, especially when given in the public media like the press and tv. Very often you find what is being said is nonsense; either no definite statement is made which you can model, or, if you can set up the model, then the results of the model do not agree with what was said.

What is my answer to this dilemma? One answer is you must concentrate on fundamentals, at least what you think at the time are fundamentals, and also develop the ability to learn new fields of knowledge when they arise so you will not be left behind, as so many good engineers are in the long run.

How are you to recognize “fundamentals”? One test is they have lasted a long time. Another test is from the fundamentals all the rest of the field can be derived by using the standard methods in the field. I need to discuss science vs. engineering. Put glibly: In science, if you know what you are doing, you should not be doing it. In engineering, if you do not know what you are doing, you should not be doing it.

The past was once the future and the future will become the past.

Often it is not physical limitations which control but rather it is human-made laws, habits, and organizational rules, regulations, personal egos, and inertia which dominate the evolution to the future.

Reading some historians you get the impression the past was determined by big trends, but you also have the feeling the future has great possibilities. You can handle this apparent contradiction in at least four ways: You can simply ignore it. You can admit it. You can decide the past was a lot less determined than historians usually indicate, and individual choices can make large differences at times. Alexander the Great, Napoleon, and Hitler had great effects on the physical side of life, while Pythagoras, Plato, Aristotle, Newton, Maxwell, and Einstein are examples on the mental side. You can decide the future is less open-ended than you would like to believe, and there is really less choice than there appears to be. It is probable the future will be more limited by the slow evolution of the human animal and the corresponding human laws, social institutions, and organizations than it will be by the rapid evolution of technology. In spite of the difficulty of predicting the future and that unforeseen technological inventions can completely upset the most careful predictions, you must try to foresee the future you will face.

It is well known the drunken sailor who staggers to the left or right with n independent random steps will, on the average, end up about steps from the origin. But if there is a pretty girl in one direction, then his steps will tend to go in that direction and he will go a distance proportional to n. In a lifetime of many, many independent choices, small and large, a career with a vision will get you a distance proportional to n, while no vision will get you only the distance . In a sense, the main difference between those who go far and those who do not is some people have a vision and the others do not and therefore can only react to the current events as they happen. One of the main tasks of this course is to start you on the path of creating in some detail your vision of your future. If I fail in this I fail in the whole course. You will probably object that if you try to get a vision now it is likely to be wrong—and my reply is that from observation I have seen the accuracy of the vision matters less than you might suppose, getting anywhere is better than drifting, there are potentially many paths to greatness for you, and just which path you go on, so long as it takes you to greatness, is none of my business. You must, as in the case of forging your personal style, find your vision of your future career, and then follow it as best you can. No vision, not much of a future.

In forming your plan for your future you need to distinguish three different questions: What is possible? What is likely to happen? What is desirable to have happen? In a sense the first is science—what is possible. The second is engineering—what are the human factors which choose the one future that does happen from the ensemble of all possible futures. The third is ethics, morals, or whatever other word you wish to apply to value judgments. It is important to examine all three questions, and insofar as the second differs from the third, you will probably have an idea of how to alter things to make the more desirable future occur, rather than let the inevitable happen and suffer the consequences. Again, you can see why having a vision is what tends to separate the leaders from the followers.

Lastly, in a sense this is a religious course: I am preaching the message that, with apparently only one life to live on this earth, you ought to try to make significant contributions to humanity rather than just get along through life comfortably—that the life of trying to achieve excellence in some area is in itself a worthy goal for your life.

Society is steadily moving from a material goods society to an information service society. At the time of the American Revolution, say 1780 or so, over 90% of the people were essentially farmers—now farmers are a very small percentage of workers. Similarly, before wwii most workers were in factories—now less than half are there. In 1993, there were more people in government (excluding the military) than there were in manufacturing! What will the situation be in 2020? As a guess I would say less than 25% of the people in the civilian workforce will be handling things; the rest will be handling information in some form or other. In making a movie or a tv program you are making not so much a thing, though of course it does have a material form, as you are organizing information. Information is, of course, stored in a material form, say a book (the essence of a book is information), but information is not a material good to be consumed like food, a house, clothes, an automobile, or an airplane ride for transportation.

Setting aside the child’s view of a robot as a machine resembling a human, but rather thinking of it as a device for handling and controlling things in the material world, robots used in manufacturing do the following: Produce a better product under tighter control limits. Produce usually a cheaper product. Produce a different product. This last point needs careful emphasis. When we first passed from hand accounting to machine accounting we found it necessary, for economical reasons if no other, to somewhat alter the accounting system. Similarly, when we passed from strict hand fabrication to machine fabrication we passed from mainly screws and bolts to rivets and welding. It has rarely proved practical to produce exactly the same product by machines as we produced by hand. Indeed, one of the major items in the conversion from hand to machine production is the imaginative redesign of an equivalent product. Thus in thinking of mechanizing a large organization, it won’t work if you try to keep things in detail exactly the same, rather there must be a larger give and take if there is to be a significant success. You must get the essentials of the job in mind and then design the mechanization to do that job rather than trying to mechanize the current version—if you want a significant success in the long run.

I suggest you must rethink everything you ever learned on the subject, question every successful doctrine from the past, and finally decide for yourself its future applicability. The Buddha told his disciples, “Believe nothing, no matter where you read it, or who said it, no matter if I have said it, unless it agrees with your own reason and your own common sense.” I say the same to you—you must assume the responsibility for what you believe.

The first commercial production of electronic computers was under Mauchly and Eckert again, and since the company they formed was merged with another, their machines were finally called univacs. Especially noted was the one for the Census Bureau. ibm came in a bit late with 18 (20 if you count secret cryptographic users) ibm 701s. I well recall a group of us, after a session on the ibm 701 at a meeting where they talked about the proposed 18 machines, all believed this would saturate the market for many years! Our error was simply we thought only of the kinds of things we were currently doing, and did not think in the directions of entirely new applications of machines. The best experts at the time were flatly wrong! And not by a small amount either! Nor for the last time!

Here in the history of the growth of computers you see a realization of the S-type growth curve: the very slow start, the rapid rise, the long stretch of almost linear growth in the rate, and then the facing of the inevitable saturation.

Once a boss says “no!” it is very hard to get a different decision, so don’t let them say “no!” to a proposal.

However, let me observe in all honesty to the department head, it was remarks by him which made me realize it was not the number of operations done that mattered, it was, as it were, the number of micro-Nobel prizes I computed that mattered. Thus the motto of a book I published in 1961: The purpose of computing is insight, not numbers. A good friend of mine revised it to: The purpose of computing numbers is not yet in sight.

There is value in the machine view of a computer, that it is just a collection of storage devices and gates processing bits, and nothing more. This view is useful, at times, when debugging (finding errors) in a program; indeed, it is what you must assume when you try to debug. You assume the machine obeys the instructions one at a time, and does nothing more—it has no “free will” or any of the other attributes such as the self-awareness and self-consciousness we often associate with humans. How different are we in practice from the machines? We would all like to think we are different from machines, but are we essentially? It is a touchy point for most people, and the emotional and religious aspects tend to dominate most arguments.

The first published book devoted to programming was by Wilkes, Wheeler, and Gill, and applied to the Cambridge, England edsac (1951). I, among others, learned a lot from it, as you will see in a few minutes.

fortran, meaning formula translation, was proposed by Backus and friends, and again was opposed by almost all programmers. First, it was said it could not be done. Second, if it could be done, it would be too wasteful of machine time and capacity. Third, even if it did work, no respectable programmer would use it—it was only for sissies! The use of fortran, like the earlier symbolic programming, was very slow to be taken up by the professionals. And this is typical of almost all professional groups. Doctors clearly do not follow the advice they give to others, and they also have a high proportion of drug addicts. Lawyers often do not leave decent wills when they die. Almost all professionals are slow to use their own expertise for their own work. The situation is nicely summarized by the old saying, “The shoemaker’s children go without shoes.” Consider how in the future, when you are a great expert, you will avoid this typical error! With fortran available and running, I told my programmer to do the next problem in fortran, get her errors out of it, let me test it to see it was doing the right problem, and then she could, if she wished, rewrite the inner loop in machine language to speed things up and save machine time. As a result we were able, with about the same amount of effort on our part, to produce almost ten times as much as the others were doing. But to them programming in fortran was not for real programmers!

To see the obvious it often takes an outsider, or else someone like me who is thoughtful and wonders what he is doing and why it is all necessary. Even when told, the old timers will persist in the ways they learned, probably out of pride for their past and an unwillingness to admit there are better ways than those they were using for so long.

I started out (1956 or so) with the following four rules for designing a language: Easy to learn. Easy to use. Easy to debug (find and correct errors). Easy to use subroutines. The last is something which need not bother you, as in those days we made a distinction between “open” and “closed” subroutines, which is hard to explain now!

Until we better understand languages of communication involving humans as they are (or can be easily trained), it is unlikely many of our software problems will vanish.

There are many things we can do to reduce “the software problem,” as it is called, but it will take some basic understanding of language as it is used to communicate understanding between humans, and between humans and machines, before we will have a really decent solution to this costly problem. It simply will not go away.

You read constantly about “engineering the production of software,” both for the efficiency of production and for the reliability of the product. But you do not expect novelists to “engineer the production of novels.” The question arises: “Is programming closer to novel writing than it is to classical engineering?” I suggest yes! Given the problem of getting a man into outer space, both the Russians and the Americans did it pretty much the same way, all things considered, and allowing for some espionage. They were both limited by the same firm laws of physics. But give two novelists the problem of writing on “the greatness and misery of man,” and you will probably get two very different novels (without saying just how to measure this). Give the same complex problem to two modern programmers and you will, I claim, get two rather different programs. Hence my belief that current programming practice is closer to novel writing than it is to engineering. The novelists are bound only by their imaginations, which is somewhat as the programmers are when they are writing software. Both activities have a large creative component, and while you would like to make programming resemble engineering, it will take a lot of time to get there—and maybe you really, in the long run, do not want to do it! Maybe it just sounds good. You will have to think about it many times in the coming years; you might as well start now and discount propaganda you hear, as well as all the wishful thinking wh...

There are many proposals on how to improve the productivity of the individual programmer, as well as groups of programmers. I have already mentioned top-down and bottom-up; there are others, such as head programmer, lead programmer, proving the program is correct in a mathematical sense, and the waterfall model of programming, to name but a few. While each has some merit I have faith in only one, which is almost never mentioned—think before you write the program, it might be called. Before you start, think carefully about the whole thing, including what will be your acceptan...

One trouble with much of programming is simply that often there is not a well-defined job to be done; rather, the programming process itself will gradually discover what the problem is! The desire that you be given a well-defined problem before you start programming often does not match reality, and hence a lot of the current prop...

Many studies have shown programmers differ in productivity, from worst to best, by much more than a factor of ten. From this I long ago concluded the best policy is to pay your good programmers very well but regularly fire the poorer ones—if you can get away with it! One way is, of course, to hire them on contract rather than as regularly employed people, but that is increasingly against the law, which seems to want to guarantee even the worst have some employment. In practice you may actually be better off to pay the worst to stay home and not get in the way of the more capable (and I am serious)!

By the 1950s I had found I was frightened when giving public talks to large audiences, this in spite of having taught classes in college for many years. On thinking this over very seriously, I came to the conclusion I could not afford to be crippled that way and still become a great scientist; the duty of a scientist is not only to find new things, but to communicate them successfully in at least three forms: Writing papers and books Prepared public talks Impromptu talks Lacking any one of these would be a serious drag on my career. How to learn to give public talks without being so afraid was my problem. The answer was obviously by practice, and while other things might help, practice was a necessary thing to do.

A scientist should not give talks merely to entertain, since the object of the talk is usually scientific information transmission from the speaker to the audience. That does not imply the talk must be dull. There is a fine, but definite, line between scientific communication and entertainment, and the scientist should always stay on the right side of that line.

You should always feel some excitement when you give a talk, since even the best actors and actresses usually have some stage fright. Your excitement tends to be communicated to the audience, and if you seem to be perfectly relaxed, then the audience also relaxes and may fall asleep!

After giving the talk a few times I realized, of course, it was not just the hardware but also the software which would limit the evolution of computing as we approached the year 2000—Chapter 4 I just gave you. Finally, after a long time, I began to realize it was the economics, the applications, which probably would dominate the evolution of computers. Much, but by no means all, of what would happen had to be economically sound.

Yes, we did some of the hardest problems on the most primitive equipment—it was necessary to do this in order to prove machines could do things which could not be done otherwise. Then, and only then, could we turn to the economical solutions of problems which could be done only laboriously by hand! And to do this we needed to develop the basic theories of numerical analysis and practical computing suitable for machines rather than for hand calculations. This is typical of many situations. It is first necessary to prove beyond any doubt the new thing, device, method, or whatever it is, can cope with heroic tasks before it can get into the system to do the more routine, and, in the long run, more useful tasks. Any innovation is always against such a barrier, so do not get discouraged when you find your new idea is stoutly, and perhaps foolishly, resisted. By realizing the magnitude of the actual task you can then decide if it is worth your efforts to continue, or if you should go do something else you can accomplish and not fritter away your efforts needlessly against the forces of inertia and stupidity.

In the early evolution of computers I soon turned to the problem of doing many small problems on a big machine. I realized, in a very real sense, I was in the mass production of a variable product—I should organize things so I could cope with most of the problems which would arise in the next year, while at the same time not knowing what, in detail, they would be. It was then I realized that computers have opened the door much more generally to the mass production of a variable product, regardless of what it is: numbers, words, word processing, making furniture, weaving, or what have you. They enable us to deal with variety without excessive standardization, and hence we can evolve more rapidly to a desired future! You see it at the moment applied to computers themselves! Computers, with some guidance from humans, design their own chips, and computers are assembled, more or less, automatically from standard parts; you say what things you want in your computer and the particular computer is then made. Some computer manufacturers are now using almost total machine assembly of the parts with almost no human intervention. It was the attitude that I was in the mass production of a variable product, with all its advantages and disadvantages, which caused me to approach the ibm 650, as I told you about in the last chapter. By spending about one man year in total effort over a period of six months, I found at the end of the year I had more work done than if I had approached each p...

What will come along to sustain this straight line logarithmic growth curve and prevent the inevitable flattening out of the S-curve of applications? The next big area is, I believe, pattern recognition. I doubt our ability to cope with the most general problem of pattern recognition, because for one thing it implies too much, but in areas like speech recognition, radar pattern recognition, picture analysis and redrawing, workload scheduling in factories and offices, analysis of data for statisticians, creation of virtual images, and such, we can consume a very large amount of computer power. Virtual reality computing will become a large consumer of computing power, and its obvious economic value assures us this will happen, both in the practical needs and in amusement areas. Beyond these is, I believe, artificial intelligence, which will finally get to the point where the delivery of what they have to offer will justify the price in computing effort, and will hence be another source of problem solving.

This one experience led us at Bell Telephone Laboratories to start putting small computers into laboratories, at first merely to gather, reduce, and display the data, but soon to drive the experiment. It is often easier to let the machine program the shape of the electrical driving voltages to the experiment, via a standard digital-to-analog converter, than it is to build special circuits to do it. This enormously increased the range of possible experiments, and introduced the practicality of having interactive experiments. Again, we got the machine in under one pretext, but its presence in the long run changed both the problem and what the computer was actually used for. When you successfully use a computer you usually do an equivalent job, not the same old one. Again you see the presence of the computer, in the long run, changed the nature of many of the experiments we did.

I began mainly talking about general-purpose computers, but I gradually took up discussing the use of a general-purpose computer as a special-purpose device to control things, such as the cyclotron and laboratory equipment. One of the main steps happened when someone in the business of making integrated circuits for people noted that instead of making a special chip for each of several customers, he could make a four-bit general-purpose computer and then program it for each special job (intel 4004). He replaced a complex manufacturing job with a programming job, though of course the chip still had to be made, but now it would be a large run of the same four-bit chips. Again this is the trend I noted earlier, going from hardware to software to gain the mass production of a variable product—always using the same general purpose computer. The four-bit chip was soon expanded to eight-bit chips, then 16, etc., so now some chips have 64-bit computers on them!

You see many more special-purpose chips around than there need be. One of the main reasons is there is a great ego satisfaction in having your own special chip and not one of the common herd.

As you go on in your careers you should examine the applications which succeed and those which fail; try to learn how to distinguish between them; try to understand the situations which produce successes and those which almost guarantee failure. Realize, as a general rule, it is not the same job you should do with a machine, but rather an equivalent one, and do it so that future, flexible expansion can be easily added (if you do succeed). And always also remember to give serious thought to the field maintenance as it will actually be done in the field—which is generally not as you wish it would be done!

After quite a few years the field of the limits of intellectual performance by machines acquired the dubious title of artificial intelligence (ai), which does not have a single meaning. First, it is a variant on the question: Can machines think? While this is a more restricted definition than is artificial intelligence, it has a sharper focus and is a good substitute in the popular mind. This question is important to you because if you believe computers cannot think, then as a prospective leader you will be slow to use computers to advance the field by your efforts, but if you believe of course computers can think, then you are very apt to fall into a first-class failure! Thus you cannot afford to either believe or disbelieve—you must come to your own terms with the vexing problem, “To what extent can machines think?”

While the problem of ai can be viewed as, “Which of all the things humans do can machines also do?,” I would prefer to ask the question in another form: “Of all of life’s burdens, which are those machines can relieve, or significantly ease, for us?” Note that while you tend to automatically think of the material side of life, pacemakers are machines connected directly to the human nervous system and help keep many people alive. People who say they do not want their life to depend on a machine seem quite conveniently to forget this. It seems to me in the long run it is on the intellectual side of life that machines can most contribute to the quality of life.

They say piloting a plane is hours of boredom and seconds of sheer panic—not something humans were designed to cope with, though they manage to a reasonable degree.

The Turing test is a popular approach, but it flies in the face of the standard scientific method, which starts with the easier problems before facing the harder ones. Thus I soon raised the question with myself, “What is the smallest or close to the smallest program I would believe could think?” Clearly, if the program were divided into two parts, then neither piece could think. I tried thinking about it each night as I put my head on the pillow to sleep, and after a year of considering the problem and getting nowhere I decided it was the wrong question! Perhaps “thinking” is not a yes/no thing, but maybe it is a matter of degree.

Physics regards you as a collection of molecules in a radiant energy field, and there is, in strict physics, nothing else. Democritus (b. around 460 bc) said in ancient Greek times, “All is atoms and void.” This is the stance of the hard ai people; there is no essential difference between machines and humans, hence by suitably programming machines, the machines can do anything humans can do. Their failures to produce thinking in significant detail is, they believe, merely the failure of programmers to understand what they are doing, and not an essential limitation. At the other extreme of the ai scale, some of us, when considering our own feelings, believe we have self-awareness and self-consciousness—though we are not able to give satisfactory tests to prove these things exist. I can get a machine to print out “I have a soul” or “I am self-aware” or “I have self-consciousness,” and you would not be impressed with such statements from a machine. But from humans you are inclined to give greater credence to such remarks, based on the belief that you, by introspection, feel you have such properties (things), and you have learned by long experience in life that other humans are similar to you—though clearly racism still exists, which asserts there are differences, me being always the better person!

This is the standard structure of a program to play a game on a computer. Programs must first require you check the move is legal before any other step, but this is a minor detail. Then there is usually a set of more or less formal rules to be obeyed, followed by some much vaguer rules. Thus a game program has a lot of heuristics in it (heuristic—to invent or discover), moves which are plausible and likely to lead you to a win but are not guaranteed to do so.

You must struggle with your own beliefs if you are to make any progress in understanding the possibilities and limitations of computers in the intellectual area. To do this adequately you must formalize your beliefs and then criticize them severely, arguing one side against the other, until you have a fair idea of the strengths and weaknesses of both sides. Most students start out anti-ai; some are pro-ai; and if you are either one of these then you must try to undo your biases in this important matter.