The Art of Doing Science and Engineering: Learning to Learn

by Richard Hamming

We live in an age of exponential growth in knowledge, and it is increasingly futile to teach only polished theorems and proofs. We must abandon the guided tour through the art gallery of mathematics, and instead teach how to create the mathematics we need. In my opinion, there is no long-term practical alternative. —Richard Hamming

This is a book about thinking. One cannot talk about thinking in the abstract, at least not usefully. But one can talk about thinking about digital filters, and by studying how great scientists thought about digital filters, one learns, however gradually, to think like a great scientist.

great people do and the rest do not.

How to be a great painter cannot be taught in words; one learns by trying many different approaches that seem to surround the subject. Art teachers usually let the advanced student paint, and then make suggestions on how they would have done it, or what might also be tried, more or less as the points arise in the student’s head—which is where the learning is supposed to occur!

Teachers should prepare the student for the student’s future, not for the teacher’s past. Most teachers rarely discuss the important topic of the future of their field, and when this is pointed out they usually reply: “No one can know the future.” It seems to me the difficulty of knowing the future does not absolve the teacher from seriously trying to help the student to be ready for it when it comes. It is obvious the experience of an individual is not necessarily that of a class of individuals; therefore, any one person’s projection into the future is apt to be somewhat personal and will not be universally accepted. This does not justify reverting to impersonal surveys and losing the impact of the personal story.

Apparently an “art”—which almost by definition cannot be put into words—is probably best communicated by approaching it from many sides and doing so repeatedly, hoping thereby students will finally master enough of the art, or if you wish, style, to significantly increase their future contributions to society.

1 Orientation

there are so many ways of being wrong and so few of being right, studying successes is more efficient, and furthermore when your turn comes you will know how to succeed rather than how to fail!

I am, as it were, only a coach. I cannot run the mile for you; at best I can discuss styles and criticize yours. You know you must run the mile if the athletics course is to be of benefit to you—hence you must think carefully about what you hear or read in this book if it is to be effective in changing you—which must obviously be the purpose of any course. Again, you will get out of this course only as much as you put in, and if you put in little effort beyond sitting in the class or reading the book, then it is simply a waste of your time. You must also mull things over, compare what I say with your own experiences, talk with others, and make some of the points part of your way of doing things.

I will show you my style as best I can, but, again, you must take those elements of it which seem to fit you, and you must finally create your own style. Either you will be a leader or a follower, and my goal is for you to be a leader. You cannot adopt every trait I discuss in what I have observed in myself and others; you must select and adapt, and make them your own if the course is to be effective.

Education is what, when, and why to do things. Training is how to do it. Either one without the other is not of much use. You need to know both what to do and how to do it.

The reason back-of-the-envelope calculations are widely used by great scientists is clearly revealed—you get a good feeling for the truth or falsity of what was claimed, as well as realize which factors you were inclined not to think about, such as exactly what was meant by the lifetime of a scientist. Having done the calculation you are much more likely to retain the results in your mind. Furthermore, such calculations keep the ability to model situations fresh and ready for more important applications as they arise. Thus I recommend when you hear quantitative remarks such as the above you turn to a quick modeling to see if you believe what is being said, especially when given in the public media like the press and tv. Very often you find what is being said is nonsense;

In science, if you know what you are doing, you should not be doing it. In engineering, if you do not know what you are doing, you should not be doing it.

In any case, I will often use history as a background for the extrapolations I make. I believe the best predictions are based on understanding the fundamental forces involved, and this is what I depend on mainly. Often it is not physical limitations which control but rather it is human-made laws, habits, and organizational rules, regulations, personal egos, and inertia which dominate the evolution to the future. You have not been trained along these lines as much as I believe you should have been, and hence I must be careful to include them whenever the topics arise.

In a lifetime of many, many independent choices, small and large, a career with a vision will get you a distance proportional to n, while no vision will get you only the distance . In a sense, the main difference between those who go far and those who do not is some people have a vision and the others do not and therefore can only react to the current events as they happen.

play. In forming your plan for your future you need to distinguish three different questions: What is possible? What is likely to happen? What is desirable to have happen? In a sense the first is science—what is possible. The second is engineering—what are the human factors which choose the one future that does happen from the ensemble of all possible futures. The third is ethics, morals, or whatever other word you wish to apply to value judgments. It is important to examine all three questions, and insofar as the second differs from the third, you will probably have an idea of how to alter things to make the more desirable future occur, rather than let the inevitable happen and suffer the consequences. Again, you can see why having a vision is what tends to separate the leaders from the followers.

I am preaching the message that, with apparently only one life to live on this earth, you ought to try to make significant contributions to humanity rather than just get along through life comfortably—that the life of trying to achieve excellence in some area is in itself a worthy goal for your life. It has often been observed the true gain is in the struggle and not in the achievement—a life without a struggle on your part to make yourself excellent is hardly a life worth living.

Great thought for the pursuit

2 Foundations of the digital (discrete) revolution

You must get the essentials of the job in mind and then design the mechanization to do that job rather than trying to mechanize the current version—if you want a significant success in the long run. I need to stress this point: mechanization requires you produce an equivalent product, not identically the same one. Furthermore, in any design it is now essential to consider field maintenance since in the long run it often dominates all other costs. The more complex the designed system, the more field maintenance must be central to the final design. Only when field maintenance is part of the original design can it be safely controlled; it is not wise to try to graft it on later. This applies to both mechanical things and to human organizations.

Among other evils of micromanagement is lower management does not get the chance to make responsible decisions and learn from their mistakes, but rather, because the older people finally retire, then lower management finds itself as top management—without having had many real experiences in management!

Furthermore, central planning has been repeatedly shown to give poor results (consider the Russian experiment, for example, or our own bureaucracy). The persons on the spot usually have better knowledge than can those at the top and hence can often (not always) make better decisions if things are not micromanaged. The people at the bottom do not have the larger, global view, but at the top they do not have the local view of all the details, many of which can often be very important, so either extreme gets poor results.

I believe computers will be almost everywhere, since I once saw a sign which read, “The battlefield is no place for the human being.” Similarly for situations requiring constant decision making.

I suggest you must rethink everything you ever learned on the subject, question every successful doctrine from the past, and finally decide for yourself its future applicability. The Buddha told his disciples, “Believe nothing, no matter where you read it, or who said it, no matter if I have said it, unless it agrees with your own reason and your own common sense.” I say the same to you—you must assume the responsibility for what you believe.

As a last observation in this area let me talk about special-purpose ic chips. It is immensely ego gratifying to have special-purpose chips for your special job, but there are very high costs associated with them. First, of course, is the design cost. Then there is the “troubleshooting” of the chip. Instead, if you find a general-purpose chip, which may possibly cost a bit more, then you gain the following advantages: Other users of the chip will help find the errors, or other weaknesses, if there are any. Other users will help write the manuals needed to use it. Other users, including the manufacturer, will suggest upgrades of the chip, hence you can expect a steady stream of improved chips with little or no effort on your part. Inventory will not be a serious problem. Since, as I have repeatedly said, technical progress is going on at an increasing rate, it follows technological obsolescence will be much more rapid in the future than it is now. You will hardly get a system installed and working before there are significant improvements which you can adapt by mere program changes if you have used general-purpose chips and good programming methods rather than your special-purpose chip, which will almost certainly tie you down to your first design. Hence beware of special-purpose chips! Though many times they are essential.

3 History of computers—hardware

4 History of computers—software

The use of fortran, like the earlier symbolic programming, was very slow to be taken up by the professionals. And this is typical of almost all professional groups. Doctors clearly do not follow the advice they give to others, and they also have a high proportion of drug addicts. Lawyers often do not leave decent wills when they die. Almost all professionals are slow to use their own expertise for their own work. The situation is nicely summarized by the old saying, “The shoemaker’s children go without shoes.” Consider how in the future, when you are a great expert, you will avoid this typical error!

History tends to be charitable in this matter. It gives credit for understanding what something means when we first do it. But there is a wise saying, “Almost everyone who opens up a new field does not really understand it the way the followers do.” The evidence for this is, unfortunately, all too good. It has been said in physics no creator of any significant thing ever understood what he had done. I never found Einstein on the special relativity theory as clear as some later commentators. And at least one friend of mine has said, behind my back, “Hamming doesn’t seem to understand error correcting codes!” He is probably right; I do not understand what I invented as clearly as he does. The reason this happens so often is the creators have to fight through so many dark difficulties, and wade through so much misunderstanding and confusion, they cannot see the light as others can, now the door is open and the path made easy. Please remember, the inventor often has a very limited view of what he invented, and some others (you?) can see much more. But also remember this when you are the author of some brilliant new thing; in time the same will probably be true of you. It has been said Newton was the last of the ancients and not the first of the moderns, though he was very significant in making our modern world.

Returning to the ibm 650 and me. I started out (1956 or so) with the following four rules for designing a language: Easy to learn. Easy to use. Easy to debug (find and correct errors). Easy to use subroutines. The last is something which need not bother you, as in those days we made a distinction between “open” and “closed” subroutines, which is hard to explain now! You might claim I was doing top-down programming, but I immediately

Great approach

But humans are unreliable and require redundancy; our spoken language tends to be around 60% redundant, while the written language is around 40%. You probably think the written and spoken languages are the same, but you are wrong. To see this difference, try writing dialogue and then read how it sounds. Almost no one can write dialogue so that it sounds right, and when it sounds right it is still not the spoken language. The human animal is not reliable, as I keep insisting, so low redundancy means lots of undetected errors, while high redundancy tends to catch the errors.

One trouble with much of programming is simply that often there is not a well-defined job to be done; rather, the programming process itself will gradually discover what the problem is! The desire that you be given a well-defined problem before you start programming often does not match reality, and hence a lot of the current proposals to “solve the programming problem” will fall to the ground if adopted rigorously.

key for having solid foundations but flexibility in methods and application

5 History of computer applications

As you go on in your careers you should examine the applications which succeed and those which fail; try to learn how to distinguish between them; try to understand the situations which produce successes and those which almost guarantee failure.

6 Limits of computer applications—AI–I

There is now a whole area known as expert systems. The idea is you talk with some experts in a field, extract their rules, put these rules into a program, and then you have an expert! Among other troubles with this idea is that in many fields, especially in medicine, the world-famous experts are in fact not much better than the beginners! It has been measured in many different studies! Another trouble is experts seem to use their subconscious, and they can only report their conscious experience in making a diagnosis. It has been estimated it takes about ten years of intensive work in a field to become an expert, and in this time many, many patterns are apparently laid down in the mind, from which the expert then makes a subconscious initial choice of how to approach the problem, as well as the subsequent steps to be used.

Can machines think? While this is a more restricted definition than is artificial intelligence, it has a sharper focus and is a good substitute in the popular mind. This question is important to you because if you believe computers cannot think, then as a prospective leader you will be slow to use computers to advance the field by your efforts, but if you believe of course computers can think, then you are very apt to fall into a first-class failure! Thus you cannot afford to either believe or disbelieve—you must come to your own terms with the vexing problem, “To what extent can machines think?”

Note, first, it really is misstated—the question seems to be more, “Can we write programs which will produce ‘thinking’ from a von Neumann-type machine?”

The question is not can machines think, it is can you write a program so the machine can think.

Is it not fair to say, “The program learned from experience”? Your immediate objection is that there was a program telling the machine how to learn. But when you take a course in Euclidean geometry, is not the teacher putting a similar learning program into you? Poorly, to be sure, but is that not, in a real sense, what a course in geometry is all about? You enter the course and cannot do problems; the teacher puts into you a program and at the end of the course you can solve such problems. Think it over carefully. If you deny the machine learns from experience because you claim the program was told (by the human programmer) how to improve its performance, then is not the situation much the same with you, except you are born with a somewhat larger initial program compared to the machine when it leaves the manufacturer’s hands? Are you sure you are not merely “programmed” in life by what chance events happen to you?

In a sense you will never really grasp the whole problem of ai until you get inside and try your hand at finding what you mean and what machines can do. Before the checkers-playing program which learned was exposed in simple detail, you probably thought machines could not learn from experience—now you may feel what was done was not learning but clever cheating, though clearly the program modified its behavior depending on its experiences. You must struggle with your own beliefs if you are to make any progress in understanding the possibilities and limitations of computers in the intellectual area. To do this adequately you must formalize your beliefs and then criticize them severely, arguing one side against the other, until you have a fair idea of the strengths and weaknesses of both sides.

7 Limits of computer applications—AI–II

Here you see again the effects of computers and how they are pushing us from the world of things into the world of ideas, and how they are supplementing and extending what humans can do. This is the type of ai that I am interested in—what can the human and machine do together, and not in the competition which can arise.

In many hospitals computers monitor patients in the emergency ward, and sometimes in other places when necessary. The machines are free from boredom, rapid in response, and will alert a local nurse to do something promptly. Unaided by computers it is doubtful full-time nurses could equal the combination of computer and nurse.

A routine response to non-routine situations can spell disaster.

Finally, perhaps thinking should be measured not by what you do but how you do it. When I watch a child learning how to multiply two, say, three-digit numbers, then I have the feeling the child is thinking; when I do the same multiplication I feel I am more doing “conditioned responses”; when a computer does the same multiplication I do not feel the machine is thinking at all. In the words of the old song, “It ain’t what you do, it’s the way that you do it.” In the area of thinking, maybe we have confused what is done with the way it is done, and this may be the source of much of our confusion in ai.

8 Limits of computer applications—AI–III

Thinking may be a matter of degree and not a yes/no thing.

Consider that in thinking, it may be the way something is done rather than what is done which determines whether it occurs or not. ai has traditionally stuck to the “what is done” and seldom considered the “how it is done.”

The plain fact is your life is often controlled by machines, and sometimes they are essential to your life—you just do not like to be reminded of it.

Another level of objection to the use of computers is in the area of experts. People are sure the machine can never compete, ignoring all the advantages the machines have (see end of Chapter 1). These are: economics, speed, accuracy, reliability, rapidity of control, freedom from boredom, bandwidth in and out, ease of retraining, hostile environments, and personnel problems. They always seem to cling to their supposed superiority rather than try to find places where machines can improve matters!

It is the combination of man and machine which is important, and not the supposed conflict which arises from their all too human egos.