The Art of Doing Science and Engineering: Learning to Learn

by Richard Hamming

It is hard to get people to aggressively think about how things in their own area might be done differently.

Since the purpose, as stated above, is to get the reader to think more carefully on the awkward topics of machines “thinking” and their vision of their personal future, you the reader should take your own opinions and try first to express them clearly, and then examine them with counterarguments, back and forth, until you are fairly clear as to what you believe and why you believe it. It is none of the author’s business in this matter what you believe, but it is the author’s business to get you to think and articulate your position clearly.

gold

the surer you are of one side, the more you should probably argue the other side!

9 n–Dimensional space

10 Coding theory—I

Having looked at computers and how they operate, we now turn to the problem of the representation of information—how do we represent the information we want to process? Recall any meaning a symbol may have depends on how it is processed; there is no inherent meaning to the bits the machine uses.

Again, all we suppose is there is such a source, and we are going to encode it for transmission. The encoder is broken into two parts. The first half is called the source encoding, which as its name implies is adapted to the source, various sources having possibly different kinds encodings.

The second half of the encoding process is called channel encoding and it is adapted to the channel over which the encoded symbols are to be sent. Thus the second half of the encoding process is tuned to the channel. In this fashion, with the common interface, we can have a wide variety of sources encoded first to the common interface, and then the message is further encoded to adapt it to the particular channel being used.

If you have information and want it later, you encode it for storage and store it. Later, when you want it, it is decoded. Among encoding systems is the identity, no change in the representation.

Thus again, in some sense, the “meaning” is not contained in the actual words I use, but you supply a great deal of surrounding information when you make the translation from my words to your idea of what I said inside your head.

We have learned to “tune” the words we use to fit the person on the receiving end; we to some extent select according to what we think is the channel noise, though clearly this does not match the model I am using above, since there is significant noise in the decoding process, shall we say. This inability of the receiver to “hear what is said” by a person in a higher management position but to hear only what they expect to hear is, of course, a serious problem in every large organization, and is something you should be keenly aware of as you rise towards the top of the organization. Thus the representation of information in the formal theory we have given is mirrored only partly in life as we live it, but it does show a fair degree of relevance outside the formal bounds of computer usage where it is highly applicable.

11 Coding th...

12 Error-correcting codes

First, I wanted to be honest with you and show you how easy, if you will follow Pasteur’s rule that “luck favors the prepared mind,” it is to succeed by merely preparing yourself to succeed. Yes, there were elements of luck in the discovery; but there were many other people in much the same situation, and they did not do it! Why me? Luck, to be sure, but also I was preparing myself by trying to understand what was going on—more than the other people around, who were merely reacting to things as they happened, and not thinking deeply as to what was behind the surface phenomena.

Of course, as you go through life you do not know what you are preparing yourself for—only you want to do significant things and not spend the whole of your life being a “janitor of science,” or whatever your profession is. Of course luck plays a prominent role. But so far as I can see, life presents you with many, many opportunities for doing great things (define them as you will) and the prepared person usually hits one or more successes, and the unprepared person will miss almost every time.

There is one nasty thing to be mentioned, however: what it takes to be great in one age is not what is required in the next one. Thus you, in preparing yourself for future greatness (and the possibility of greatness is more common and easy to achieve than you think, since it is not common to recognize greatness when it happens under one’s nose), you have to think of the nature of the future you will live in. The past is a partial guide, and about the only one you have besides history is the constant use of your own imagination. Again, a random walk of random decisions will not get you anywhere near as far as those taken with your own vision of what your future should be.

13 Information theory

Thus one purpose of this presentation of information theory, besides its usefulness, is to sensitize you to this danger, or, if you prefer, to how to use it to get what you want! It has long been recognized the initial definitions determine what you find, much more than most people care to believe. The initial definitions need your careful attention in any new situation, and they are worth reviewing in fields in which you have long worked so you can understand the extent to which the results are a tautology and not real results at all. There is the famous story by Eddington about some people who went fishing in the sea with a net. Upon examining the size of the fish they had caught, they decided there was a minimum size to the fish in the sea! Their conclusion arose from the tool used and not from reality.

14 Digital filters—I

15 Digital filters—II

This is a common, endlessly made mistake; people always want to think that something new is just like the past—they like to be comfortable in their minds as well as their bodies—and hence they prevent themselves from making any significant contribution to the new field being created under their noses. Not everything which is said to be new really is new, and it is hard to decide in some cases when something is new, yet the all too common reaction of “it’s nothing new” is stupid. When something is claimed to be new, do not be too hasty to think it is just the past slightly improved—it may be a great opportunity for you to do significant things. But then again, it may be nothing new.

16 Digital filters—III

17 Digital filters—IV

In Chapter 26 I will take up the problem of dealing with the expert. Here you see a simple example of what happens all too often. The experts were told something in class when they were students first learning things, and at the time they did not question it. It becomes an accepted fact, which they repeat and never really examine to see if what they are saying is true or not, especially in their current situation.

My contribution? Mainly, first identifying the problem, next getting the right people together, then monitoring Kaiser to keep him straight on the fact that filtering need not have exclusively to do with time signals, and finally, reminding them of what they knew from statistics (or should have known and probably did not). It seems to me from my experience that this role is increasingly needed as people get to be more highly specialized and narrower and narrower in their knowledge. Someone has to keep the larger view and see to it things are done honestly.

In closing, if you do not, now and then, doubt accepted rules, it is unlikely you will be a leader into new areas; if you doubt too much you will be paralyzed and will do nothing. When to doubt, when to examine the basics, when to think for yourself, and when to go on and accept things as they are is a matter of style, and I can give no simple formula on how to decide. You must learn from your own study of life. Big advances usually come from significant changes in the underlying beliefs of a field. As our state of knowledge advances, the balances between aspects of doing research change. Similarly, when you are young serendipity has probably a long time to pay off, but when you are old it has little time and you should concentrate more on what is at hand.

18 Simulation—I

A simulation is the answer to the question, “What if…?”

This brings up another belief of mine—doing simple simulations at the early stages lets you get insights into the whole system which would be disguised in any full-scale simulation. I strongly advise, when possible, to start with the simple simulation and evolve it to a more complete, more accurate, simulation later so the insights can arise early. Of course, at the end, as you freeze the final design, you must put in all the small effects which could matter in the final performance. But (1) start as simply as you can, provided you include all the main effects, (2) get the insights, and then (3) evolve the simulation to the fully detailed one.

an active mind can contribute to a simulation even when you are dealing with experts in a field where you are a strict amateur. You, with your hands on all the small details, have a chance to see what they have not seen, and to make significant contributions, as well as save machine time! Again, all too often I have seen things missed during the simulation by those running it, and hence were not likely to get to the users of the results.

Beware of jargon—learn to recognize it for what it is, a special language to facilitate communication over a restricted area of things or events. But it also blocks thinking outside the original area it was designed to cover. Jargon is both a necessity and a curse. You should realize you need to be active intellectually to gain the advantages of the jargon and to avoid the pitfalls, even in your own area of expertise!

19 Simulation—II

The relevant accuracy and reliability of simulations are a serious problem. There is, unfortunately, no silver bullet, no magic incantation you can perform, no panacea for this problem. All you have is yourself.

Remember this fact—older minds have more trouble adjusting to new ideas than do younger minds—since you will be showing new ideas, and even making formal presentations, to older people throughout much of your career.

Old people are not very quick to grasp new ideas—it is not that they are dumb, stupid, or anything else like that, it is simply that older minds are usually slow to adjust to radically new ideas.

You are responsible for your decisions, and you cannot blame them on those who do the simulations, much as you wish you could. Reliability is a central question with no easy answers.

20 Simulation—III

center on the old expression “garbage in, garbage out,” often abbreviated gigo. The idea is if you put ill-determined numbers and equations (garbage) in, then you can only get ill-determined results (garbage) out. By implication the converse is tacitly assumed: if what goes in is accurate, then what comes out must be accurate. I shall show both of these assumptions can be false.

You are lovingly taught how one theory was displaced by another, but you are seldom taught to replace a nice theory with nothing but randomness! And this is what was needed: the ability to say the theory you just read is no good and there was no definite pattern in the data, only randomness.

Simulation is essential to answer the “what if…?,” but it is full of dangers and is not to be trusted just because a large machine and much time has been used to get the nicely printed pages or colorful pictures on the oscilloscope.

If you are the one to make the final decision, then in a real sense you are responsible. Committee decisions, which tend to diffuse responsibility, are seldom the best in practice—most of the time they represent a compromise which has none of the virtues of any path and tends to end in mediocrity. Experience has taught me that generally a decisive boss is better than a waffling one—you know where you stand and can get on with the work which needs to be done!

The “what if…?” will arise often in your futures, hence the need for you to master the concepts and possibilities of simulations, and be ready to question the result...

21 Fiber ...

22 Computer-aided instruction (CAI)

Hawthorne effect, that it is necessary to explain. At the Hawthorne plant of Western Electric, long, long ago, some psychologists were trying to improve productivity by making various changes in the environment. They painted the walls an attractive color, and productivity rose. They made the lighting softer, and productivity rose. Each change caused productivity to rise. One of the men got a bit suspicious and sneaked a change back to the original state, and productivity rose! Why? It appears that when you show you care, the person on the other end responds more favorably than if you appear not to care. The workers all thought the changes were being made for their benefit and they responded accordingly.

Flying, to a fair extent, I point out, is a conditioned response being trained into the human. It is not much thinking, though at times thinking is necessary; it is more training to react rapidly and correctly, both mentally and physically, to unforeseen emergencies. It seems to me that for this sort of training, where there is a conditioned response to be learned, machines can do a very good job. It happens as a child I learned fencing. In a duel there is no time for local thinking; you must make a rapid conditioned response. There is indeed a large overall planning of a duel, but moment to moment it must be a response which does not involve the delay of thinking.

What you learn from others you can use to follow; What you learn for yourself you can use to lead.

I argued that the speed in learning was a significant matter to organizations—rapid learners were much more valuable than were slow learners (other things being the same); it was part of our job to increase the speed of learning and mark for society those who were the better ones. Again, this is opinion, but surely you do not want very slow learners to be in charge of you. Speed in learning new things is not everything, to be sure, but it seems to me it is an important element.

Students are remarkably able to memorize their way through many math classes, and many do so. But when I get to analytic integration (I give the students a function and ask for its indefinite integral), there is no way they can memorize their way through the course the way I teach it. They must learn to recognize in an almost infinite number of disguises. For the first time in their career they are forced to learn to recognize forms independent of the particular representation—which is a basic feature of mathematics and general intelligence. To take analytic integration out of the course, or transfer it to routines in computers, is to defeat the purpose of a stage of learning something that is essential, in my opinion, unless something of equivalent difficulty is put in. The students must master abstract pattern recognition if they are to progress and use mathematics later in their careers.

In summary, as best I can, clearly in low-level conditioned response situations, typically associated with training, I believe computers can greatly aid in the learning process, but at the other end, high-level thinking, education, I am very skeptical. Skeptical mainly because we ourselves do not understand either what we want to do, nor what we are presently doing! We simply do not know what we mean by “the educated person,” let alone what it will mean in the year 2020. Without that knowledge, how am I to judge the success of any proposal which is tried? Between low-level training and high-level education there is a large area to be explored and exploited by organizations outside the universities, as well as inside.