Induction is badly misunderstood. Here is an abridged and augmented excerpt (how’s that!) from Uncertainty’s Chapter 3.
There is no knowledge more certain than that provided by induction. Without induction, no argument could, as they say, get off the ground floor; this is because induction provides that ground floor. No argument could even be phrased if it were not for induction, because phrasing requires language and language requires induction. When we say apple, we know it applies to all apples via a form of induction.
All arguments must trace eventually back to some foundation. This foundational knowledge is first present in the senses. Through noesis or intellection, i.e. induction, first principles, universals, and essences are discovered. Induction is what accounts for our being certain, after observing only a finite number of instances or even one and sometimes even none, that all flames are hot, that all men are mortal, that white is always white, that for all natural numbers x and y, if x = y, then y = x, and for the content and characteristics of all other universals and axioms. Because we know these indubitable propositions more surely than any other, induction produces greater certainty than deduction.
Mistakes in induction occur, as they do in every area of intellectual activity. When a man sees several white swans and reasons, “All swans are white”, he is proved wrong when a black swan in sighted (as in Australia). It would be a gross mistake to condemn induction because of this error. It would be like throwing out algebra because middle schoolers miscalculate.
Since at least Hume it has been fashionable to pretend mystification about why induction is “justified”, or to claim that it is not. Hume said, “We have no reason to believe any proposition about the unobserved even after experience!” The prominent Bayesians Howson & Urbach say that there is no “solution” to induction and that this sad fate “is no longer controversial.” Karl Popper asked, “Are we rationally justified in reasoning from repeated instances of which we have experience [like the hot flames] to instances of which we have had no experience [this flame]?” His answer: “No”.
Fisher was not of the same skeptical bent, but he agreed in principle with Popperian ideas and used these beliefs to build his system of statistics. Theories (propositions) could only be “rejected” and never verified and so on (hell, p-values and hypothesis testing).
One reason induction is widely misunderstood, even considered a “problem” in the academic sense, is because it is analogical. Here, I largely follow Louis Groarke’s wonderful An Aristotelian Account of Induction, which is must reading. There is no way to adequately summarize the entire work, which is long and deep. Only a few highlights sufficient to dispel the sense that induction is “problematic” are given here.
“The goal of induction,” Groarke tells us, “is not simply to prove that something is the case but to provoke an understanding of the general case.” Induction moves from the particularities collected by the senses, and moves to unobservable, unsensible generalities or universals, such as knowledge of a thing’s essence. Induction starts with the finite and progresses to the infinite; so although we can never entirely grasp the infinite, we can and even must know part of it.
According to Groarke’s view, induction is “the cognitive/psychological mechanism that produces the leap of insight” necessary for all understanding. He gives five flavors, aspects, or facets of induction. These are (in my modified terms more useful for probability) (1) induction-intellection, (2) induction-intuition, (3) induction-argument, (4) induction-analogy, and (5) the most familiar induction-probability. The order is from that which provides the most to the least certainty.
(1) Induction-intellection is “induction proper” or “strict induction”. It is that which takes “data” from our limited, finite senses and provides “the most basic principles of reason.” Our senses provide information of the here-and-now (or there-and-then), but induction-intellection tells us what is always true everywhere and everywhen. We move with certainty from the particular to the general, from the finite to the infinite. Without this kind of induction, no argument can ever get anywhere, no argument can ever even start; without it language would not be possible.
Induction-intellection produces “Abstraction of necessary concepts, definitions, essences, necessary attributes, first principles, natural facts, moral principles.” In this way, induction is a superior form of reason than mere deduction, which is something almost mechanical. The knowledge provided by induction-intellection comes complete and cannot be deduced: it is the surest knowledge we have. Numbers come from this form of induction. We see one apple, two apples, three. And then comes 1, 2, 3, … Deduction has much to say about that “…”, but knowing that we can reason deductively comes from this form of induction.
(2) Induction-intuition is similar to induction-intellection. It “operates through cleverness, a general power of discernment of shrewdness” and provides knowledge of “any likeness or similitude, the general notion of belonging to a class, any discernment of sameness or unity.” Axioms arise from this form of induction. Axioms are of course the ground of all mathematical reasoning. We have to be careful because some use the word axiom loosely, and merely mean assumption, a proposition which is not necessarily believed but is desirable. By axiom I mean those base propositions which are fundamental and believed by all those who consider them (like Peano’s axioms, etc.).
The foundational rules of logic are provided to us by this form of induction. We observe that our mom is now in this room and now she isn’t, and from that induce the principle of non-contradiction, which cannot be proven any other way. No universal can be known except inductively because nobody can ever sense every thing. Language exists, and works, because induction-intuition.
(3) Induction-argument, given by inductive syllogisms, is the “most rigorous form of inductive inference” and provides knowledge of “Essential or necessary properties or principles (including moral knowledge)”. An example is when a physicist declines to perform an experiment on electron number 2 because he has already performed the experiment on electron number 1, and he claims all electrons are identical. Induction-argument can provide conditional certainty, i.e. conditional truth.
(4) Induction-analogy is the least rigorous but most familiar (in daily life) form of induction and provides knowledge of “What is plausible, contingent or accidental; knowledge relating to convention, human affairs.” This form of induction explains lawyer jokes (What’s the difference between a good lawyer and a bad lawyer? A bad lawyer makes your case drag on for years. A good lawyer makes it last even longer).
(5) Induction-probability of course is the subject of most of this book. It provides knowledge of “Accidental features, frequency of properties, correlations in populations” and the like. It is, as is well known by anybody reading these words, the most prone to error. But the error usually comes not in failing to see correlations and confusing accidental properties with essences, but in misascribing causes, in mistaking correlation for causation even though everybody knows the admonition against it.
See also the articles on the so-called problem of grue.