Applied Logic - Lecture 4 part 1 Inductive reasoning

Applied LogicLecture 4 part 1 – Inductive reasoning

Marcin Szczuka

Institute of Informatics, The University of Warsaw

Monographic lecture, Spring semester 2018/2019

Marcin Szczuka (MIMUW) Applied Logic 2019 1 / 30

Lecture plan

1 Introduction

2 Incomplete inductive reasoningTypes of inductive reasoning

3 Towards Inductive Logic



Deduction vs. induction

All of the reasoning systems (logics) we have seen so far were based ondeduction. The worked with use of certain inference rules that allowedderivation of consequences from axioms. In particular, deductive systemsare closed with respect to creation of new notions and inference of trueconsequences.Reasoning based on pure deduction is very rare in real life. In most casesthey are limited to precise, mathematical models of reality (theoreticalphysics, mathematics, informatics, ..). A great example of work based onrigorous deductive reasoning is the monumental work by Euclid ofAlexandria, the Elements. In real-life scenarios it is usually hard to preservethe strict rules of deduction and the inferred conclusions are not alwaysabsolutely true.Therefore, in complement to deductive reasoning in real-life situation wefrequently use also induction and/or abduction.


Inductive reasoning

In simplest terms, inductive reasoning can be seen as inference

from particular to generalor from examples to rules.

Inductive reasoning is by nature imprecise.

Inductive reasoning is built on human capability of finding patterns andrules on the basis of observation of a finite (and possibly incomplete andimperfect) sample.

For example, on the basis of obserwation any “reasonable” thinker willconsider the following sentence to be empirically proven.

All crows are black, but not all cats are black.


Nuances of induction

The inference on the basis of empirical observation is as old as scientificreasoning itself. However, due to the lack of formalised approach, up to theend of middle ages the Aristotelean deductive reasoning was considered theonly “true” method for proving.

While Aristotle considered briefly the inductive method, but limited only tothe primitive complete enumerative induction. Nowadays the reasoning byinduction is much more precise and regularised.

First important questions regarding validity and practicality of simplified,enumerative approach to induction were posed by Francis Bacon(1561-1626). Bacon proposed more practical eliminative induction(induction by elimination). The principle of eliminative induction wasformulated more precisely by American philosopher John Stuarta Mill in ASystem of Logic, Ratiocinative and Inductive (1843) in the form of five socalled Mill’s methods (cannons).


Nuances of induction

After Bacon, eliminative induction principle was disputed by David Hume(1748). Immanuel Kant pitched in too. Hume, in his philosophical worksabout cognition and causality, proposed a novel view on induction andprovided a constructive critical Hume’s propositions became a cornerstoneof the modern understanding of inductive reasoning. Hume divides allreasoning into demonstrative, by which he means deductive, andprobabilistic, by which he means the generalization of causal reasoning.Today the idea that our knowledge of the world is not complete and preciseis commonplace. However, in Hume’s era it sounded like a shockingparadox, especially in light of recently formulated foundations of Newtonianphysics.Contemporary understanding of inductive reasoning system diverted fromthe Hume’s and Kant’s ideas towards inductive logic. Nowadays, instead ofanswering the question “what justifies truthfulness of the statement?” theresearchers concentrate on the question “why a given statement ispossible/probable?”. A prominent representative of this kind of approach is(amongst others) Rudolf Carnap.


Types of inductive reasoning

Complete inductionComplete induction (complete enumerative induction, exhaustive induction)is a reasoning method that establishes truthfulness of a rule (proposition)by checking all possible cases when the rule applies.Complete induction is in fact a certain, deductive method. It eliminatespossible contradiction by exhaustive enumeration of all positive cases. Inmost non-trivial situations the complete induction method is highlyinefficient.A special case on complete induction is the mathematical inductioncommonly used to prove mathematical theorems, especially in discretedomains. Somewhat against its name, mathematical induction is adeductive proof technique.


Eliminative induction

“...when you have eliminated all which isimpossible, then whatever remains, howeverimprobable, must be the truth.”

Sherlock Holmes

ŹAfter: Arthur Conan Doyle, The Blanched Soldier



Eliminative inductionThe simplest eliminative induction scheme (Bacon) is based on building alist of mutually exclusive hypotheses and then eliminating all but one ofthem through empiric experiment.J.S. Mill extended the eliminative induction principle by setting five rulesfor elimination of hypotheses (Mills methods). Mill’s methods allow for(partial) formalisation of inference by establishing relationships of the type“ Cause A yields result a” on the basis of series of observations. Forexample, the direct method of agreement (first method) facilitates thefollowing reasoning:Situation 1: We record appearance of causes A, B, C and results a, b, c.Situation 2: We record appearance of causes A, D, E and results a, d, e.Conclusion: We eliminate non-repeating (not directly agreeing)observations and we get “Cause A yields result a”.



Incomplete inductionIncomplete induction (incomplete enumerative induction) is the methodthat establishes a general rule (proposition) on the basis of a finitenumber of statements (observations) that confirm the rule. We reasonfrom sample about general regularities.Incompleteness of this reasoning scheme is a manifestation of the nature ofreality that we attempt to describe. In real life we almost never are able toobserve sufficiently many (or all) possible situations. Incompleteness alsomeans, that previously constructed theories may need to be modified(completed) as new observations emerge. For example, Einstein’s relativitytheory extends Newtonian mechanics.Incomplete induction is one of the most basic tools for all experimentalsciences. Many disciplines have developed frameworks for dealing withuncertainty introduced through use of incomplete induction, for exampleerror calculus, sta


Lecture plan

1 Introduction




The issue of induction

The issue of incomplete inductive reasoning has been considered byresearchers for centuries. The discussion about validity and necessity of itsuse in describing the universe can be traced back to Sextus Empiricus (3-2century B.C.). Throughout the ages some of the most illustrious minds,including Bacon, Cartesius, Kant, Newton, Mill, Hume and others,addressed the issue. In contemporary discourse on induction as a mean todiscovery important additions were made by eminent philosophers of sciece,including Karl Popper, Wesley C. Salmon and David Miller.

Construction of a logical system based on incomplete inductive reasoningposes a challenge. On the one hand, such logic should extend deductivesystems by addition of a mechanism for deriving conclusion that may notbe absolutely true. On the other hand we are not willing to part with theessential property of deductive systems:

True premises guarantee true conclusions.


Criterion of Adequacy

In order for inductive logic to be considered useful we usually expect that ithas a mechanism for establishing a level of support for conclusionsexpressed in it. This mechanism measures the degree of influence ofpremises’ truthfulness on validity of the conclusion. We expect that themeasure used by this mechanism satisfies the Criterion of Adequacy (CoA).

CoA - Criterion of AdequacyAs evidence accumulates, the degree to which the collection of trueevidence statements comes to support a hypothesis, as measured by thelogic, should tend to indicate that false hypotheses are probably false andthat true hypotheses are probably true.


Peculiarities of induction

To be able to use inductive reasoning principles (inductive logic) properly itis necessary to eliminate the possibility of creating paradoxical results orsophismata.

Inductive proof of immortalityFact 1 – n Many times (n� 1) I heard that somebody died.Fact n+ 1 Every time I heard that somebody died – it was not me.conclusion There are no observations supporting the fact of me dying.

Hence, I am immortal.

Obviously, this reasoning is a fallacy. It does not take into account anynegative information as well as considers only a part of positive informationthat by no “decent” measure can be considered complete. However, inpractical applications of inductive reasoning it is always prudent tointroduce safeguards against nonsense.


Lecture plan

1 Introduction





In everyday practice of using inductive inference we apply some standardschemes (methods). Among them are:

1 Inductive generalisation.2 Statistical syllogism.3 Simple/direct induction.4 Argument from analogy.5 Prediction.6 Causal inference. Etiology.

NOTE: Argument from analogy can be considered as a (very) special caseof simple/direct induction.


Inductive generalisation

Inductive generalisation is a method that proceeds from a premise about asample to a conclusion about the whole population.

RulePremise:In sample p taken from population P proportion q of cases meets thecondition A.Conclusion: The proportion q of the population P meets the condition A.

Note, that at the moment do not concern ourselves with the size andrepresentativeness of the sample or the value of p. In real-life scenarios theinfluence of these parameters has to be carefully checked in order to obtainvalid outcome.


Statistical syllogism

Syllogism is an inference technique that uses two premises, which share acommon element, to produce a conclusion that consist of two elementsthat appear in exactly one of premises.A statistical syllogism proceeds from a premise about entire population to aconclusion about an individual.

RulePremises:– In population P proportion q of cases meet the condition A.– NEW case s is in P .Conclusion:There is a probability which corresponds to q that s meets A.

Reasoning with use of statistical syllogism is prone to fallacies of typesecundum quid, typical for all syllogisms.


Fallacia dicto simpliciter

Errors in reasoning (fallacia) of secundum quid type appear when we usesyllogisms in an improper way. For example, a type of syllogism introducedby Aristotle:All men are mortal (major premise) and Socrates is a man (minor premise)hence we may validly conclude that Socrates is mortal.In case of statistical syllogism we may encounter two sub-types of thefallacy.

1 Accident – Fallacia a dicto simpliciter ad dictum secundum quid –inference of particular conclusion from general rule while ignoringimportant limitation, e.g., “If there are so many lazy students thenthere must be some lazy students in the room right now”.

2 Reverse accident – Fallacia a dicto secundum quid ad dictumsimpliciter – inference of general statement from particular one byomission of important specifying (narrowing) condition, e.g., “If it isallowed to kill in self-defense then killing is OK”.


Simple/direct induction

Direct (simple) induction proceeds from a premise about a group ofexamples (part of population) to a conclusion about another, previouslyunseen individual.

RulePremises:– In population P proportion q of known instances meets condition A.– NEW case s is in P .Conclusion:There is a probability which corresponds to q that s meets A.

In this particular example the rule for simple induction is a composition ofrules for generalisation and statistical syllogism. The conclusion fromgeneralisation becomes the first premise in statistical syllogism.


Argument from analogy

Similarity in some aspects determine similarity in other aspects.

RegułaPremises:– Cases (objects) s and t agree on conditions A,B,C.– Case (object) s meets condition D.Conclusion:It is very likely that t meets D.

Argument from analogy is frequently used in common sense reasoning aswell as scientific, legal and philosophical. A limited and strictly regulatedversion of this reasoning method is in the basis of a branch of AI known asCase Based Reasoning (CBR).


Prediction

A prediction draws a conclusion about a new (future) case by observationof collected (in the past) sample.

RulePremise:I dotychczas zaobserwowanej population P seen so far the proportion qof cases meets condition A.Conclusion:Newly observed s meets A with probability proportional to q.

Prediction is one of most frequently used methods of inductive reasoning.When regularised and formalised it is a basis of major fields of CS/AI suchas Machine Learning and Knowledge Discovery in Databases (KDD).


Etiology – causality

Etiology (αιτιoλoγια) is a branch of science dealing with causality(investigating causes) of phenomena, processes or facts, especially indomains such as criminology or disease control.In view of inductive reasoning, especially the part that is used in computerscience, investigation of causalities is frequently reduced to:

Finding causality from dataIn the simplest case, let us consider two facts (two variables) X and Y .Usually, we assume that X and Y are time-dependent.We check, using available data, which of relations X → Y or Y → Xtreated as a hypothesis (potential conclusion) has more supportingevidence in data.Particular methods and algorithms diffre in the way they establish support.


Lecture plan

1 Introduction




Expectations for inductive logic

From a (quasi-)formal system that we dare to call inductive logic weexpect:

1 Fulfillment of the Criterion of Adequacy (CoA).2 Ensuring, that the degree of confidence in the inferred conclusion is no

greater than the confidence of the premises and inference rules.3 Ability to clearly discern between proper conclusions (hypotheses) and

nonsensical ones (vide: proof of immortality).Additional expectation is the intuitive interpretation. However, intuition isnot always helpful, as demonstrated in the next slide.


Monty Hall Paradox

Called that to commemorate the host of quiz show “Let’s Make a Deal”running for many years on TV in the US. It’s not really a paradox. It israther a demonstration that our intuitive understanding of“statistical/probabilistic” rules is frequently shallow and error-prone.

Monty Hall ParadoxThe Player faces three doors. Behind each door is either a prize (e.g. newcar) or one of two goats. The Host asks the Player where (behind whichdoor) is the prize. Then, the Host (who knows where the prize is) opensone of the doors not chosen by the Player. Behind that door is the goat.Now, the Host asks the Player if he wants to change the initial door choice.What should the Player decide in order to maximise his chance of winningthe prize. Should he stick to the first choice or make switch?

The answer is so counter-intuitive that even Paul Erdős did not believe in ituntil in 1995 he was shown a proof with use of a decision tree andcomputer simulation.


Inductive = statistical?

Frequently, practical inductive reasoning systems are based on elements ofprobabilistic and/or statistical inference. If all proper precautions are taken,this is not a bad approach. Historically, probabilistic approach was one ofthe first properly formalised and practically used.

One of the most frequently used methodologies is Bayesian (probabilistic)reasoning. This approach is sometimes – a liitle bit excessively – calledBayesian LOGic (BLOG).

If we identify support, certainty or plausibility measures with empiricalprobability we may utilise all of the very nice and powerful “machinery” ofprobability theory, hence obtaining a proper formal reasoning scheme.

However, when using probabilistic interpretations in inductive reasoning onehas to be cautious, as they tend to diverge from the intuitive understandingof “natural” induction.


Uncertain reasoning

As mentioned earlier, most models of inductive inference representincomplete uncertain reasoning. In majority of cases they are alsonon-monotonic, i.e., with appearance of new evidence (new observations)the conclusions drawn inductively before may be eliminated (contradicted).Reasoning in the presence of uncertainty is widely recognised andinvestigated in many branches of scientific investigations. Some significantapproaches include:

Plausibility relations – Relacje wiarygodności;Dempster-Shafer belief functions – Funkcje przekonańDempstera-Shafera;Qualitative probability relations – Jakościowe relacjeprawdopodobieństwa;Probability functions – Funkcje probabilistyczne;Possibility functions in Fuzzy Logic – Funkcje possybilistyczne(rozmyte);Ranking functions – Funkcje rankujące (sic!).


Uncertain reasoning

Figure below present some approaches to uncertain reasoning. Arrowsindicate “strength” of approach – from more to less general.

Qualitative probability relations

Possibilitic functions

(fuzzy)

Plausibility relations

Ranking functions

Probabilistic functions

Dempster-Shafer belief functions


Documents

Applied Logic - Lecture 4 part 1 Inductive reasoning