ENTROPY-ITS SIGNIFICANCE

SUBMITTED BY:NAINA GUPTA

ENTROPYENTROPY

• Entropy is a mathematical quantity describing disorder or discontinuity.

• Boltzmann introduced the concept of entropy to measure the disorder in a thermodynamic system.

• Shannon used the concept of informational entropy to measure the uncertainty associated with given information.

• According to Wilson, four ways to view the concept of entropy can be defined:

- Entropy as a measure of system properties (eg order and disorder,reversibility and irreversibility, complexity and simplicity);

- Entropy as a measure of information, uncertainty, or probablity;

- Entropy as stastistics of a probability distribution for measure of information or uncertainity;

- Entropy as the negative of Bayesian log likelihood function for measure of function.

Information TheoryInformation Theory

SHANNON’S ENTROPYSHANNON’S ENTROPY

• It is generally defined as an average value of information rate to eliminate uncertainty, which is given by finite number of alternative events.

• Shannon defines the entropy as:

H = - ∑i=1P(si)log P(si)

• Where S is the system with finite number of possible events si, P(si) represents the probablity of event si and the summation is over the ranges 1,2,3…n.

• Maximum Entropy when all the probablities are equal and the most uncertain situation occurs.

• Mininum entropy when H=0

Entropy in case of two possibilities with Entropy in case of two possibilities with probablities p and (1-p)probablities p and (1-p)

• The entropy units are in bits and the binary logarithm is used.

• If the common logarithm is used, then the units are dits.

• And it is nits in case natural logarithm is used.

ELIMINATING THE ENTROPYELIMINATING THE ENTROPY

SPATIAL ENTROPY

• This was defined by Batty,from the information theory basis presented by Shannon.

• Where,xi represents the spatial interval size.This spatial component is implemented and this equation is more applicable in spatial analysis, such as comparison between different regions.

ENTROPY AS A USEFUL PART OF A SPATIAL VISUALIZATION AND

MODELLING• This figure shows the dependency between

the number of observation and the outcoming kriging estimators of the spatial phenomenon. Here ,there are number of observations that gives us no useful information for the outcoming spatial modelling.

• This figure shows the dependency of the outcoming amount of information, measured with the help of the entropy function on the number of modelled points.

• The last part is the spatial visualization of the natural phenomena.In this fig., the function of spatial entropy is the convex function and therefore there exists points where the next added interval to the visualization of the spatial phenomenon gives us much less information like the previous one.

• In this there are 4 GRID layers of the same climatic phenomemon but with different numbers of intervals,increasing from upper left to lower right picture.Each layers contains various amount of information,depending on various factors.

• The information theory,entropy and its spatial form is widely used in geographical research.

• Entropy concepts can be used to investigate channel networks. Much of the work employing entropy concepts in hydrology have been done with reference to the informational entropy.

CONCLUSIONCONCLUSION

Thank You!Thank You!