Taming chaos: Calculating probability in complex systems

Description

The use of measures from Information Theory for complex systems' analysis requires the estimation of probabilities. In practice, these probabilities need to be derived from finite data-sets, namely, electroencephalogram (EEG) signals coming from different brain regions, electrocardiogram (EKG) signals coming from the heart, or temperature anomalies coming from different Earth regions. Respectively, the complex systems in these cases are the brain, the heart, and the Earth climate — all being systems composed of many dynamically interacting components. The main reason behind using measures from Information Theory to analyse complex systems is that these measures help to better understand and predict their behaviour and functioning. However, calculating probabilities from observed data is never straightforward; in particular, up-to-now, we lack practical ways to define them without losing useful (or adding meaningless) information in the process. In order to minimise these spurious additions or losses, we propose here a method to derive these probabilities optimally. Our method makes an entropy-based encoding of the measured signals, thus, transforming them into easy-to-handle symbolic sequences containing most of the relevant information about the system dynamics. Consequently, we can find the Information Theory measures, or any other spatio-temporal average, when we seek analysing a complex system