Independent Component Analysis
Published: 2023-08-17 . Back to ≈

Understanding the independent components of a complex system is one of the most powerful tools available in engineering. If a system yields independent components, I only need to understand individual the dynamics of each component.

Independence takes the vast exponential space of possibilities of a system and “factorizes” it into a simple, intelligible, linear sort of space.

(Not every system consists of a set of independent factors. Sometimes, there are couplings–higher order interactions of components that are essential to the how the system works. But even in such instances, its often useful to start by finding the “approximately” independent components, and then considering how a small amount coupling modifies the system.)

Understanding independences is important for life as well.

  • If things are coupled in ways you don’t realize, you’ll make bad decisions
  • If things aren’t coupled in ways that you assume them to be, you’ll miss opportunities.

Independence is arguably even at the heart of language itself. Language naturally emerges in a way that factorizes the world (This needs an illustration). So when we want to understand the independence structure of something, language already goes a long way: what are the different words that serve as its traits or descriptors?

But language can sometimes give us more of a “statistical” independence rather than “causal” one. Where culture or norms bundle and correlate things together, words will often follow suit. This isn’t necessarily bad. But it means that there’s work to be done.

What are the independent components of our needs, for instance?

Many of our needs are met in bundles, which reflect higher order couplings between the needs (both statistical and causal). Meals bundle physical nutrients. Friendships bundle social nutrients.

But as in the engineering cases, if we want to understand the complex system of ourselves, it’s better to discard these couplings (at first) and get to know the finer needs.

This is especially true, e.g. of romantic relationships, which ambitiously aspire to bundle everything from sexual gratification to ultimate meaning and purpose. If we have no awareness of the nature of this bundle, but acknowledge only an unexamined and amorphous super-need by the name of “relationship,” we’re probably headed for rocky waters.

So let’s pick apart the the independent components of some things in our lives.

To be continued…