Conditional entropy

The conditional entropy is an entropy measure used in information theory. The conditional entropy measures how much entropy a random variable Y has remaining if we have already learned completely the value of a second random variable X. It is referred to as the entropy of Y conditional on X, and is written H(Y | X). Like other entropies, the conditional entropy is measured in bits.

Given random variables X and Y with entropies H(X) and H(Y), and with a joint entropy H(X,Y), the conditional entropy of Y given X is defined as H(Y|X) \equiv H(X,Y) - H(X). Intuitively, the combined system contains H(X,Y) bits of information. If we learn the value of X, we have gained H(X) bits of information, and the system has H(Y | X) bits remaining.

H(Y | X) = 0 if and only if the value of Y is completely determined by the value of X. Conversely, H(Y | X) = H(Y) if and only if Y and X are independent random variables.

In quantum information theory, the conditional entropy is generalized to the conditional quantum entropy.

This article is licensed under the GNU Free Documentation License. It uses material from Wikipedia article. Browse Wikipedia for more information.