Suppose $X$ and $Z$ are conditionally independent given $Y.$ Then, using Bayes' Theorem we have

Then,

Now assume we have Markovicity, i.e $p(x,y,z)=p(x)p(y|x)p(z|y).$ Again, using Bayes:

So, for a fixed $y$ we have $$\frac{p(x,y,z)}{p(y)}=p(x|y)p(z|y).$$

$◻$

Using Bayes' Theorem,

Then,

$◻$

If $Z=f(Y),$ then $Z$ is independent of $X,$ so

$$p(y,z|x)=p(y|x,z)=p(y|x)p(z|y),$$

and then

$$p(x,y,z)=p(x)p(y|x)p(z|y).$$

$◻$

Due to the magic of linear algebra, if we have a row vector $\mathbf{x}$ representing the probability of starting in each state, we can compute the probability of ending up in any given state after $m$ steps as

$$\mathbf{x}\cdot P^m.$$

$◻$

Let $E$ be the set of probability vectors in ${\mathbb{R}}^n.$ That is,

$$E=\{\mathbf{p}\in {\mathbb{R}}^n:{\mathbf{p}}_i\ge 0,\sum _{i=1}^n{\mathbf{p}}_i=1.\}$$

(Note: $E$ is a $n-1$ dimensional @simplex).

Suppose $\mathbf{X},\mathbf{Y}\in E.$ Then

$$\sum _{i=1}^n{\mathbf{X}}_i=1,\sum _{i=1}^n{\mathbf{Y}}_i=1$$.

Then, let $\lambda \in [0,1].$ Now,

$$\sum _{i=1}^n\lambda {\mathbf{X}}_i=\lambda ,\sum _{i=1}^n(1-\lambda ){\mathbf{Y}}_i=1-\lambda ,$$

Since either $\lambda$ or $1-\lambda$ are non-negative, all entries of this convex combination are non-negative, so the combination is again in $E$ and is therefore a probability vector.

We can use Heine-Borel to show that $E$ is compact by showing it is closed and bounded.

Noe that for each $\mathbf{p}\in E,$ and each @coordinate $i,$

$$\{\mathbf{p}\in {\mathbb{R}}^n:{\mathbf{p}}_i\ge 0\}$$

is a closed half-space. Now, note that the map

$$f:{\mathbb{R}}^n\to \mathbb{R},f(\mathbf{p})=\sum _{i=1}^n{\mathbf{p}}_i$$

is continuous, the pre-image of $1$ under $f$ is

$$\{\mathbf{p}\in {\mathbb{R}}^n:\sum _{i=1}^n{\mathbf{p}}_i=1.\}=f^{-1}(\left\{1\right\}).$$

Because $1$ is closed in $\mathbb{R},$ by A mapping $f$ of a metric space... we have that

$$\{\mathbf{p}\in {\mathbb{R}}^n:\sum _{i=1}^n{\mathbf{p}}_i=1.\}$$

is closed

Now,

$$E=\left(\bigcap _{i=1}^n\{\mathbf{p}:{\mathbf{p}}_i\ge 0\}\right)\cap \{\mathbf{p}:\sum _{i=1}^n{\mathbf{p}}_i=1\}$$

is an intersection of closed sets, which is also closed.

$E$ is obviously bounded - the sum of all coordinates in $\mathbf{p}\in E$ is always $1,$ so $E\subset$ the closed ball of radius $1$ centered at the origin.

Therefore, $E$ is closed and bounded, and is compact.

TODO: we can generalize this to the fact that @simplexes are convex and compact.

$◻$