Time Evolution of Biochemical Materials: Markov chains and Markov-States Models
Orchidea Maria Lecian
Sapienza University of Rome, Rome, Italy
*Corresponding author
Orchidea Maria Lecian, Sapienza University of Rome, Rome, Italy.
ABSTRACT
The biochemical materials are described in terms of the opportune Hierarchical Markov-State Model and of the originating chain(s).
The time evolution of the equations of motion of the Markov chain is controlled; to this aim, the transitions form the unrestrained simulations and those between the local Markov-State Models are compared.
The formalisms of quantum-mechanical systems are applied in the opportune measure spaces.
The ergodicity of the Markov chains is controlled.
The numerical simulations, the properties to be requested on numerical approximations are studied.
As a result, the ergodicity of the Markov chains provides with the possibility to impose the Sinai Markov partitions, which enable the von Neumann conditions on the Bloch equations.
Keywords: Finite Markov Chains, Ergodic Markov Chains, Biochemical Materials, Measure Theory, Numerical Approximations
Introduction
The Markov chains from which the MSM’s of biological macromolecules in molecular processes originate are investigated. The Markov Models are presented as the MSM and the Hierarchical MSM.
The quantum-mechanical properties of the Markov chains are described on Banach spaces; the specifications arising from the stochastic approach allow one to improve to a Hilbert space. The von Neumann conditions are requested after the density operator.
The request of ergodic chains allows one to adopt the proper Sinai partition. The hypotheses to be imposed to prove the ergodicity of the perturbed chain that must be realised are envisaged. Long-time-scale EOM’s of biological macromolecules is investigated: for this the propagation of the Markov chain is studied.
The Hierarchical MSM of molecular processes is set: the time evolution of the transition probabilities are investigated, the implied time-scales and number of the macro-states are scrutinised, the decay properties are enquired, the corre- sponding construction of the Markov-State-Models is indicated.
The Markovian Time evolution of quantum-mechanical systems is studied: to this purposes, several paradigms are studied. The open quantum-mechanical systems and control-of-interactions methods are explored. In particular, stochastic systems are used: the control of time evolution of transition probabilities between different states in the local MSM, obtained from classical ’unrestrained’ simulations, and that of the transition probabilities between states in differ- ent MSM’s are identified. More in detail, the control of the derivatives of the stationary distributions in perturbation theory from (finite) Markov chains is implemented.
The request of the ergodicity of the Markov chains is requested for the Sinai Anosov partitions to apply, for the von Neumann conditions to be imposed on the corresponding Bloch equations.
The study of the measure of the chains implied in the biochemical processes is performed in [1]. The paper is organised as follows.
In Section 2, the longtime scale of the equations of motion of the systems corresponding to biological macromolecules is established.
In Section 3, the Markov-State Model corresponding to biological macromolecules systems is constructed.
In Section 4, the Markovian time evolution of open quantum-mechanical systems is set.
In Section 5, the perturbation theory of finite Markov chains is presented and used.
In Section 6, the request of ergodicity of the Markov chain is enquired. In Section 7, the numerical simulations are explored.
Perpsective studies are envisaged in Section 8.
Long-Time-Scale EOM’s of Biological Macro- Molecules
The molecular processes from Hierarchical MSM are described from X(t) the ‘probability’ of the n - th state to be occupied at eh time t, with the (constant) transition-rate matrix, whose entries are kij the transition from i to j.
The memory-less EOM’s are given as
after the time steps τ.
The transition-probability matrix is defined, whose entries are specified as pij the transition from i to j.
Propagation of the Markov Chain
The propagation of the Markov chain is described as
The Hierarchical MSM of molecular processes is defined after from Eq.’s (2) as one with pi(t) the probability of the system to occupy the state i at the time t, wij the transition rate from i to j determined from the unrestrained simulations in the local MSM, kij the transition rate form i to j such that each state is in a different local Markov State.
Let n* be the MSM containing the state i; the partition Sn is defined as the set of Markov states composing the Markov model. This way, pi(t) are the evolution of probabilities, which evolve in time as after the equations of motion
where wp represent the transitions from the unrestrained simulations, and kp describe transition between local MSM’s. The implied time-scales and number of the macro-states can therefore be evaluated.
The eigen-vectors decomposition of the transition-probability matrix are the Xi states-distribution from which the transition between groups of states hap- pens: the transition-probability matrix TPM (τ ) is defined as
for which the eigenvectors-decomposition of the transition-probability matrix are defined.
The eigenvalue of a mode is defined from the decay of the occupancy of a mode.
As an example, the i - th mode at t = 0 is considered within the transition towards Ni(0) = 1, with t = τ and Ni(τ ) = µi.
The decay properties can be studied as well.
The time dependence of each mode is shaped as an exponential-decay one, i.e. one with the decay constant. The implied time scale is τi, i.e. the ’life-time’ of the transition mode used to construct the time-scale of the dynamics of the system in order to identify the modes.
Construction of the MSM
The objects Ni(t) and Ni(τ ) are defined from [5] as
where τi used to determine the Markovian time-scales of the system. More in detail, all the micro-states are requested to be Markovian, and all the implied time-scales are imposed to be constant, independently of the ’lag’ time. These conditions are analytically expressed as
Differently, the ’wished’ Markovian time-scales should have a raise and then flatten. As an example, it is useful to remark that, at small ’lag’ times, a MSM is required to have more macrostates in order to make sure that each micro-state is memory- less; differently, a shorter ’lag’ time is used to describe higher-resolution MSM’s, for which more energy minima are described. A a lower-resolution MSM, only a few macro-states are separated after high-energy barriers
Markovian Time-Evolution of Open Quantum- Mechanical System
The Markovian time-evolution of open quantum-mechanical systems is analysed within the weak-coupling limit as a general finite-dimensional system weakly-coupled with a finite-heat bath for which the relaxation to the Gibbs state is wished as follows [8].
Let B be a Banach space. Be 0 the projector to the Banach space
this way, B0 is defined as
B0 ≡ B (8)
and is designed to describe the system, while B1 is defined as
B1 ≡ 1 (9)
and is intended to describe a heat bath.
The free evolution after a strong continuous, one-parameter group Ut of isometries on B is studied in [12].
The isometries leave B0 and B1 invariant: let be an infinitesimal generator, closed, densely-defined as
i ≡ i . (10)
Be the perturbation: is bounded on B, i.e. after that
Aij ≡ PiAPj (11)
Under the hypothesis
A00 = P0AP0 ≡ 0, (12)
the following definition is given:
Definition: Uλt is one-parameter group generated after the infinitesimal generator defined after the parameter λ
Utλ ≡ Z + λA11 Ɐt : (13)
the property
[Utλ , 0] = 0 (14)
is implied.
Definition: The operator is defined as
Vtλ = Z + λA, (15)
and is decomposed as
The integral in Eq. (16) is bounded everywhere and strongly continuous. The properties of the operator are investigated in [7].
The operator is defined as
where the followinf representations hold
The system investigated lives on B0: from now on, no more further reference to the projector P0 is needed.
Definition: As
A11 ≠ 0, (19)
then Ar is expressedas
Ar = U-trAUtr. (20)
Definition: Given ϕ B0, it is defined as
ϕt ≡ Wtλ ϕ, (21)
which is further expressed as
The EOM’s are formally obtained as
The element Wtλ is implied to contain memory terms: the memory terms are small (in λ2) wrt the free term. It is therefore nencessary to study the behaviour of the limit λ → 0.
The following defifnition is needed.
Definition: the operator is written as
Ytλ ≡ U-tWλt (24)
the limit t → 0 must be discussed [8].
The behaviour λ → 0 is discussed in comparison with the stochastic approach.
Stochastic-Differential-Equations Approach
Stochastic differential equations on B0 can be studied.
As an example, be the triple (Ω, B0, dω) the probability space. Let B be the space B = L1(Ω, B0, dω).
Definition: the projector P0 is defined s.t.
with
Ut(f)(ω) = f(tω). (26)
This way, after tω ΩⱯt and ω Ω, the interaction term is given as A(ω): A(ω) is a ’random’ operator-valued function.
The following request is to be investigated:
|| P1 ||= 1 (27)
The irreverisibility of the process due to the initial conditions starting from an evolution equation on B is finishing with a semigroup on B0 [6].
Free Heat Bath
A system is demonstrated in Markovian equilibrium as the equilibrium is achieved as as its Gibbs state [8].
B0 is (also) the space of some trace class operators P0 : B → B0 in the partial trace.
Let ρ be an arbitrary trace class operator whose free evolution is given after
the one-parameter group of isometries on B with (formally-defined) infinitesimal generator.
The perturbation A is introduced.
The equilibrium state of the Markov processes is given as condition on the temperature.
In the weak-coupling limit, the exponential-decay law is obtained [13]. The statistical approach is recovered at B a Banach space: now A = A(t) is a strongly-continuous bounded operator-valued function on B which defines
of the Banach-space evolution equation
f′(t) = λA(t)f(t) (29)
The Method of Stochastic Differential Equations
The method of stochastic differential equations is discussed in [12]. Let B0 Banach space, and let (Ω, F, dω) be the triple of the pertinent probability space. The following items of information are gathered. Let B be a Banach space s.t.
B = L∞(Ω, F, dω) (30)
B is the space of the essentially-bounded strongly F-measurable B0 -valued functions on Ω.
Furthermore, B0 identified as the constant functions on Ω.
An Example of Dissipative Operator
As an alternative example, Z a dissipative operator is conisdered in [11], for which the evolution on a Banach space is controlled after a Markov process.
An Example in a Hilbert Space
From a different perspective let B be a Hilbert space, and let eZt be a unitary group on B: the self-adjoint-ness of iZ is implied [12].
Furthermore, let P be the orthogonal projector onto the null space of Z:
if iA is a self-adjoint operator, symmetric operators are obtained for the description of the Markov process.
Perturbative-Approach Theory of (Finite) Markov Chains
The perturbation formalism of finite Markov chains [14] holds in the present case.
After a Markov chain containing a single irreducible set of states, the derivatives of the stationary distributions are defined, and those the fundamental matrix wrt transition probabilities hold.
Be α an N-state stationary Markov chain, endowed with TPM: the time averaged TPM always exists, and the fundamental matrix always exists.
The hypothesis is taken, that α contains only one subchain (i.e. only one irreducible set of states).
The following Theorem holds:
Theorem: the solution of the equation of the stationary distribution always exists.
The following corollary holds:
Corollary: so do the qualities of the system β close to α.
Ergodic Markov Chains
The hypothesis is taken, that the unperturbed system geometrically be ergodic (Foster-Ljapunov drift conditions); the perturbation is taken to be uniform in the weak sense on bounded time intervals: these hypotheses opens the way to the construction of the Markov states, i.e. such that the comparison with holds [16].
Numerical Simulations
Numerical simulation can be explored in the cases of randomly-impulsed ODE’s, of I’to SDE’s, and of stochastic parabolic PDE’s (where the white noise can be approximated as Gaussian noise).
In the case of stochastic case PDE’s, in the geometrically-ergodic case, long-time weak convergence is proven: the perturbation theory is arising from numerical approximation [15].
Perspective Studies
The exponential-decay law is recovered from initial conditions. The density operator of an open quantum system can be defined after the inverse of the dynamical map which governs the evolution of the density operator. The quantum dissipation is studied from the von Neumann conditions in the Bloch equation: the first Born approximation on a Hilbert space is obtained after the short-time approximation of the EOM’s and after the short-time approximation of the EOM’s obey the von Neumann conditions [17].
The study the hypotheses to be imposed to prove the ergodicity of the perturbed chain are taken after [15].
After the ergodicity is proven, and the Siani Anosov partitions are allowed to be applied, the implication on the Markov approximations due to the correlations decay of Anosov flows is needed to be controlled [18].
The ergodicity of the Markov chains therefore allows one to impose the von Neumann conditions on the Bloch equations.
References
- Zeng J, Duan J, Wu C. A new distance measure for hidden Markov models, Expert Systems with Applications. 2010. 37: 1550-1555.
- Zhdanok AI. Finitely additive measures in the ergodic theory of Markov chains I. Sib Adv Math. 2003. 13: 87-125.
- Zhdanok AI. Finitely additive measures in the ergodic theory of Markov chains II. Sib Adv Math. 2003. 13: 108-125.
- Wolfe DK, Persichetti JR, Sharma AK, Hudson PS, Woodcock HL, et al. Hierarchical Markov State Model Building to Describe Molecular Processes. J Chem Theory Comput. 2020. 16: 1816-1826.
- Da L-T, Sheong FK, Silva D-A, Huang X. Application of Markov State Models to Simulate Long Timescale Dynamics of Biological Macro- molecules, in: Han Kl, Zhang X, Yang Mj. (eds) Protein Conformational Dynamics. Advances in Experimental Medicine and Biology. Springer. 2014. Cham. 80.
- Papanicolaou GC, Varadhan SRS. Comm Pure Appl Math. 1973. 26: 497-524.
- Kato T, Perturbation theory for linear operators, Springer: Berlin- Heidelberg-New York. 1966.
- Davies EB. Commun math Phys. 1973. 33: 171-186.
- Pule’ JV. The Bloch equations, Communications in Mathematical Physics. 1974. 38: 241-256.
- Balslev E, Verbeure A. Commun math Phys. 1968. 7: 55-76.
- Kurtz TG. A limit theorem for perturbed operator semigroups with ap- plications to random evolutions. J Functional Analt. 1973. 12: 55-67.
- Davies EB. Markovian master equations III, Annales de l’Institut Henri Poincar´e, Section B, Calcul des probabilit´es et statistiques 1975. 11: 265-273.
- Davies EB. Markovian Master Equations II. Math Ann. 1976. 219: 147-158.
- Schweitzer PJ. Perturbation Theory and Finite Markov Chains, Journal of Applied Probability. 1968. 5: 401-413.
- Shardlow T, Stuart AM. A Perturbation Theory for Ergodic Markov Chains and Application to Numerical Approximations, SIAM Journal on Numerical Analysis. 2000. 37: 1120-1137.
- Ya G. Sinai, Construction of Markov partitions, Functional Analysis and Its Applications. 1986. 2; 245-253.
- Wonderen AJ, Lendi K. Quantum Theory of Dissipative Processes: The Markov Approximation Revisited, Journal of Statistical Physics. 1995. 80: 273-304.
- Chernov NI. Markov Approximations and Decay of Correlations for Anosov Flows, Annals of Mathematics Second Series. 1998. 147: 269-324.
