The Unifying View
on Ordinary Differential Equations and Automatic Differentiation

The basic terminology. The seed of the theory discussed here originates in one chapter of the book by Ramon Moore "Interval Analysis" (1966) where he linked the optimized n-order differentiation of general expressions with the elementary functions. Though he didn't offer his own term, in most straightforward way his idea must be called the Optimized Taylor Method. Later the practitioners attached to this method the name Automatic Differentiation (AD) which relies on the formulas of the Optimized n-order differentiation of general expressions by Moore (using them for automation in converting the code computing the functions into a code computing the n-order derivatives of those functions). In the frame of the "Unifying View on ODEs and AD" we use intermittently either the terms Optimized Taylor method or AD (yet in the sense of the optimized formulas only). We strongly suggest to stay with these two terms not adding any alternatives like "Power Series Method" or  "Differential Transform Method", and other terminology emerging because of unawareness about the original idea about the Optimized Taylor Method by R. Moore now framed in the "Unifying View on ODEs and AD".

Here we discuss and deal with ODEs which are represented by holomorphic functions in complex plane and whose solutions are therefore holomorphic function in a particular domain. That means that the right hand sides of the ODEs are expendable into a multi-variate Taylor series with non-zero convergence radii for each variable at every point of the domain. The derivatives therefore are understood as complex derivatives (even if the integration occurs along the real axis only).

The main formulas for n-order derivatives including what was later named the Faa di-Bruno formula were introduced by Antoine Arbogast as early as in 1800 (see the Reference compiled by the late Harley Flanders). Brook Taylor suggested and studied the expansions of (holomorphic) functions into power series (the Taylor expansions) even earlier.

During those two centuries that the idea of the Taylor expansion and the classical Taylor method for integration of ODEs were around, the possibility of computing n-order derivatives of functions and expressions over them was rather taken for granted. It appears however that the algorithms of n-order differentiation are practically applicable only to a particular sub-class of holomorphic functions derived from the so called general elementary vector-functions. The Taylor method exists in a frame of the concept called the Unifying view on Ordinary Differential Equations and Automatic Differentiation outlined in the Preface of the article with the same title [2].

The central concept of the Unifying view is the definition of the general elementary vector-functions (widening the class of conventional elementary functions as defined by Liouville). Since Ramon Moore in the 1960s, the general elementary functions were defined as vector-solutions of systems of m excplicit rational ODEs of order 1. It is also possible to introduce a competing definition for a stand alone or scalar elementary function as a solution of one n-order rational ODE. However the issue of equivalency of both definitions remains open depending on the not yet resolved Conjecture.

First this Conjecture was published in 2008 [1], then in 2009 [2]. In the most comprehensive form and with the necessary context it is posted in the Power point presentation (and pdf) [3], while the report about everything known about it in the moment (August 2019) is here.

Integration of holomorphic ODEs in fact is a process of analytical continuation which allows to reach any point of the domain at which the right hand sides of the ODEs are regular (holomorphic). But what if the ODEs happen to be singular at a particular point of the phase space?

Even if the given ODEs are singular at a particular point, their solution may exist and be holomorphic at this very point [1] such as the ODE
x' = 2x/t,   x|t=0 = 0
whose solution is x(t)=t2 satisfying also a regular ODE  x' = 2t. 

Is such a replacement of a singular rational ODE with a regular one always possible? As it was shown in [1], there exist such special points in holomorphic functions at which the function can satisfy no regular rational ODE of any order at these points. Though such a special point in a function is a point of holomorphy, its specialty is in that it cannot be removed by choosing some regular rational ODE satisfied by this function. This specialty is unremovable and intrinsic for the function, such as the point t=0 for x(t)=sin(t)/t. However in order to classify such a point as a point where elementariness of the function x(t)  is violated, we need to prove the Conjecture.

Though the Conjecture looks like a small gap in the Table on slide 4 (and pdf) , its role in this theory is absolutely crucial. Everybody are welcomed to brainstorm this challenge.
  1. Unremovable "removable" singularities. Complex Variables and Elliptic Equations, Vol. 53, No. 7, July 2008, pp. 633-642.
  2. The ordinary differential equations and automatic differentiation unified. Complex Variables and Elliptic Equations, Vol. 54, No. 9, September 2009, pp. 825-854.
  3. The Unifying View on ODEs and AD yet with a gap to fill  (and pdf) 2012  - the most brief presentation of the gap (slide 4), the Conjecture (slides 6-8) and its context.