Introduction
7 months ago Elegia published a blog about D-finite functions (part1, part2) in which he showed some examples of problems that can be solved by D-finite functions.
Here I will explain a fast evaluation algorithm of the coefficients of a D-finite function.
Let $$$K$$$ be some field of characteristic $$$0$$$. Let $$$K[ [x_{1},\ldots,x_{d}] ]$$$ be the ring of formal power series on it. Let $$$K[x_{1},\ldots,x_{d}]$$$ be the polynomials of it. Let $$$K(x_{1},\ldots,x_{d})=\mathop{\mathrm{Frac}}(K[x_{1},\ldots,x_{d}])$$$ be the field of rational functions over it. Let $$$ [ x_{1}^{i_{1}}\ldots x_{d}^{i_{d}} ] f$$$ be the coefficient in $$$f$$$ of the monomial $$$x_{1}^{i_{1}}\ldots x_{d}^{i_{d}}$$$.
Definitions
A function $$$f \in K[ [x_{1},\ldots,x_{d}] ]$$$ is called D-finite if for each $$$i$$$ there exists a positive integer $$$n_{i}$$$ and polynomials (not all zeroes) $$$P_{i,0}(x_{1}, \ldots,x_{d}),\ldots,P_{i,n_{i}}(x_{1},\ldots,x_{d})$$$ such that $$$P_{i,0}f+P_{i,1}\frac{df}{dx_{i}}+\ldots+P_{i,n_{i}}\frac{d^{n_{i}}(f)}{(dx_{i})^{n_{i}}}=0$$$.
An infinite table of coefficients $$$a(i_{1},\ldots,i_{d})$$$ is called P-recursive if there exists $$$k \in \mathbb{N}$$$ such that for each $$$j \in \{ 1,\ldots,d \} $$$ there exists a mapping $$$p: \{ 0,\ldots,k \} ^{d} \rightarrow K[x]$$$ such that at least one $$$p_{v} \neq 0$$$ and for all $$$i_{1},\ldots,i_{d} \geq k$$$ $$$\sum\limits_{v \in \{0,\ldots,k\}^{d}}^{} p_{v}(i_{j}) a(i_{1}-v_{1},\ldots,i_{d}-v_{d}) = 0$$$.
In particular, for $$$d=1$$$ it means that there exist such polynomials (not all zeroes) $$$P_{i}$$$: $$$ \forall n \geq k : \sum\limits_{0 \leq i \leq k}^{} P_{i}(n)a(n-i)=0$$$.
Properties:
1 The definition is equivalent to the "linear space of partial derivatives of $$$f$$$ over $$$K(x_{1},\ldots,x_{d})$$$ has finite dimension".
In particular, every rational function is obviously D-finite
2 They form a linear algebra over $$$K(x_{1},\ldots,x_{d})$$$ (we can multiply them by a scalar, add them and multiply them, and the result will be D-finite)
3 Every algebraic over $$$K(x_{1},\ldots,x_{d})$$$ function is D-finite
4 If $$$u_{1},\ldots,u_{d} \in K[ [u_{1},\ldots,u_{d}] ]$$$ are algebraic, and $$$f$$$ is D-finite and composition $$$f(u_{1},\ldots,u_{d})$$$ is well-defined in some sense, then it is D-finite.
5(Lipshitz, 1988) If $$$f$$$ is D-finite, then $$$\sum\limits_{i_{12},i_{3},\ldots,i_{d}}^{} x_{1}^{i_{12}}x_{3}^{i_{3}}\ldots x_{d}^{i_{d}} ([x_{1}^{i_{12}}x_{2}^{i_{12}}x_{3}^{i_{3}}\ldots x_{d}^{i_{d}}] f)$$$ is D-finite. That is, you can contract the function $$$f$$$ to the diagonal coefficients. (You can do a series of contractions to select only coefficients on the diagonal)
6(Lipshitz, 1989) The function $$$f=\sum\limits a(i_{1},\ldots,i_{d}) x_{1}^{i_{1}}\ldots x_{d}^{i_{d}}$$$ is D-finite if and only if its coefficients are P-recursive.
7(Lipshitz, 1988) If $$$f$$$ is D-finite, then for each $$$i_{1}$$$: $$$\sum\limits_{i_{2},i_{3},\ldots,i_{d}}^{} x_{2}^{i_{2}}x_{3}^{i_{3}}\ldots x_{d}^{i_{d}} ([x_{1}^{i_{1}}x_{2}^{i_{2}}x_{3}^{i_{3}}\ldots x_{d}^{i_{d}}] f)$$$ is D-finite. Moreover, the number of polynomials and their degrees in the D-finiteness relations are uniformly bounded.
Note: D-finite functions are not a field
Let's prove 7
Note: Here in the proof we actually have shown that the non-negative (by degree of $$$x_{1}$$$) part of a D-finite function is D-finite.
What functions are D-finite?
Can you guess which of these functions are D-finite (that is, the sequence/table of coefficients is P-recursive)?
(1) $$$a_{n} = [ x^{n+1} ] (1+x+x^{2})^{n}$$$ ($$$f(x)=\sum\limits_{}^{} a_{n}x^{n}$$$)
(2) $$$a_{n}=n!$$$ ($$$f(x)=\sum\limits_{}^{} n!x^{n}$$$)
(3) $$$a_{n,k}=S(n,k)$$$ Stirling numbers of the second kind, number of ways to partition a set of $$$n$$$ objects into $$$k$$$ non-empty subsets.
$$$\sum\limits_{n=0}^{\infty} \sum\limits_{k=0}^{n} S(n,k) \frac{x^n}{n!} y^k = e^{y(e^x-1)}$$$
(It is not so hard to see, that the D-finiteness of the OGF of some sequence/table is equivalent to the D-finiteness of EGF of this sequence/table)
(4) Eulerian numbers $$$a_{n,k}=E(n,k)$$$ is a number of $$$n$$$-element permutations with $$$k$$$ ascents.
$$$\sum\limits_{n=0}^{\infty} \sum\limits_{k=0}^{n} E(n,k)t^{k} \frac{x^n}{n!} = \frac{t-1}{t - e^{(t-1)\,x}} = \left(1-\frac{e^{(t-1)x}-1}{t-1}\right)^{-1}$$$
(5) $$$a_{u,v,w}=\sum\limits_{0 \leq x \leq u, 0\leq y \leq v, 0\leq z \leq w} \frac{(x+y)!(y+z)!(z+x)!}{x!y!z!}$$$.
The D-finite functions with $$$d=1$$$ are just P-recursive sequences. And we can evaluate the first $$$n$$$ terms of a P-recursive sequence in a stupid way in $$$O(n)$$$, or there is an algorithm to evaluate the $$$n$$$-th term (only one) in $$$O(\sqrt{n}\log(n))$$$.
But how to solve problems using this?
Let us assume that some function $$$f$$$ is D-finite, for example $$$f=\frac{P(x,y)}{Q(x,y)}$$$ where $$$P(x,y)$$$ and $$$Q(x,y)$$$ are fixed polynomials. And we want to compute $$$[x^{n}y^{m}] f$$$ (number of variables actually does not matter).
Let's notice that $$$h(u)=[x^{n}y^{u}] f$$$ is D-finite (this is the property 7) that is equivalent to P-recursive, and we can compute $$$h(m)$$$ in $$$O(\sqrt{m}\log(m))$$$ from some first values of $$$h$$$: $$$h(0),\ldots,h(k)$$$ where $$$k$$$ is a constant.
To compute $$$h(l)$$$ we will do the same, we will notice that $$$g(u)=[x^{u}y^{l}] f$$$ is P-recursive, so we can compute $$$g(l)$$$ in $$$O(\sqrt{n}\log(n))$$$.
So we can compute $$$[x^{n}y^{m}] f$$$ in $$$O(\sqrt{n+m}\log(n+m))$$$.
But when I say "Let's compute the coefficient of this P-recursive sequence", how do I know that the P-recursive relation holds?
One can compute it from the proof of property $$$\textbf{7}$$$, but I think this is a lot of pain. So to find P-recursive relation let's compute the first $$$k$$$ coefficients (for some $$$k$$$) and run a Gauss.
Note: We can also notice from the proof of property $$$\textbf{7}$$$, that the coefficients of P-recursion of $$$h(u)$$$ can be expressed as polynomials of $$$u$$$, and then we can try to evaluate them from some interpolation of a rational function, though I have not implemented it yet.
Example of this algorithm:
Assume we have a function $$$f(n,m)=[x^ny^m] e^{\frac{x+y}{(1-x)(1-y)}}$$$. And we want to compute $$$f(10^{7},10^{8}) \pmod{998\,244\,353}$$$.
This function $$$f$$$ is D-finite.
Let's compute its values for $$$0 \leq n \lt 25$$$ and $$$0 \leq m \lt 25$$$. Then we can extend
$$$f(\cdot,0),f(\cdot,1),f(\cdot,2),\ldots,f(\cdot,24)$$$ to $$$f(10^{7},0),f(10^{7},1),\ldots,f(10^{7},24)$$$ by P-recursion. Then we can extend $$$f(10^{7},\cdot)$$$ from the first $$$25$$$ values to $$$f(10^{7},10^{8})$$$ by P-recursion too. It works in $$$O(25\sqrt{10^{7}}\log(10^{7})+\sqrt{10^{8}}\log(10^{8}))$$$ which takes ~3.1 sec.
Note: for $$$f(10^{8},10^{8})$$$ we can compute it from diagonal (property 5).
Implementation:
Here is a code
Code works for all prime moduli. Explanation:
findPrecursionfinds P-recursionevaluatePrecursionevaluates P-recursion from the initial values if possibleevaluatePrecursionfastevaluates the element of a sequence from P-recursion and the initial valuessequenceextenderis trying to extend the sequenceoptimalgetvaluebyidis trying to get the element of an extended sequence by its positionextendtableextends table A, finding A(que[i].first,que[i].second)getcolumnoftableandgetrowoftableget column or row of an extended table
A "small" problem
But there is one caveat in this algorithm. Assume that there is a sequence $$$a_{i}$$$ with P-recursion with relation $$$a_{n}P_{0}(n)+\ldots+a_{n-l}P_{l}(n)=0$$$.
You know $$$a(0),\ldots,a(l-1)$$$ and you want to extend this sequence to $$$a(0),\ldots,a(N)$$$. But what if $$$P_{0}(k)=0$$$ for some $$$l \leq k \leq N$$$? Then we can't extend this sequence. And I don't know what to do with that, but luckily, I suppose that for almost all functions when we can use it, it can't be zero.
Example where it matters
Let's try to compute $$$f(10^{6},10^{6})$$$ with $$$f(n,m)=[x^{n}y^{m}] \frac{1}{1-xy}$$$. We will try to calculate some first columns, and then a row. But even if we managed to somehow guess the first columns, we would be unable to get a row, because our initial values in this row will all be zeroes, and we want to get $$$1$$$ somehow, but we can't.
Examples of the problems
1) Atcoder Regular 202 problem D
2) We can also solve 1747E - List Generation in $$$O(\sqrt{n}\log(n))$$$ per test (compute the coefficient of $$$(1-x)^{2}(1-y)^{2}(1-2x-2y+2xy)^{-2}$$$), but actually, because there is a multitest and the constraints are big this solution is hard to pass (I have managed, but after a lot of pain and with some strange FFT: 330486092).
I think there are some other examples of problems that can be solved by this method.
Conclusion (Warning)
When I got it at first, it seemed to me that "every" function in combinatorics is D-finite, but it is actually wrong (see before). I think that one should use this algorithm only if one has the proof that some function is D-finite, otherwise it can be a waste of time. So I don't recommend using this algorithm as magic.
Thanks to orz for editing this blog!







