%0 Journal Article
%T Emphatic TD Bellman Operator is a Contraction
%A Assaf Hallak
%A Aviv Tamar
%A Shie Mannor
%J Computer Science
%D 2015
%I arXiv
%X Recently, \citet{SuttonMW15} introduced the emphatic temporal differences (ETD) algorithm for off-policy evaluation in Markov decision processes. In this short note, we show that the projected fixed-point equation that underlies ETD involves a contraction operator, with a $\sqrt{\gamma}$-contraction modulus (where $\gamma$ is the discount factor). This allows us to provide error bounds on the approximation error of ETD. To our knowledge, these are the first error bounds for an off-policy evaluation algorithm under general target and behavior policies.
%U http://arxiv.org/abs/1508.03411v2