




Publications  Parity Objectives in Countable MDPs





Reference:
Stefan Kiefer, Richard Mayr, Mahsa Shirmohammadi, and Dominik Wojtczak. Parity objectives in countable MDPs. In Proceedings of the 32nd Annual Symposium on Logic in Computer Science (LICS). IEEE, 2017.
Abstract:
We study countably infinite MDPs with parity objectives, and special cases with a bounded number of colors in the Mostowski hierarchy (including reachability, safety, Büchi and coBüchi). In finite MDPs there always exist optimal memoryless deterministic (MD) strategies for parity objectives, but this does not generally hold for countably infinite MDPs. In particular, optimal strategies need not exist. For countable infinite MDPs, we provide a complete picture of the memory requirements of optimal (resp., epsilonoptimal) strategies for all objectives in the Mostowski hierarchy. In particular, there is a strong dichotomy between two different types of objectives. For the first type, optimal strategies, if they exist, can be chosen MD, while for the second type optimal strategies require infinite memory. (That is, for all objectives in the Mostowski hierarchy, if finitememory randomized strategies suffice then also MDstrategies suffice.) Similarly, some objectives admit epsilonoptimal MDstrategies, while for others epsilonoptimal strategies require infinite memory. Such a dichotomy also holds for the subclass of countably infinite MDPs that are finitely branching, though more objectives admit MDstrategies here.
Suggested BibTeX entry:
@inproceedings{17KMSWLICSMDPs,
author = {Stefan Kiefer and Richard Mayr and Mahsa Shirmohammadi and Dominik Wojtczak},
booktitle = {Proceedings of the 32nd Annual Symposium on Logic in Computer Science (LICS)},
publisher = {IEEE},
title = {Parity Objectives in Countable {MDPs}},
year = {2017}
}




