Résumé : Coarse-graining is a standard method of extracting a simple Markov process from a more complicated one by identifying states. Here we extend coarse-graining to open Markov processes. An "open" Markov process is one where probability can flow in or out of certain states called "inputs" and "outputs". One can build up an ordinary Markov process from smaller open pieces in two basic ways: composition, where we identify the outputs of one open Markov process with the inputs of another, and tensoring, where we set two open Markov processes side by side. In previous work, Fong, Pollard and the first author showed that these constructions make open Markov processes into the morphisms of a symmetric monoidal category. Here we go further by constructing a symmetric monoidal double category where the 2-morphisms are ways of coarse-graining open Markov processes. We also extend the already known "black-boxing" functor from the category of open Markov processes to our double category. Black-boxing sends any open Markov process to the linear relation between input and output data that holds in steady states, including nonequilibrium steady states where there is a nonzero flow of probability through the process. To extend black-boxing to a functor between double categories, we need to prove that black-boxing is compatible with coarse-graining.