A state of the art performance overview of current high end GPUs used for Deep Learning. All tests are performed with the latest Tensorflow version 1.15 and optimized settings. The results can differ from older benchmarks as latest Tensorflow versions have some new optimizations and show new trends to achieve best training performance and turn around times. Also the performance for multi GPU setups is evaluated.