Convolution neural network(CNN) is the de-facto standard for medical imaging tasks. However, CNNs are known to be computation expensive. To take advantage of GPU's computing power, techniques such as Image Block to Column(im2col) are adopted at the cost of increasing memory usages. Such reasons prevent convolution-based architectures from training with large batch sizes and applying to high-definition and high-resolution input. In this work, we propose PISTACHIO: patch importance sampling to accelerate CNNs via a hash index optimizer. With efficient hashing-based sampling, we reduce the memory consumption of CNNs while preserving the final training accuracy.