This paper presents a novel approach to solve simultaneously the problems of human whole-body motion prediction and action recognition for real-time applications. Starting from the dynamics of human motion and motor system theory, the notion of mixture of experts from deep learning literature has been extended to solve this problem. The work is accompanied by 66-DoFs human model experiments.