The subjective assessment of facial paralysis relies on the expertise of clinicians; the main limitation is intra-observer and inter-observer reproducibility. In this paper, we proposed a deep learning approach combining point clouds of facial movements with expert consensus to objectively quantify the severity of facial paralysis. A dynamic 3D photogrammetry imaging system was used to capture the facial movements of five facial expressions. Point clouds of the face at rest and at maximum expressions were extracted. These were integrated with the experts grading of the severity of facial paralysis to train a PointNet network to quantify the severity of facial paralysis. The results showed an accuracy exceeding 95% for assessing facial paralysis.