This article considers aspects of application of advanced parallel computing techniques to computational linguistics Big Data problems, which spring from creation Web-based corpora on base of digitization of huge collections of historic texts scans. Solving of the majority of linguistic research problems requires per word processing that is the natural task for massive parallel processing provided by unconventional, high performance and their hybrid computing. Testing our approach, we focus on application of P system computing to diachronic analysis of historic texts.