Solving the system of linear systems is of paramount importance in the field of science and technology. The applications of linear equations have been widely found in diverse fields. As an effect of the massive rise of big data, developing computational algorithms for solving systems of linear equations of large size has gained utmost importance in the field of data science. Since solving large systems of linear equations serially can be time consuming and slow process, parallelizing the algorithms with appropriate parallel constructs provides the accurate solutions with less time complexity. Parallel implementation of three algorithms namely back substitution, conjugate gradient and Gauss Seidel to solve large systems of linear equations using OpenMP is proposed in this paper. To determine the most time and space efficient method among the three, comparative analysis of both the serial and parallel execution of each algorithm are presented. The algorithms are further optimized to get the best results for the execution time. The algorithms for solving the system of linear equations generally involve multiple steps. So, by scheduling the number of threads for each process the execution time of the process could be optimized. While executing the programs parallelly different number of threads run simultaneously on different processors. These aspects have been considered to propose the implementation of the three algorithms. The limitations of particular algorithms are considered, their laws of convergence are discussed and comparative analysis is developed considering those kinds of matrices which can be solved accurately by all the three algorithms. A complete comparative study of all the three algorithms computed parallelly and serially with detailed analysis of the performance are presented.