Military health research often involves large and disparate data types. The execution of large longitudinal studies and the availability of multiple TBI research repositories globally have afforded opportunities for exploratory analysis, hypothesis generation, validation of existing results, and predictive modeling of recovery or risk factor outcomes. Machine learning (ML) tools have been widely used in military health research, and over the past decade, the number of publications in the domain of military health with ML have increased exponentially. A critical issue, however, is: (1) poor repeatability, (2) small or unclear effect size and (3) lack of generalizability for application of ML toward meaningful clinical translation. At the fundamental level, questions on appropriateness of ML tools, the difference between ML and classical statistical methods, and the strengths and limitations of each method need to be clearly understood. Machine Learning (ML) tools focus on maximizing prediction accuracy, through forecasting unobserved outcomes or future results. This is a data driven approach that does not always require understanding of the underlying mechanisms and the results may not have biological correlates. Classical statistical methods are associated with (1) developing models that describe relationships among variables, (2) making inferences with respect to these relationships; and (3) addressing a specific hypothesis or observation about a system. Validation, significance and generalizability for each method is different, as are the resource consumption and application or translation of the respective models developed. The objective of this work is to present a clear understanding of the strengths and limitations of ML and classical statistics as it pertains to military health research. In addition, we present a two-part framework for research analytical decisions: (part 1) Technical, accessibility and scientific considerations for the selection and use of appropriate analytical tools for the specific research question at hand and (part 2) how to evaluate efficacy of published ML and classical statistical tools in research literature. The significance of this framework is to assist researchers and clinicians with the selection of appropriate, and effective tools and second, assist investigators with understanding strengths and limitations in the growing published literature using ML.