Critical review of research.
Article #1
Digital game-based learning: Impact of instructions and feedback on motivation and learning effectiveness.
Don Maurer

1. Identify the clarity with which this article states a specific problem to be explored.
Digital game-based learning (DGBL) is the referenced medium to be discussed. The article discusses how DGBL is supposed to increase motivation and cognition. The authors do not describe any specific area to be studied but instead state that they want to determine whether there is value added through the difference in instructions provided. This description is very unclear. The authors meaning of value added is not stated and the information they are attempting to glean is not covered under a stated hypothesis.
2. Comment on the need for this study and its educational significance as it relates to this problem.
The proposed reason for the study, value added through instruction, is one that has been discussed more than a few times as is shown by the references cited within the article. There is no problem presented though. Yes, discovery of whether DGBL is a worthwhile effort is valid but this study does not address that. Educationally, the determination of whether DGBL is actually useful in providing motivation and deep cognitive learning is a relevant question. This study does not provide information to answer that question at all.
3. Comment on whether the problem “researchable”? That is, can it be investigated through the collection and analysis of data?
If the article provided a better hypothesis or research question it would be researchable. This article fails to give a sound reason for the research. That also affects the methodology of the study. With the right collection of data, a working theory should be able to be produced. Through the research in this study though there is no relevant information provided. The research does not discuss a problem but instead highlights DGBL and points at it from multiple directions without any really significant direction for the research.

Theoretical Perspective and Literature Review
4. Critique the author’s conceptual framework.
The authors’ framework is very unscientific, more just descriptive. They are effective at reporting what other researchers have found and what they have proposed but they do not tie in other research to what they are proposing to study themselves. There are no critical views of what they have found, just descriptions or prior research and opinions. They do not identify any prior problems with previous research or theory, just state what has already been reported. The authors do not seem to have a clear conceptual framework anywhere within this research.
There are very few contradictions made in the article. They seem to agree with all of the cited articles and only throw in a different view in a couple places within the article. This is not even to support the need for their proposed research but instead to point out that there were other things found, differences, but they don’t apply to what the research here is supposed to be looking at. There is little evidence of references being used as alternatives to the issue being researched. Instead it seems to be used as filler, to show that someone has looked at DGBL prior to this but the relevance to this study is just not defined.

5. How effectively does the author tie the study to relevant theory and prior research? Are all cited references relevant to the problem under investigation?
The author uses multiple references to other research that supposedly state the same ideas. They make statements about the theories that others have discussed but the relevance to what the authors are studying is not always relevant. There are connections to DGBL and they are quite good, well, some of them. Not all references are relevant to the problem under investigation. The problem is not even defined well enough to decide if the references really matter to the study at all.
6. Does the literature review conclude with a brief summary of the literature and its implications for the problem investigated?
No. The literature review discusses DGBL and various aspects of how it may be useful in education. Motivation, cognition, deep content, etc. are all discussed within the literature review. There is no direct connection to the experiments performed for most of the literature discussed. The authors seem to think that by referencing many different aspects of DGBL they will be able to make their research more significant. There is not a summary of the literature that is discernable. The authors’ attempt to compare the cited research to the implications for the new research is feeble. They just used the same words to point out that they were doing something similar.
7. Evaluate the clarity and appropriateness of the research questions or hypotheses.
The question proposed is hard to understand. There really isn’t a question proposed until the description of the first experiment. Then the authors discuss their idea to compare DGBL with conventional learning media. There is no actual comparison between the two. they only researched whether students acquire a deeper learning by changing the instructions provided. This does not seem to match anything else proposed in the paper. The proposed hypothesis that is finally presented in the paper is not what is researched at all.

Research Design and Analysis
8. Critique the appropriateness and adequacy of the study’s design in relation to the research questions or hypotheses.
Their study’s design does not relate to the research question at all. There is not a comparison made between the study’s design and the research question at all. There is not a well-stated hypothesis to consider and determining if the research applies to the actual question being studied is a vague guess at best. If there was an actual hypothesis that pertained to the study performed then a critique of the design would be possible. In this case, the design is simplistic and does not address the possible hypothesis that is hinted at in the beginning.
9. Critique the adequacy of the study’s sampling methods (e.g., choice of participants) and their implications for generalizability.
The study does not provide data that can be used for any other comparisons. The participants are all college-aged students, attending a university, without medical knowledge. This is a very restrictive type of study and cannot be used to reflect any other population. The participants are too narrow of a scope.
10. Critique the adequacy of the study’s procedures and materials (e.g., interventions, interview protocols, data collection procedures).
The study did not use a DGBL design. They used a multimedia designed course where students watched, listened, and then picked out an answer. The authors’ chosen format for DGBL renders the findings of the study to be immaterial. DGBL and multimedia can go hand in hand but in this study DGBL was not used. The procedures for the conduct of the study made this sound more like taking an ASVAB test than playing a game. The purpose of the research was not well defined and therefore the research itself does not provide relevant information.
11. Critique the appropriateness and quality (e.g., reliability, validity) of the measures used.
The measures used seemed inadequate. Comparing data using a mean and standard deviation did not provide the relevant information that should have been found. The reliability and validity of the measures are fine, but what does the data actually represent? In this study it is just a bunch of numbers that show that there really wasn’t any difference in the two groups or the two experiments.
12. Critique the adequacy of the study’s data analyses. For example: Have important statistical assumptions been met? Are the analyses appropriate for the study’s design? Are the analyses appropriate for the data collected?
This question is not required to be answered this time.
C. Interpretation and Implications of Results
13. Critique the author’s discussion of the methodological and/or conceptual limitations of the results.
The authors provided a detailed description of the results of the study. That description does not follow what they discussed in the experiment sections of the paper though. The study shows that there is little to no difference in the data but the authors state there is a definite call for further study based on the results of this research. There is some discussion of the limitations concerning learning goals or intrinsic motivation. But in the study itself intrinsic motivation is discussed. The authors also discussed other concepts that were not elaborated on within the experiment itself.
14. How consistent and comprehensive are the author’s conclusions with the reported results?
The reported results do not show much variation; they are the same for both control groups with very little difference between the two. The authors claim that the results show a variation in learning that is not supported by the results themselves. The results and the explanation seem to contradict each other. Analyzing the results and the conclusions made about further research and how the difference in the type of instruction makes a difference in the learning of the student would be easier to do if there was a clear hypothesis to start with.
15. How well did the author relate the results to the study’s theoretical base?
The theoretical base of the study is minimal if existent at all. There was never a defined theory to read and understand the research from. Distinguishing what the real purpose of the study proved to be problematic. To be able to relate results found to the theoretical base requires that there be a theoretical base that can be understood first. Without it the results are just numbers that really provide no meaning other than to show that the study was performed and something was found. What was found and the relevance to the missing hypothesis is something that remains to be discovered. The authors claim that research “opens up several new avenues of study”. The avenues of study are not defined in any way. This leaves the reader confused as to the authors intent.
16. In your view, what is the significance of the study, and what are its primary implications for theory, future research, and practice?
This study seems to try and point out that DGBL is something that can be considered useful in the classroom. The authors try to show that with different types of instructions it can improve cognitive learning. They do not provide any real data to support any statements that are made except that other factors may have affected the results of the study. This is the only thing that they can say without argument because other factors can always affect a study. They seem to have put a lot of time and effort into something that does nothing.
The significance of the study proves one thing without a doubt. DGBL is something that needs to be looked into further. Unfortunately, it is not the authors’ study that provides that insight but the research they reference.