Research and Assessment Methods

Survey has been a widely used data collection method for a variety of purposes in educational research. Although response styles have the potential to contaminate survey results, educational researchers often do little to control for such negative effects. Under discussion are five common response issues, their impact on survey data, and the methods that may be used to minimize the negative impact of these response issues on survey data. The five response issues in question are acquiescence (including disacquiescence), careless responding, extreme response, social desirability, and item-keying effect. Acquiescence (disacquiescence) refers to a respondent’s general tendency to agree (or disagree) with an item regardless of its content. This response style can distort item and construct correlations, compromising the results of factor analytic and correlational findings. Careless responding refers to a respondent’s tendency to pay insufficient attention to item content before responding, which can also lead to a biased estimation of relationships. Extreme response refers to the tendency of selecting extreme response options (e.g., strongly agree or strongly disagree) over middle options (e.g., neutral). Social desirability refers to a respondent’s tendency to rate him- or herself in an overly positive light. Finally, item-keying effect refers to a respondent’s differential responses to regular-keyed and reverse-keyed items. This effect often creates the illusion that items with opposite keying directions measure distinct constructs even when they may not.

A growing amount of research has been done on how to control for the negative impact of these response styles, although the research may be limited and uneven for different response issues. A variety of approaches and methods exist for handling these response issues in research practice. Different response issues may require considerations at different stages of research. For example, effective handling of acquiescence response may require steps in both survey construction (e.g., including a hidden measure of acquiescence) and survey data analytic treatment (partial correlation technique), while controlling for item-keying effect may require more sophisticated modeling techniques (e.g., multitrait-multimethod confirmatory factor analysis).

Authentic tasks replicate real-world challenges and standards of performance that experts or professionals typically face in the field. The term “authentic assessment” was first coined by Grant Wiggins in K‒12 educational contexts. Authentic assessment is an effective measure of intellectual achievement or ability because it requires students to demonstrate their deep understanding, higher-order thinking, and complex problem solving through the performance of exemplary tasks. Hence authentic assessment can serve as a powerful tool for assessing students’ 21st-century competencies in the context of global educational reforms. The review begins with a detailed explanation of the concept of authentic assessment. There is a substantial body of literature focusing on the definitions of authentic assessment. However, only those that are original and relevant to educational contexts are included.. Some of the criteria for authentic assessment defined by the authors overlap with each other, but their definitions are consistent. A comparison of authentic assessment and conventional assessment reveals that different purposes are served, as evidenced by the nature of the assessment and item response format. Examples of both types of assessments are included. Three major themes are examined within authentic assessment research in educational contexts: authentic assessment in educational or school reforms, teacher professional learning and development in authentic assessment, and authentic assessment as tools or methods used in a variety of subjects or disciplines in K‒12 schooling and in higher education institutions. Among these three themes, most studies were focused on the role of authentic assessment in educational or school reforms. Future research should focus on building teachers’ capacity in authentic assessment and assessment for learning through a critical inquiry approach in school-based professional learning communities or in teacher education programs. To enable the power of authentic assessment to unfold in the classrooms of the 21st century, it is essential that teachers are not only assessment literate but also competent in designing and using authentic assessments to support student learning and mastery of the 21st-century competencies.

Autoethnography is an increasingly popular form of postpositivist narrative inquiry that has recently begun to appear in educational contexts. The multiple lineages of autoethnography include the insider accounts of early anthropologists, literary approaches to life history and autobiography, responses to the ontological/epistemological challenges of postmodern philosophies, feminist and postcolonial insistence on including narratives of the marginalized, performance and communication scholarship, and the interest in personal stories of contemporary therapeutic and trauma cultures. Approaches vary widely from fragmented, experimental, performative, and multimodal texts through to realist tales. Advocates claim that autoethnography enables us to live more reflective, more meaningful, and more just lives.

The Common Core State Standards (CCSS) are part of a third wave of school reform in the United States. With accompanying tests, these standards combine calls for increased academic rigor, beginning in the 1980s, with more recent efforts to hold schools, teachers, and students accountable for learning outcomes in publicly funded schools. Origins of CCSS can be traced to the 1996 National Education Summit where the National Governors Association (NGA), philanthropic foundations, and business leaders founded Achieve to broker rigorous high school graduation requirements. In 2009, Achieve became the project manager for the construction of CCSS. In 2010, implementation began with incentives from the Obama administration and funding from the Gates Foundation.

Advocates choose among a variety of rationales: faltering American economic competitiveness, wide variability among state standards and educational outcomes, highly mobile student populations, and/or a growing income achievement gap. Critics cite federal intrusion in states’ rights, a lack of an evidentiary base, an autocratic process of CCSS production, and/or a mis-framing of problems facing public schools. With the reauthorization of the Elementary and Secondary Education Act (ESEA) as the Every Student Succeeds Act of 2015, federal advocacy of CCSS ended officially.

The European Commission launched a renewed agenda for adult learning with the objective of ensuring access to high-quality educational opportunities to adult learners for the promotion of their personal and professional development. Thus, European researchers in this field are paying attention to lifelong learning actions in order to address this challenge. Studies in this area are exploring how adult education can strengthen adults’ skills, in particular those required in the current knowledge society (information and communication technologies, problem solving, foreign languages, etc.). Simultaneously, some investigations focus in depth on the role that adult education can play in overcoming social exclusion for the most underserved groups. This paper describes the contributions of these investigations as well as the steps carried out by programs and theories that have contributed the most to adult learning. Lastly, future developments and challenges on this field are explained.

Transnational flows of educational knowledge and research are fundamentally guided by the global geopolitics of knowledge—the historically constituted relations of power born out of the continuing legacy of modernity/coloniality. In the early nation-building stage of the 19th century, state-funded education was at the core of states’ pursuit for economic and social progress. Newly formed nation states actively sought new educational knowledge from countries considered more advanced in the global race toward modernity and industrialization. The transnational lesson drawing in education at the time was guided by the view of modernity as originating in and diffusing from the West. This created the unidirectional flow of educational influence from advanced economies of the West to the rest of the world. Central to the rise of modernity in Western state formation is the use of education as a technology of social regulations. Through the expansion of state-funded education, people were turned into the people, self-governing citizens, and then the population that was amenable to a state’s social and economic calculation and military deployment. But this development was embedded in the geopolitical context of the time, in which Western modernity was deeply entangled with its underside, coloniality in the rest of the world. Various uses of education as a social control were tested out first in colonial peripheries and then brought back to the imperial centers.

Today, the use of education for the modernist pursuit of perfecting society has been intensified through the constitution of the globalized education policy space. International organizations such as the World Bank and the Organisation for Economic Co-operation and Development (OECD) act as the nodes through which transnational networks of education policy actors are formed, where the power of statistics for social and educational progress is widely shared. Both developed and developing countries are increasingly incorporated into this shared epistemological space, albeit through different channels and due to different factors. The rise of international academic testing such as OECD’s Programme for International Student Assessment (PISA) has certainly changed the traditional pattern of education research and knowledge flows, and more lesson drawing from countries and regions outside the Anglo-European context is pursued. And yet, the challenges that PISA poses to the Eurocentric pattern of educational knowledge and research flows are curtailed by the persistence of the colonial legacy. This most clearly crystalizes in the dismissive and derogatory characterization of East Asian PISA high achievers in the recent PISA debate. Hence, the current globalization of education knowledge and research remains entangled with the active legacy of coloniality, the uneven global knowledge structure.

The longitudinal study of teachers gives a time perspective on the life and work of teachers, instead of just a snapshot at a particular point. The time period in question may be just a few intense months, as in some ethnographic research, or several decades, as in some life-history research. Longitudinal research is useful in exploring such topics as how teachers change and grow over their careers, changes in teachers’ professional satisfaction over the years, patterns of teacher retention and drop-out, the impact of teachers on their students over time, and the influence of preservice and/or in-service teacher education on teachers.

Continuous study of the same teachers over many years is challenging and accordingly not common. It is typically expensive and time-consuming, and extends beyond the time span of most research funding; moreover, many participants either leave the profession or move to other locations, making it difficult to keep in touch with them. Accordingly, additional ways to do longitudinal research need to be found: for example, studying teachers intensively for a shorter period; asking teachers to recall earlier phases in their life and/or career; or studying different cohorts of teachers at various career points (as in the classic Huberman study and parts of the U.K. VITAE research). Each of these methods has limitations but maintains the valuable outcome of providing a time perspective.

Where it can be arranged, however, interviewing the same teachers at intervals over several years has the advantage of enabling researchers to get to know the participants well. As a result, the researchers are in a better position to understand what the participants are saying in the interviews, and assess the veracity of their self-reporting about their views and practices, past and present. Also, a degree of trust is established such that the teachers are more likely to be frank about their feelings, challenges, and concerns. But one danger of the emerging relationship is that the support the relationship it provides may positively impact the teachers’ experience (e.g., helping them fine-tune their practice and maintain their morale to an unusually high level). This limitation has to be weighed against the advantages in deciding whether or not to use this approach to the longitudinal study of teachers.

At core, moral and character education aims to develop the moral person. How this end state develops has been hindered by interest from different theoretical positions, differences between practitioners and theoreticians, different assumptions about how far character is educable, and associated measurement problems. Traditionally, moral education is concerned with the interpretation and strategies one uses to understand moral phenomenon and defines the moral person as a predominantly thinking entity, whereas character education emphasizes the development of habits and dispositions as a precondition for the moral person. Current interest is in finding commonalities across these traditions towards the achievement of human flourishing. These points of intersection have often been overlooked, but current work is demonstrating the importance of interdisciplinary and multidisciplinary approaches for practitioners, researchers and policymakers.

This is an advance summary of a forthcoming article in the Oxford Research Encyclopedia of Education. Please check back later for the full article.

From rites of passage to closer community bonding, the practice, enjoyment, exchange, and transmission of music—regardless of the setting—is an integral element of the history of human civilization. While the field of music education research has long focused on school music and institutional teaching, it is increasingly reaching out to the wider community, in the process involving people at different life stages who are operating in a variety of societal contexts. Consequently, research in music education explores a broad spectrum of musical engagements (including composition and improvisation, in addition to singing, playing, and listening) and a wide-ranging repertoire (including jazz, popular music, folk, and world music), together with diverse pedagogies both inspired by and borrowed from these genres. This process reveals how these forms of musical transmission can, on the one hand, create new meanings and experiences at individual levels, and, on the other, shape collective identity formation through the facilitation of cultural sustainability and transformation. By means of quantitative, qualitative, historical, and philosophical methods, and typically drawing on the fields of—among others—psychology, sociology, and anthropology, music education researchers have addressed social, cultural, pedagogical, and ethical issues of music teaching and learning.

Emerging in the learning sciences field in the early 1990s, qualitative design-based research (DBR) is a relatively new methodological approach to social science and education research. As its name implies, DBR is focused on the design of educational innovations, and the testing of these innovations in the complex and interconnected venue of naturalistic settings. As such, DBR is an explicitly interventionist approach to conducting research, situating the researcher as a part of the complex ecology in which learning and educational innovation takes place.

With this in mind, DBR is distinct from more traditional methodologies, including laboratory experiments, ethnographic research, and large-scale implementation. Rather, the goal of DBR is not to prove the merits of any particular intervention, or to reflect passively on a context in which learning occurs, but to examine the practical application of theories of learning themselves in specific, situated contexts. By designing purposeful, naturalistic, and sustainable educational ecologies, researchers can test, extend, or modify their theories and innovations based on their pragmatic viability. This process offers the prospect of generating theory-developing, contextualized knowledge claims that can complement the claims produced by other forms of research.

Because of this interventionist, naturalistic stance, DBR has also been the subject of ongoing debate concerning the rigor of its methodology. In many ways, these debates obscure the varied ways DBR has been practiced, the varied types of questions being asked, and the theoretical breadth of researchers who practice DBR. With this in mind, DBR research may involve a diverse range of methods as researchers from a variety of intellectual traditions within the learning sciences and education research design pragmatic innovations based on their theories of learning, and document these complex ecologies using the methodologies and tools most applicable to their questions, focuses, and academic communities.

DBR has gained increasing interest in recent years. While it remains a popular methodology for developmental and cognitive learning scientists seeking to explore theory in naturalistic settings, it has also grown in importance to cultural psychology and cultural studies researchers as a methodological approach that aligns in important ways with the participatory commitments of liberatory research. As such, internal tension within the DBR field has also emerged. Yet, though approaches vary, and have distinct genealogies and commitments, DBR might be seen as the broad methodological genre in which Change Laboratory, design-based implementation research (DBIR), social design-based experiments (SDBE), participatory design research (PDR), and research-practice partnerships might be categorized. These critically oriented iterations of DBR have important implications for educational research and educational innovation in historically marginalized settings and the Global South.