How can assessment improve student learning 2018

how does the internet improve student learning and how does the use of technology improve student learning
Dr.KeiraCollins Profile Pic
Dr.KeiraCollins,United States,Professional
Published Date:07-07-2017
Your Website URL(Optional)
Comment
Learning from Leadership p roject Investigating the Links to Improved Student Learning Final report o F r esearch Findings z z z Karen seashore l ouis Kenneth l eithwood Kyla l. Wahlstrom stephen e. anderson University of Minnesota University of Toronto commissioned by center for applied research ontario institute for The Wallace Foundation and educational improvement studies in educationContents Starting Points ................................................................................................................................7 Part One: What School Leaders Do to Improve Student Achievement .................................16 Preface ........................................................................................................................................16 1.1 Collective Leadership Effects on Teachers and Students ....................................................19 1.2 Shared Leadership: Effects on Teachers and Students of Principals and Teachers Leading Together .......................................................................................................37 1.3 Patterns of Distributed Leadership by Principals: Sources, Beliefs, Interactions, and Influences……………………………… ...................................................................................54 1.4 Leadership Practices Considered Instructionally Helpful by High Performing Principals and Teachers .............................................................................................................66 1.5 Instructional Leadership: Elementary vs. Secondary Principal and Teacher Interactions and Student Outcomes ...........................................................................................77 1.6 Poverty, Size, Level and Location: The Influence of Context Variables on What Leaders Do and What They Accomplish ..................................................................................94 1.7 A Synthesis of Implications for Policy and Practice about School Leadership ................103 Part Two: Districts and Their Leaders: How They Foster School Improvement and Student Learning ...............................................................................................................105 Preface ......................................................................................................................................105 2.1 How Districts Harness Family and Community Energies for School Improvement ........107 2.2 Principals‘ Efficacy: A Key to District Effects on Schools and Students ..........................127 2.3 How Districts Build Principals‘ Sense of Efficacy for School Improvement ....................148 2.4 Ensuring Productive Leadership Succession .....................................................................165 2.5 Data Use in Districts and Schools: Findings and Limitations ...........................................179 2.6 District Approaches to Improving Teaching and Learning ...............................................197 2.7 A Synthesis of Implications for Policy and Practice about District Leadership ...............215 Part Three: State Leadership and Relationships with Districts ...........................................218 Preface ......................................................................................................................................218 3.1 State Political Cultures and Policy Leadership .................................................................220 3.2 The Changing Leadership Role of State Education Agencies ...........................................231 3.3 District and School Responses to State Leadership ..........................................................246 3.4 State Leadership for School Improvement: A Synthesis of Implications for Policy and Practice ..................................................................................................................279 Conclusion ..................................................................................................................................282 References ..................................................................................................................................285 Appendices .................................................................................................................................309 5 Starting Points Purposes for the Study Education is widely held to be crucial for the survival and success of individuals and countries in the emerging global environment. U.S. politicians of all stripes have placed education at the center of their political platforms, and education has been at the center of many European and Asian policy agendas. Comparable agreement is also evident about the contributions of leadership to the implementation of virtually all initiatives aimed at improving student learning and the quality of schools. It is therefore difficult to imagine a focus for research with greater social justification than research about successful educational leadership. That was the broad focus for this six-year study funded by the Wallace Foundation: to identify the nature of successful educational leadership and to better understand how such leadership can improve educational practices and student learning. More specifically, we sought to do the following:  Identify state, district, and school leadership practices that directly or indirectly foster the improvement of educational practices and student learning.  Clarify how successful leadership practices directly and indirectly influence the quality of teaching and learning.  Determine the extent to which individuals and groups at state, district, school, and classroom levels possess the will and skill required to improve student learning, and the extent to which their work settings allow and encourage them to act on those capacities and motivations.  Describe the ways in which, and the success with which, individuals and groups at the state, district, school, and classroom levels help others to acquire the will and skill required to improve student learning.  Identify the leadership and workplace characteristics of districts and schools that encourage the values, capacities, and use of practices that improve student learning. The Educational Leadership Effect Although leadership is widely thought to be a powerful force for school effectiveness, this popular belief needs to be justified by empirical evidence. There are five types of such evidence, each offering its own estimate of the size of leader effects. One type is evidence from qualitative case studies. Studies providing this type of evidence typically are conducted in exceptional school settings, selected as exemplars of 1 effectiveness. Some such studies report large leadership effects—on student learning and on an array of school conditions. Other qualitative studies focus on ―typical‖ schools 1 See, e.g., Gezi (1990); Reitzug & Patterson (1998). 7 rather than outliers; these studies often produce complex pictures of how leadership 2 operates in different settings. Many educators and scholars find the descriptions provided by case studies to be interesting and informative. But descriptions of a small number of cases do not yield explanations of leadership effects for a more general 3 population of schools. The second type of evidence derives from large-scale quantitative studies of leadership effects on schools and students. Evidence of this type, as reported and 4 reviewed since about 1980, suggests that the direct and indirect effects of school leadership on student learning are small but significant. Leadership explains five to seven percent of the variation in student learning across schools (not to be confused with the very large within-school effects that are likely). Five to seven percent, however, is about one quarter of the total across-school variation (12 to 20 percent) explained by all school- 5 level variables, after controlling for student intake or background factors. (Classroom factors explain more than a third of the variation.) To date, however, research of this sort has done little to clarify how leaders achieve the effects in question, and its implications for leadership practice are, therefore, limited. A third type of evidence derives from studies (also large-scale and quantitative) focused on the effects of specific leadership practices. Some evidence of this sort can be found in the research briefly summarized above. But a meta-analysis conducted by Waters, Marzano and McNulty (2003) extends our understanding of the explanatory potential of this type of research. Waters et al. identify 21 leadership ―responsibilities‖ (behaviors); then they calculate an average correlation between each responsibility and the measures of student learning used in the original studies. From these data they calculate estimated effects of the respective responsibilities on student test scores. For example: there would be a 10 percentile point increase in student test scores resulting from the work of an average principal if she improved her ―demonstrated abilities in all 21 responsibilities by one standard deviation‖ (2003, p. 3). Extending this line of inquiry, Marzano et al. (2005) provide a comparable analysis of research on district-level leadership, identifying five broad categories of superintendent leadership. A fourth type of evidence derives from studies of leadership effects on student engagement, as distinct from effects on student learning. Some evidence suggests that 6 student engagement is a strong predictor of student learning. Recently, at least 10 large- scale, quantitative studies, similar in design, have assessed the effects of leadership 7 behavior on student engagement; all have reported significant positive effects. 2 Spillane, Diamond, & Burch et al. (2002). 3 See, e.g., Mortimore (1993), and Scheurich (1998). 4 See, e.g., Hallinger & Heck (1996b); Leithwood & Jantzi (2005); Marzano, Waters & McNulty (2005); and Robinson, Lloyd, & Rowe (2008). 5 Creemers & Reetzig (1996), and Townsend (1994). 6 See Fredricks, Blumenfeld, & Paris (2004) for a review, especially at p. 70. 7 Leithwood & Jantzi (1999a, 1999b); Leithwood et al. (2004a); Silins & Mulford (2002b); and Silins, Mulford, & Zarins (2002). 8 Finally, a different but quite compelling sort of evidence about leadership effects derives from research on leadership succession. Unplanned principal succession, for example, is a common source of adverse effects on school performance, regardless of what teachers might do. Studies by Macmillan (2000) and Fink & Brayman (2006) demonstrate the devastating effects of rapid principal succession, especially on initiatives intended to increase student learning. And rapid succession is very common. Clearly, leadership matters. In developing a starting point for this six-year study, we claimed, based on a 8 preliminary review of research, that leadership is second only to classroom instruction as an influence on student learning, After six additional years of research, we are even more confident about this claim. To date we have not found a single case of a school improving its student achievement record in the absence of talented leadership. Why is leadership crucial? One explanation is that leaders have the potential to unleash latent capacities in organizations. Put somewhat differently: most school variables, considered separately, 9 have only small effects on student learning . To obtain large effects, educators need to create synergy across the relevant variables. Among all the parents, teachers, and policy makers who work hard to improve education, educators in leadership positions are uniquely well positioned to ensure the necessary synergy. Meanings of Leadership Leadership can be described by reference to two core functions. One function is providing direction; the other is exercising influence. Whatever else leaders do, they provide direction and exercise influence. This does not imply oversimplification. Each of these two leadership functions can be carried out in different ways, and the various modes of practice linked to the functions distinguish many ―models‖ of leadership. In carrying out these two functions, leaders act in environments marked variously by stability and change. These conditions interact in complementary relationships. While stability is often associated with resistance and maintenance of the status quo, it is in fact difficult for leaders and other educators to leap forward from a wobbly foundation. To be more precise, it is stability and improvement that have this symbiotic relationship. Leaping forward from a wobbly foundation may well produce change, but not change of the sort that most of us value—falling flat on your face is the image that comes to mind. Wobbly foundations and unwise leaping help to explain why the blizzard of changes adopted by our schools over the past half century have had little effect on the success of our students. School reform efforts have been most successful in those schools that have 10 needed them least. These have been schools with well-established processes and capacities in place, providing foundations on which to build—in contrast to those schools, the ones most often of concern to reformers, short on essential infrastructure. How do these concepts come together in a clarification of leadership? Leadership is all about organizational improvement; more specifically, it is about establishing 8 Leithwood, Louis, Anderson & Wahlstrom (2004) 9 Creemers & Reetzigt, 1996 10 Elmore (1995) 9 agreed-upon and worthwhile directions for the organization in question, and doing whatever it takes to prod and support people to move in those directions. Our general definition of leadership highlights these points: it is about direction and influence. Stability is the goal of what is often called management. Improvement is the goal of leadership. But both are very important. One of the most serious threats to stability in a school district is frequent turnover in the ranks of superintendents, principals, and vice principals. Instability at the school level often reflects a failure of management at the district level. Alternative Models of Leadership Reflected in the Literature Leadership in non-school contexts. Research on leadership in non-school contexts is frequently driven by theory referred to by one of our colleagues as ―adjectival leadership models.‖ A recent review of such theory identified, for example, 21 leadership approaches that have been objects of considerable theoretical and empirical 11 development. Seventeen have been especially attractive, and some of them have 12 informed research in school contexts. Here are some examples.  Contingent leadership. Encompassing research on leadership styles, leader problem solving, and reflective leadership, this two-dimensional conception of leadership explains differences in leaders‘ effectiveness by reference to a task or relationship style and to the situations in which leaders find themselves. To be most effective, according to this model, leaders must match their styles to their settings.  Participative leadership. Addressing attention to leadership in groups, shared 13 14 leadership, and teacher leadership, this model is concerned with how leaders involve others in organizational decisions. Research informed by the model has investigated autocratic, consultative, and collaborative sharing styles.  Transformational and charismatic leadership. This model focuses on ways in which leaders exercise influence over their colleagues and on the nature of leader-follower relations. Both forms of leadership emphasize communicating a compelling vision, conveying high performance expectations, projecting self confidence, modeling appropriate roles, expressing confidence in followers‘ ability to achieve goals, and 15 emphasizing collective purpose. Leadership in education. Leadership research also has been informed by models developed specifically for use in school- and district-level settings. Of these, the instructional leadership model is perhaps the most well known. (It bears some 16 resemblance to more general, task-oriented leadership theories. ) The instructional leadership concept implies a focus on classroom practice. Often, however, specific 11 Yammarino, Dionne, Chun, & Dansereau (2005). 12 Leithwood & Duke (1999). 13 E.g., Pearce & Conger (2003). 14 E.g., York-Barr & Duke (2004). 15 E.g., Leithwood & Jantzi (2006). 16 Dorfman & House (2004). 10 leadership practices required to establish and maintain that focus are poorly defined. The main underlying assumption is that instruction will improve if leaders provide detailed feedback to teachers, including suggestions for change. It follows that leaders must have the time, the knowledge, and the consultative skills needed to provide teachers—in all the relevant grade levels and subject areas—with valid, useful advice about their instructional practices. While these assumptions have an attractive ring to them, they rest on shaky ground, at best; the evidence to date suggests that few principals have made the time and 17 demonstrated the ability to provide high quality instructional feedback to teachers. Importantly, the few well-developed models of instructional leadership posit a set of responsibilities for principals that go well beyond observing and intervening in 18 classrooms—responsibilities touching on vision, organizational culture, and the like. In addition, studies of school leadership are replete with other adjectives purporting to capture something uniquely important about the object of inquiry—for example, learning 19 20 21 leadership, constructivist leadership, and change leadership. Few of these efforts, however, have been products of a sustained line of inquiry yielding the sort of evidence needed to justify their claims. This observation influenced our approach as we began our study. Eschewing any particular model of leadership, we examined the actual practices, across models, for which there was significant evidence of desirable effects. Significant Features of Our Research The investigation reported here was among the largest of its kind at the time we conducted it. Its particularly noteworthy features, as against other educational leadership studies, include the size of the data base, the use of multiple theoretical and methodological approaches to the research, and the comprehensive sources of leadership examined. Size of the data base. We collected data from a wide range of respondents in nine states, 43 school districts, and 180 elementary, middle, and secondary schools. At the state level, we conducted interviews with legislators, stakeholders, and members of state education agencies. In districts, we interviewed senior district leaders, elected board members, representatives of the media, and other informants. We used survey instruments and interviews with teachers and administrators, and we conducted classroom observations with most of the teachers we interviewed. We collected survey data in the first and fourth years of the study; we conducted interviews in districts and schools in three cycles over the five years of the project. These efforts yielded, by the end of the project, survey data from a total of 8,391 teachers and 471 school administrators; interview data from 581 teachers and administrators, 304 district level informants, and 124 state personnel; and observational data from 312 classrooms. Finally, we obtained student achievement data for literacy and mathematics in elementary and secondary grades, using scores on the states‘ tests for measuring Adequate Yearly Progress as 17 E.g., Nelson & Sassi (2005). 18 Andrews & Soder (1987), Duke (1987), and Hallinger (2003). 19 Reeves (2006). 20 Lambert et al. (1995). 21 E.g., Wagner et al. (2006). 11 mandated by the No Child Left Behind Act of 2002. (For a detailed description of the data base, see the Methodological Appendix.) Multiple methodological approaches. We used qualitative and quantitative methods to gain certain advantages associated with multiple-methods research. The advantages typically include ―rich opportunities for cross-validating and cross- fertilizing…procedures, findings, and theories‖ (Brewer & Hunter, 1989, p. 13). Our particular use of multiple methods offered opportunities that we had not fully appreciated in the early stages of our work. These included opportunities to discover significant patterns and relationships in our quantitative evidence, which we were then able to pursue in greater depth, thanks to our qualitative evidence. One example appears in Section 2.2. From the analysis of our first-round survey data we found that one of the most powerful sources of districts‘ influence on schools and students was through the development of school leaders‘ collective sense of efficacy about their jobs. With this connection well established quantitatively, we then mined principal-interview data to learn in greater detail what districts actually did to develop a sense of efficacy among principals. Similar examples of this approach to our data can be found in Sections 2.4, 2.5, and (taken as a whole) Sections 1.1 to 1.3. Multiple theoretical perspectives. In collecting data and working to make sense of our results, we drew upon conceptual tools from sociology, socio-psychology, political science, and organizational theory. Sociological concepts informed our understanding of shared leadership (1.2), contexts for leadership (1.5), and community engagement (2.1). Socio-psychological perspectives helped us analyze leader efficacy (2.2) and (along with organizational theory) the nature of successful leadership practices (1.4), as well as the use of evidence in districts and schools (2.5), and leader succession (2.4). Political science concepts framed our research about state leadership (3.1). Our goal with this seemingly eclectic approach was to draw on the theoretical perspectives best suited to the question at hand—an approach especially well suited to a project like ours with multiple principal investigators who had studied and used each strand of theory in their prior work. We shared the view that using multiple methods and theoretical perspectives can provide a powerful antidote to the unintended self-deceptions that sometimes arise from the use of more unitary approaches. Our approach, however, also challenged us to develop a valid and coherent storyline from the data. In that effort, inevitably, we have sacrificed some measure of coherence in order to present a rich account of our findings. Comprehensiveness of sources of leadership. Most leadership studies in education focus on a single institutional role. The bulk of it focuses on the principals‘ 22 23 role, with a growing but still modest body of attention to district-level leadership. Over the past decade, researchers have also begun to study leadership provided by 24 teachers. 22 E.g., Robinson et al. (2008). 23 Marzano, Waters & McNulty (2005). 24 York-Barr & Duke (2004). 12 The recent flurry of attention to a broader spectrum or distribution of leadership has begun to sensitize us to the remarkable array of people who exercise formal or informal leadership in schools and districts. Research of this sort also shows that the influence of leadership on organizational outcomes arises from the behaviors of these various people acting as leaders in either an ―additive‖ or ―holistic‖ manner (Gronn, 2009. We cannot push our understanding of leadership influence much further without considering the many sources of leadership in the education system and also the web of interaction created by these sources. To date, our study is one of only a few to have examined leadership at each organizational level in the school system as a whole—state, district, school, classroom, and community. The comprehensive approach reminds us that every leader is at the same time constrained and enabled in some measure by the actions of others (including other leaders), and by the consequences of those actions. Without a better understanding of such antecedents and consequences, we are left with an impoverished appreciation of why leaders behave as they do. Invoking social theory, the more comprehensive perspective has the potential to shift the field of educational leadership research from a dominant preoccupation with ―agency‖ (explaining leaders‘ behaviors as a function of individual capacities, motivations, and traits), toward a more balanced understanding of how the structures within which leaders work also shape the work that they do. Framework Guiding the Study The framework guiding our study emerged from a review of scholarship 25 completed prior to our data collection and summarized in Figure 1. According to information summarized in this figure, features of state and district policies, practices, and other characteristics interact with one another and exert an influence on what school leaders do. These features also influence conditions in schools, classrooms, and the professional community of teachers (for the sake of simplicity, we do not connect these variables in Figure 1). Other stakeholder groups, including the media, unions, professional associations, and community and business groups also influence school leadership practices. And of course leaders are influenced by their own professional learning experiences and by student and family backgrounds. 25 Leithwood, Louis, Anderson, & Wahlstrom (2004). 13 State Leadership, Student/ Family Policies and Background Practices School Conditions District Leadership, School Student Policies and Teachers Leadership Learning Practices Leaders’ Classroom Professional Other Conditions Development Stakeholders Experiences Figure 1. Leadership Influences on Student Learning School leadership, from formal and informal sources, helps to shape school conditions (including, for example, goals, culture, and structures) and classroom conditions (including the content of instruction, the size of classrooms, and the pedagogy used by teachers). Many factors within and outside schools and classrooms help to shape teachers‘ sense of professional community. School and classroom conditions, teachers‘ professional communities, and student/family background conditions are directly responsible for the learning of students. Overview of the Report The six-year study reported here focuses on leadership at the school, district, and state levels. The report is organized in three main parts, with one part dedicated to each leadership level. Within each part (following a preface) there are three to six sections describing the results of sub-studies conducted within the larger project, in pursuit of specific research goals. Each section begins with an overview of the significant findings for that particular sub-study. We chose to provide the Key Findings at the beginning as a way to orient the reader‘s attention to the details that follow. Also, each section concludes with ―Implications for Policy and Practice‖. Again, we wanted to direct the reader‘s thinking to what could or should be done in schools and districts to support or improve reform efforts. Our assertions for changes in policy and practice, as based on our findings, are not intended to be definitive, but rather as a starting place for the reader. Part One focuses on school-level leadership. It summarizes three perspectives on the sources and distribution of school-level leadership practices; it identifies effects on 14 students and features of the school that influence the size of those effects; and it describes successful leadership practices. Part Two focuses on school district leadership. It describes ways in which districts engage parents and the community in their school-improvement efforts; it explores the impact of such engagement on students; it tells how districts develop school leaders‘ sense of efficacy; it explains what districts can do to ensure productive leader succession; and it describes ways in which typical and exemplary districts use school data. One section of Part Two paints a broad and integrated picture of district approaches to improving teaching and learning. Part Three focuses on state-level leadership. Three sections describe variations in the forms of leadership exercised by states through the development and implementation of education policy. A fourth section describes the leadership provided by state education agencies and the quite different relationship districts develop with their states. 15 Part One What School Leaders Do to Improve Student Achievement Preface With its focus on school-level leadership, Part One seeks to identify, elaborate, and clarify existing knowledge about successful leadership practices. Because leadership is enacted by many people in schools, we begin by addressing the nature, causes, and consequences of the alternative forms and patterns of leadership among school and district staff members. Our evidence about leadership distribution contributes to an ongoing conversation among researchers and practitioners aimed at determining 26 implications for school improvement. To obtain evidence about leadership distribution and its effects, we conducted our examination through the use of distinctly different lenses. Our observations made by way of these lenses yield a richer understanding of leadership distribution than we could have attained via a narrower approach. Section 1.1 is concerned with the influence various stakeholders (parents and other community members, for example) may have on school decisions. Our work in this section has some bearing on the definition of leadership. Many texts describe leadership 27 as an ambiguous, evolving concept, yet to be clearly defined. Indeed, Stogdill argued many years ago that ―there are almost as many definitions of leadership as there are persons who have attempted to define the concept‖ (1974, p. 259). Our own reading suggests, however, that Yukl is correct in claiming that almost all definitions assume leadership entails at least some form of social influence which might be ―viewed as a property of an individual or a property of a social system‖ (1994, p. 3). Collective leadership, for our purposes, is defined by this minimalist but basic conception of leadership-as-influence—and as a property of the system rather than an individual. Evidence about collective leadership reported in Section 1.1 reveals the extent of influence exercised by most stakeholders in and around schools on decisions in the school. This section also indicates that there is considerable variation across schools in the nature and extent of stakeholders‘ influence, and it suggests that student achievement benefits from relatively greater influence by all stakeholders in school decisions. Section 1.2 adopts a ―shared‖ conception of distributed leadership, one typically reflecting a group- or team-level approach in which all members share responsibility for 28 leading contingent upon the task, the time required, and the expertise needed. In their recent text on shared leadership, Pearce and Conger (2003) trace the roots of this conception to two early studies. The first of these (Follett, 1924) essentially advocated 26 Comprehensive overviews of this research can be found in Harris (2009), and Leithwood, Harris & Hopkins (2008), for example. 27 E.g., Rost (1991). 28 Yammarino et al. (2005). 16 leadership through expert rather than positional power, whereas the second (Bowers & Seashore, 1966) provided evidence that peer sources of leadership in large organizations could have significant effects on organizational outcomes. We stipulated a narrower conception of shared leadership for the research reported in Section 1.2. This conception is oriented toward shared and contingent responsibility, but it focuses on leadership exercised by those most directly responsible for student learning—principals and teachers. Section 1.2 examines the effects on students of principals and teachers assuming shared responsibility for leadership; it also identifies some conditions that influence the emergence and mediate the effects of this approach to leadership in schools. The examination of distributed leadership in Section 1.3 introduces explicit leadership practices. By reference to a qualitative data set, this section discloses who enacts which practices, how different patterns of leadership enactment emerge, and whether variation in such patterns makes a difference for schools and students. Viewed from a principal‘s perspective, this research also suggests implications for how leadership might be distributed more productively in schools. Sections 1.4 and 1.5 identify the actual practices or behaviors, however distributed, giving rise to leadership influence on teaching and learning. Both sections report the perceptions of principals and teachers, selected according to quite different criteria, about the leadership practices they believe are helpful in improving classroom instruction. Section 1.4 is informed by a synthesis of results from a body of prior evidence about leadership practices demonstrably successful across organizational sectors 29 and national cultures. Using qualitative evidence from principals and teachers, this section assesses the relevance of these practices across different school contexts and provides greater detail about how they are enacted in those contexts. In Section 1.5, we take an additional step in our efforts to identify productive leadership practices. We adopt a grounded-theory approach to a different set of data, also collected from principals and teachers. This sub-study distinguishes between efforts by school leaders to create a vision and climate among staff members, on the one hand, and, on the other, the actions leaders take to realize that vision. Together, Sections 1.4 and 1.5 offer a detailed account of the leadership behaviors deemed by those closest to the action to be influential in shaping teachers‘ work with students. These sections also point to substantial differences in the extent to which these actions are enacted by formal leaders in elementary as compared to secondary schools. Section 1.6, building on analyses from the previous two sections, demonstrates that leaders, to be successful, need to be highly sensitive to the contexts in which they work. From one perspective, such contexts moderate (enhance or mute) the influence of any given set of leadership practices. From a more practical perspective, different 29 For example, see Leithwood et al. (2006); Robinson et al. (2008); and Waters et al. (2003). 17 contexts call for quite different enactments of the same basic set of successful leadership practices. Section 1.7 synthesizes implications for policy and practice arising from the six sections in Part One. 18 1.1 Collective Leadership Effects on Teachers and Students Key Findings  Collective leadership has a stronger influence on student achievement than individual leadership.  Almost all people associated with high-performing schools have greater influence on school decisions than is the case with people in low-performing schools.  Higher-performing schools award greater influence to teacher teams, parents, and students, in particular.  Principals and district leaders have the most influence on decisions in all schools; however, they do not lose influence as others gain influence.  Schools leaders have an impact on student achievement primarily through their influence on teachers‘ motivation and working conditions; their influence on teachers‘ knowledge and skills produces less impact on student achievement. Introduction Collective leadership, as the term is used in this component of our study, refers to the extent of influence that organizational members and stakeholders exert on decisions in their schools. This relatively narrow but fundamental perspective on leadership focuses attention on the combined effects of all sources of leadership, along with possible differences in the contributions made by each of these sources (e.g., administrators, teachers, students, parents). Guided by this conception of leadership, the sub-study set out to estimate the following:  the relative influence on school decision making of each of the individuals or groups potentially contributing to a school‘s collective leadership;  the impact of collective leadership on teacher feelings and beliefs and on student learning; and  whether differences in the extent of influence exerted by the respective participants is related to differences in levels of student achievement. 19 Prior Evidence Leadership as Influence The conception of collective leadership used for this study overlaps with Rowan‘s 30 conception of organic management, defined as follows: a shift away from conventional, hierarchical patterns of bureaucratic control toward what has been referred to as a network pattern of control, that is, a pattern of control in which line employees are actively involved in making organizational decisions, and staff cooperation and collegiality supplant the hierarchy as a means of coordinating work flows and resolving technical difficulties. (Miller & Rowan, 2006, p. 219-220) Conceptualizing collective leadership as a network of influence and control also locates our study in relation to other research about organizational control structures. A seminal paper by Tannenbaum (1961), for example, introduced the ―control graph‖ as a means of displaying patterns of control in formal organizations. The horizontal axis of a control graph designates each of the ―levels‖ (designated positions) in the organization, while the vertical axis represents the degree of perceived influence or control exercised at each level. Tannenbaum used the control graph to illustrate four prototypical control modes or approaches to leadership: autocratic (influence rises with the hierarchical level of the role), democratic (higher levels of influence are ascribed to those in hierarchically lower levels or roles), anarchic (relatively little influence by any level or role), and polyarchic (high levels of influence by all levels or roles). Reflecting Rowan‘s (1990) expectations for organic management under conditions of uncertainty, Tannenbaum also hypothesized that organizational effectiveness will be related to: (a) more democratic, and (b) more polyarchic forms of control. The first of these hypotheses arises from two sets of expectations. First, more democratic forms of control will be more consistent with employees‘ beliefs and values in a democratic society and contribute to higher levels of job satisfaction and morale, whereas autocratic forms of control are expected ―to reduce initiative, inhibit identification with the organization and to create conflict and hostility among members‖ (Tannenbaum, 1961, p. 35). Second, more control by those lower in the hierarchy will lead to greater acceptance of jointly-made decisions along with an increased sense of responsibility for and motivation to accomplish organizational goals. Such participation may also contribute to more effective coordination through mutual influence mechanisms. The second of Tannenbaum‘s hypotheses, sometimes called the ―power equalization‖ hypothesis, is justified, Tannenbaum claims, by certain results—by improved organizational efficiency realized when more control is exercised by those lower in the hierarchy, and by improved motivation and identification with the organization on the part of those whose power is enhanced. Reasons offered in the 30 Miller & Rowan (2006); Rowan (1990). 20 current literature about distributed leadership are quite similar to the justification Tannenbaum‘s offers for his two hypotheses. Collective Leadership Effects What evidence is there to show that democratic, supportive, and shared forms of leadership are effective? Some empirical evidence may be found in research on teacher 31 participation with peers in planning and decision making and in research on 32 transformational leadership. Several lines of related theory also give rise to expectations of a positive association between organizational effectiveness and the distribution of 33 34 influence, including theories of organizational learning, distributed cognition, and 35 communities of practice. Nonetheless, there is substantial evidence to the contrary, especially from research in which organizational effectiveness is defined as the organization‘s bottom line (some measure of productivity) and assessed using objective indicators, such as student test scores. Tannenbaum was able to provide only limited support for his hypotheses about organizational control structures. And after about 15 years of programmatic research about organic management, Miller and Rowan reported that ―the main effects are weak, and positive effects appear to be contingent on many other conditions‖ (2006, p. 220). A recent, comprehensive review of research on teacher leadership found only a small handful of studies in which researchers had actually inquired about effects of teacher 36 leadership on students, and the results were generally not supportive. To date, most research about school leadership has focused on the work of teachers and school administrators. It is certainly possible, however, to conceive of people acting in other roles—as parents, students, interested members of the community—to exercise influence in schools. The work of Pounder, Ogawa and Adams (1995) provides one example (there are not many) of research that examines leadership exercised by a broader array of participants. Pounder et al. test a model of the influence of principals, teachers, parents, and secretaries on a number of mediating variables, as well as a range of school outcomes, providing a useful model for our approach a decade later . The current sub-study looks beyond the school setting in its examination of leadership. Staff members in district roles also have an obligation to influence what schools do, although most studies of collective, shared, and distributed leadership have 37 not examined the contribution of district personnel. Our study concerned itself with all of these potential sources of influence. 31 Talbert & McLaughlin (1993). 32 Leithwood & Jantzi (2005). 33 Hutchins (1996). 34 Perkins, 1993; Tsoukas (2005). 35 Wenger, McDermott & Snyder (2002). 36 York-Barr & Duke (2004). 37 But see Firestone (1989), and Firestone & Martinez (2007). 21 Antecedents of Teacher Performance Miller and Rowan (2006) sought to assess certain effects of organic management. In this effort they did not attend to variables potentially mediating the effects of leaders on student learning. This is an important limitation, given prior work (Pitner, 1988; Hallinger & Heck, 1996a) showing that the effects of leadership on students are largely indirect. Studies designed to explore direct effects of leadership rarely detect significant effects, whereas many studies of indirect effects do. Most studies since 1996 have been 38 guided by complex causal models which include a wide array of potential mediators. The framework for this sub-study assumed indirect leadership effects and conceptualized as mediators a set of teacher performance antecedents including motivation, capacity, and the situations in which people work. These are variables in a general model of employee performance and how it improves. Our own modification of this framework is based on theoretical and empirical accounts of the conditions required for development of motivation and capacity on the part of school people to engage productively in improvement efforts. Our modification also incorporates accounts of organizational conditions and characteristics of the infrastructure which facilitate the successful implementation of large-scale reform, or what van den Berg, Vandenberghe, 39 and Sleegers (1999) refer to as the organization‘s ―innovative capacity.‖ New Evidence Method Sample. This sub-study is based on data collected in the first round of surveys for the larger study. The achieved sample included responses by 2,570 teachers (77% response rate) from a total of 90 schools in which seven or more teachers completed 40 usable surveys and for which usable student achievement data were available. Table 1.1.1 below presents a summary of the characteristics of our achieved sample. TABLE 1.1.1 Sample School Characteristics Mean SD Student Diversity (1=Low, 3 = High) 1.97 .71 Percent of Students Eligible for Free Lunch 43.82% 27.67 Achievement (Mean % at Proficiency or Above) 67.19% 24.27 38 For example, Leithwood & Levin (2005) and Leithwood, Louis, Anderson, & Wahlstrom (2004). 39 For a more detailed explanation of how these variables were defined and measured, see Leithwood & Jantzi (2008). 40 We were able to generate data on the SES of only 76 of these schools, so the calculations for tables drawing on SES have been adjusted to use this smaller sample. 22 Sources of evidence. To measure student achievement across schools, we collected data from state websites. These data comprised school-wide results on state- mandated tests of language and mathematics at several grade levels over three years (2003 to 2005). We represented a school‘s level of student achievement by the percentage of students meeting or exceeding the proficiency level (usually established by the state) on language and mathematics tests. We averaged these percentages across grades and 41 subjects in order to increase the stability of scores, arriving finally at a single achievement score for each school for each of three years. Our analysis also included an achievement change score, calculated as the gain in percentage of students attaining or exceeding the state-established proficiency level from the first to the third year for which we had evidence. Teacher responses to 49 items from a 104-item survey provided the remaining data for this sub study. The survey, which required about 20 minutes to complete, measured the collective leadership and teacher-performance antecedents described in our framework: 9 items measured collective leadership, 9 items measured teacher capacity, 17 items measured teacher motivation, and 14 items measured teacher work settings or conditions. Each of the nine items used to measure collective leadership pertained to a single source of influence from a set including district administrators, principals, other school administrators, some individual teachers, teachers with designated leadership roles, staff teams, some individual parents, parent advisory groups, and students. About each source of influence, we asked respondents to rate the extent of direct influence on school decisions (on a 6-point scale). We also asked respondents to rate the extent to which they agreed with statements about each of the three antecedents of teacher performance, also on a 6-point scale. Analysis. We merged individual responses to the teacher survey, aggregated to the school level, with school-level student achievement results. We used SPSS to calculate means, standard deviations, and reliabilities (Cronbach‘s alpha) for scales measuring the four variables. We used paired-sample t-tests to compare mean ratings of various sources of leadership. We tested the factor structure of the teacher variables included in the study. We used hierarchical multiple regression to examine the moderating effects of student SES on some relationships in our framework. Finally, we used LISREL to test a model of the relationships among collective leadership, teacher motivation, capacity and setting, and student achievement. This path-analytic technique allows for testing the validity of causal inferences for pairs of variables while controlling for the effects of other variables. We analyzed data using the LISREL 8.80 analysis of 42 covariance structure approach to path analysis and maximum likelihood estimates. We used four goodness-of-fit statistics to assess the fit of our path model with the data: the Root Mean Square Error of Approximation test (RMSEA), the Norm-fit index (NFI), the adjusted Goodness of Fit index (GFI) and the mean Root Mean Square Residual (RMR). 41 Linn (2003). 42 Joreskog & Sorbom (1993). 23 Results We begin with a summary of responses to the teacher survey and with information about the statistical properties of our measures, including the results of a factor analysis of the measures of teacher capacity, motivation, and setting. The remaining sections report evidence relevant to each of three questions addressed by the study: the impact of collective leadership on key teacher variables and student learning; the relative influence of different collective leadership sources; the relationship between different patterns of collective leadership and student achievement. Table 1.1.2 reports the internal reliabilities (Cronbach‘s alpha) of the scales used to measure each of the three antecedents of teacher performance—capacity, motivation and work setting—and the measure of collective leadership. Overall mean ratings of the three antecedents are not reported because z-scores had to be calculated to accommodate the use of different response scales. We calculated variable reliabilities using z-scores. Responses to all variables ranged between slight agreement and moderate agreement, with low to moderate standard deviations. All scales achieved acceptable levels of reliability (between .72 and .96). TABLE 1.1.2 Scale Reliability for Variables (N = 90 Schools) Cronbach‘s Alpha Capacity .86 Motivation .96 Setting .91 Collective leadership .72 Note: z-scores were used to calculate the aggregate values for the capacity, motivation, and setting scales. Collective leadership was calculated from the sum of nine sources of leadership, each rated on a 6-point scale from ‗no influence‘ to ‗very great influence.‘ Of the 40 items used to measure the three teacher antecedents, 9 measured capacity, 17 measured motivation, and 14 measured work setting. We analyzed the dimensionality of these 40 items using principal component factor analysis. We used the scree test and the interpretability of the factor solution to determine the number of factors to rotate. We rotated three factors using a Varimax rotation procedure. The rotated solution yielded three interpretable factors which corresponded very closely with the three variable categories: capacity, motivation, and setting. The capacity factor accounted for 14.4% of the item variance; the motivation factor accounted for 13.9% of the item variance; and the setting factor accounted for 8.6% of the item variance. Although our initial conception of the three teacher variables suggested a number of distinct sub-dimensions, these were not supported by the factor analysis. Thus, we used aggregate scores for each of the three teacher-performance antecedents in all 24