How to take Research notes

how to research notes thaumcraft and how to organize research notes | Download free pdf
LottieBarners Profile Pic
Published Date:12-07-2017
Your Website URL(Optional)
Research Notes Issue 50 November 2012 ISSN 1756-509XResearch Notes Issue 50 / November 2012 A quarterly publication reporting on research, test development and validation Guest Editor Dr Jayanti Banerjee, Research Director, Cambridge Michigan Language Assessments Senior Editor and Editor Dr Hanan Khalifa, Head of Research and Publications, Research and Validation Group, Cambridge ESOL Coreen Docherty, Senior Research and Validation Manager, Research and Validation Group, Cambridge ESOL Editorial Board Dr Nick Saville, Director, Research and Validation Group, Cambridge ESOL Production Team Rachel Rudge, Marketing Production Controller, Cambridge ESOL John Savage, Editorial Assistant, Cambridge ESOL Printed in the United Kingdom by Océ (UK) Ltd. CAMBRIDGE ESOL : RESEARCH NOTES : issue 50 / november 2012 1 r esearch notes Contents Guest editorial 2 Jayanti Banerjee Applying a model for investigating the impact of language assessment within educational contexts: The Cambridge esoL approach 4 Nick Saville An investigation into the effect of intensive language provision and external assessment in primary education in Ho Chi minh City, v ietnam 8 Hanan Khalifa, Thuyanh Nguyen and Christine Walker The Hebei impact Project: A study into the impact of Cambridge english exams in the state sector in Hebei province, China 20 Lucy Chambers, Mark Elliott and Hou Jianguo An initial investigation of the introduction of Cambridge english examinations in mission laïque française schools 24 Angeliki Salamoura, Miranda Hamilton and Viviane Octor The beDA impact project: A preliminary investigation of a bilingual programme in spain 34 Karen Ashton, Angeliki Salamoura and Emilio Diaz A small-scale pilot study investigating the impact of Cambridge english: Young Learners in China 42 Xiangdong Gu, Hanan Khalifa, Qiaozhen Yan and Jie Tian impact of Cambridge english: Key for schools and Preliminary for schools – parents’ perspectives in China 48 Xiangdong Gu and Nick Saville Editorial notes Welcome to issue 50 of Research Notes, our quarterly publication reporting on matters relating to research, test development and validation within University of Cambridge ESOL Examinations (Cambridge ESOL). The theme of this issue is the impact of Cambridge English exams in a variety of contexts. The issue benefits from the guest editorship of Dr Jayanti Banerjee, Research Director at Cambridge Michigan Language Assessments. Following Dr Banerjee’s guest editorial, Nick Saville outlines Cambridge ESOL’s approach to investigating the impact of its exams, and the following six studies represent different aspects of this approach. The first two articles describe studies that are investigating the impact of Cambridge English exams as part of larger educational reform initiatives. Hanan Khalifa, Thuyanh Nguyen and Christine Walker describe the first phase of a study investigating the effect of introducing Cambridge English: Young Learners into an intensive English programme in Ho Chi Minh City, Vietnam while Lucy Chambers, Mark Elliott and Hou Jianguo’s study investigates the impact of using Cambridge English exams in a pilot programme in Hebei province in China. The next pair of articles are baseline studies. The first by Angeliki Salamoura, Miranda Hamilton and Viviane Octor explores the anticipated effects of introducing Cambridge English exams in the Mission laïque française schools, an international association of schools teaching the French curriculum. The next article by Karen Ashton, Angeliki Salamoura and Emilio Diaz describes a preliminary investigation into the impact on stakeholders of a bilingual programme developed by a federation of Spanish religious schools in Madrid. The last two articles focus on stakeholder perceptions of Cambridge English exams in China. Xiangdong Gu, Hanan Khalifa, Qiaozhen Yan and Jie Tian describe a pilot study investigating Cambridge English: Young Learners exams in China. The last article by Xiangdong Gu and Nick Saville focuses on parents’ attitudes and perceptions of Cambridge English for Schools exams. © UCLES 2012 – The contents of this publication may not be reproduced without the written permission of the copyright holder.2 CAMBRIDGE ESOL : RESEARCH NOTES : issue 50 / november 2012 Guest editorial JAYAnTi bAnerJee CAMBRIDGE MICHIGAN LANGUAGE ASSESSMENTS, USA In the almost two decades since Alderson and Wall (1993) examinations. Individually and together they provide insights asked the question ‘does washback exist?’, there has been into the effect of the examinations within different educational a growing body of research confirming not only that it does contexts, whether they are government-initiated reforms, exist but also that it is a multi-faceted phenomenon. Language language learning initiatives taken by chains of independent tests and examinations have a complex effect upon the schools, or the result of national education policy. attitudes, beliefs, motivation, and actions of language learners The issue begins with an overview of Cambridge ESOL’s and teachers as well as upon the broader educational context approach towards the investigation of impact in language and upon society as a whole. Consequently, the field has assessment. Saville shows how the organisation’s early moved away from the very early assumptions that tests would model of test impact has evolved into a meta-framework inevitably have negative effects (see, for example, Kirkland entitled ‘impact by design’ (Saville 2009) whereby tests are 1971, Madaus 1988) towards a more modulated view. It is designed to promote and encourage positive impact. Key now agreed that tests can be instruments of beneficial change within this framework is an appreciation of context and the (see, for example, Pearson 1988, Swain 1984) but that this interaction between the different layers (sub-contexts) within cannot be guaranteed simply by designing a good test. For the a society, for the nature and the degree of influence of an nature and strength of the effect that a test has upon teaching, exam can vary depending on the local or national context. learning, and the wider social context, is in turn dependent Additionally, echoing Wall (2005), the framework calls for upon that cultural and educational context. impact to be regularly monitored. Test developers should seek Numerous studies have catalogued the areas of resistance to achieve the intended impact of the exam and to predict that slow or block the effect of a test within the teaching and unintended, negative consequences (what Saville collectively learning micro context. For instance, Alderson and Hamp- terms ‘anticipated impact’). The latter should be ameliorated Lyons (1996) and Watanabe (1996) report that teachers may through the test review and design process. change the way that they teach when preparing students The papers that follow embody this approach, identifying for an examination but that the methodology adopted varies the ‘anticipated impacts’ of different Cambridge English from teacher to teacher, suggesting that it is not the test itself examinations. All the studies employ mixed methods but their beliefs about the test that influence the teaching designs (see Creswell and Plano Clark 2011), combining activities that are used in class. Cheng (2005) shows how thematic analyses of focus groups and interviews with the the structure of the educational system may constrain the statistical analyses of questionnaires and test performances. degree to which teachers are able to adapt their teaching Most of the studies draw on a set of core data collection methodology to a new test. Stoneman (2006) finds that instruments, allowing (in the future) for useful cross-context the commitment of students to language learning and test analyses. Many of the studies also exemplify the benefits of preparation is influenced by their perception of the status of collaborations between Cambridge ESOL-based researchers that exam. An exam with little perceived status or usefulness and researchers with local knowledge who provide an is less likely to effect changes upon the students’ approach to understanding of and insights into the specific local context language learning or their test preparation. As Wall (2005) being studied. explains, the effect of a test upon teaching and learning needs The papers by Khalifa, Nguyen and Walker (this issue) to be understood within a much broader framework. and Chambers, Elliott and Jianguo (this issue) are studies This calls for investigations of the macro context such of carefully targeted government-initiated reform. Khalifa as Saville’s (2009) meta-analysis of three case studies of et al investigate the impact of the Cambridge English: Young test impact: the International English Language Testing System Learners (YLE) examinations within Ho Chi Minh City (HCM) (IELTS) impact study, the Italian Progetto Lingue 2000, and the in Vietnam as part of an intensive English programme (IEP). Florence Language Learning Gains Project (FLLGP). While This context is particularly interesting because the IEP is not up to that point few authors had explicitly distinguished mandatory for all HCM schools and access to the programme between the terms washback and impact, often using them is by selection. Chambers et al explore the effect of a pilot interchangeably (see Cheng, Watanabe and Curtis 2004), programme to introduce Cambridge English: Key (KET) for Saville (2009) firmly establishes the usefulness of the Schools and Cambridge English: Preliminary (PET) for Schools to distinction presented by Wall (1997:291): that ‘washback’ primary and junior high schools in Hebei province in China. refers specifically (and narrowly) to the effects of tests upon For both these government reforms the teachers are carefully teaching and learning while ‘impact’ refers to the effects that selected and trained and, as a result, are highly educated. a test can have upon both the micro context of the classroom Additionally, in the case of the IEP programme in Vietnam, and the macro context of the school, educational system, and the uptake among the students is much higher among wider society. children with educated parents who hope that their children This issue of Research Notes focuses upon several will become internationally mobile in the future. This results investigations into the impact of Cambridge English © UCLES 2012 – The contents of this publication may not be reproduced without the written permission of the copyright holder. CAMBRIDGE ESOL : RESEARCH NOTES : issue 50 / november 2012 3 in somewhat rarefied groups of informants, an interesting References finding in itself. Alderson, J C and Hamp-Lyons, L (1996) TOEFL preparation courses: A The papers by Salamoura, Hamilton and Octor (this issue), study of washback, Language Testing 13 (3), 280–297. and by Ashton, Salamoura and Diaz (this issue) are pilot or Alderson, J C and Wall, D (1993) Does washback exist?, Applied baseline studies that investigate the impact of introducing Linguistics 14 (2), 115–129. Cambridge English examinations within chains of independent Cheng, L (2005) Changing Language Teaching through Language Testing: schools. One context, the Mission laïque française (Mlf), is A Washback Study, Studies in Language Testing volume 21, Cambridge: a global association of schools based in 46 countries while UCLES/Cambridge University Press. the other, a federation of religious schools based within the Cheng, L, Watanabe, Y and Curtis, A (Eds) (2004) Washback in autonomous community of Madrid (FERE), is much more Language Testing: Research Contexts and Methods, Mahwah, NJ: Lawrence Erlbaum Associates. geographically circumscribed. The final papers in this issue present small-scale studies Cresswell, J W and Plano Clark, V L (2011) Designing and Conducting Mixed Methods Research (2nd Edition), Thousand Oaks, CA: Sage. led by Professor Xiangdong Gu and explore the impact of Cambridge English examinations in China as a result of Kirkland, M C (1971) The effects of tests on students and schools, Review of Educational Research 41 (4), 303–350. changes in the national education policy. The paper by Gu, Khalifa, Yan and Tian (this issue) reports on the pilot phase Madaus, G F (1988) The influence of testing on the curriculum, in Tanner, L N (Ed.) Critical Issues in Curriculum: Eighty-seventh Yearbook of a project investigating the impact of the Cambridge English: of the National Society for the Study of Education, Chicago, IL: University Young Learners examinations in China. Focusing on a large of Chicago Press, 83–121. private language teaching institute in Chongqing, Gu et al Pearson, I (1988) Tests as levers for change, in Chamberlain, D and report that the Cambridge English: Young Learners exams are Baumgartner, R (Eds), ESP in the Classroom: Practice and Evaluation, having a positive effect upon teaching and learning. The ELT Documents volume 128, London: Modern English Publications, paper by Gu and Saville (this issue) focuses on parents as 98–107. key stakeholders in the introduction of Cambridge English: Key Saville, N (2009) Developing a Model for Investigating the Impact for Schools and Cambridge English: Preliminary for Schools. Like of Language Assessments within Educational Contexts by a Public Khalifa et al (this issue), Gu and Saville find that the majority Examination Provider, unpublished PhD thesis, University of Bedfordshire. of the parents are well educated and actively involved in their children’s language learning. Stoneman, B (2006) The Impact of an Exit English Test on Hong Kong Undergraduates: A Study Investigating the Effects of Test Status on Each of these preliminary studies into the impact of the Students’ Test Preparation Behaviours, unpublished PhD thesis, Hong Cambridge English examinations indicates the positive role Kong Polytechnic University. that the examinations are playing in these different contexts. Swain, M (1984) Large-scale communicative testing: A case study, in They also demonstrate the benefits of an iterative approach Savignon, S J and Berns, M (Eds) Initiatives in Communicative Language to gathering impact data as they have collectively revealed Teaching, Reading, MA: Addison-Wesley, 185–201. areas where further support and information is needed for Wall, D (1997) Impact and washback, in Clapham, C and Corson, D teachers and parents. The follow-up studies that are planned (Eds) Encyclopaedia of Language and Education, Language Testing and will no doubt explore in much greater depth the opportunities Assessment volume 7, Dortrecht, The Netherlands: Kluwer, 291–302. and challenges of introducing examinations into such different Wall, D (2005) The Impact of High-Stakes Examinations on Classroom cultural and educational contexts, the different points of Teaching: A Case Study Using Insights from Testing and Innovation Theory, Studies in Language Testing volume 22, Cambridge: UCLES/ resistance as well as the different ways in which the same Cambridge University Press. information is interpreted and operationalised. Watanabe, Y (1996) Does grammar-translation come from the entrance examination?, Language Testing 13 (3), 319–333. © UCLES 2012 – The contents of this publication may not be reproduced without the written permission of the copyright holder.4 CAMBRIDGE ESOL : RESEARCH NOTES : issue 50 / november 2012 Applying a model for investigating the impact of language assessment within educational contexts: The Cambridge ESOL approach niCK sAviLLe RESEARCH AND VALIDATION GROUP , CAMBRIDGE ESOL of test impact which was explicitly designed to meet the Introduction needs of Cambridge ESOL. They proposed four maxims as In Research Notes 42 (2010), I explained why Cambridge follows: ESOL as an international test provider needs a model to guide its work in investigating the impact of its examinations. In Maxim 1 PLAN this article I set out some features of the model now being Use a rational and explicit approach to test development developed and explain how it can be applied in the case of the Cambridge English examinations. The operational practices Maxim 2 SUPPORT needed to implement this approach are being introduced Support stakeholders in the testing process incrementally and are being adapted and revised in light of Maxim 3 COMMUNICATE experiences in conducting projects which are now underway Provide comprehensive, useful and transparent information in many parts of the world. Maxim 4 MONITOR and EVALUATE Collect all relevant data and analyse as required Impact research within Cambridge ESOL These maxims were designed to capture key principles and Impact research investigates and seeks to understand to provide a basis for practical decision-making and action the effects and consequences which result from the use planning – and they still remain central to the Cambridge ESOL approach today (see Section 4.4 in Cambridge ESOL’s of tests and examinations in educational contexts and Principles of Good Practice: Quality Management and Validation throughout society. As a field of enquiry it appeared in the language testing literature as an extension of washback in in Language Assessment (2011)). the 1990s. (See Cheng, Watanabe and Curtis 2004 for an Under Maxim 1, Cambridge ESOL endeavours to develop overview of washback.) The PhD theses of Wall (2005), systems and processes to plan effectively using a rational and Cheng (1997, 2005) and Green (2007) published in the explicit model for managing the test development processes Studies in Language Testing series, looked at different aspects in a cyclical and iterative way. It requires regular reviews and revisions to take place and for improvements to be of washback and extended the earlier work of Hughes made when necessary (Cambridge ESOL 2011:18–22, Saville (1989) and Bailey (1996). While these studies inevitably touched on considerations related to impact, none proposed 2003:57–120). a comprehensive model which would allow complex Maxim 2 focuses on the requirement to support all the relationships to be examined across wider educational and stakeholders involved in the processes associated with societal contexts. This has been the aim of the team working international examinations. This is an important aspect of in Cambridge ESOL. the approach because examination systems only function effectively if all stakeholders collaborate to achieve the The origin of the Cambridge ESOL approach dates back intended outcomes. to the early 1990s and to the time when the current test development and validation strategies were first introduced. Maxim 3 focuses on the importance of developing In those early stages, Bachman’s work was influential as he appropriate communication systems and of providing was the first to present impact as a ‘quality’ of a test which essential information to the stakeholders (Cambridge ESOL should be integrated within the overarching concept of test 2011:12–14). Maxim 4 focuses on the essential research requirement usefulness (Bachman and Palmer 1996). Following his lead, to collect as much relevant data as possible and to carry Cambridge ESOL also introduced impact as one of the four out routine analyses as part of the iterative model (noted essential qualities, which together with validity, reliability, and practicality comprise the VRIP features of a test (Saville under Maxim 1). The nature of the data needed to investigate 2003:65). impact effectively and how it can be collected, analysed and By conceptualising impact within VRIP-based validation interpreted under operational conditions has become an processes from the start, there was an explicit attempt increasingly important part of the model in recent years. Three major impact studies were also carried out between to integrate impact research into routine procedures for 1995 and 2004. Project 1 was the survey of the impact of accumulating validity evidence. Subsequent work on impact IELTS (International English Language Testing System). This has been framed by these considerations and since the initial stage it has been recognised that a proactive approach is project helped conceptualise impact research including the needed to achieve intended effects and consequences. design and validation of suitable instruments. Project 2 was In 1996, Milanovic and Saville proposed an early model the Italian Progetto Lingue 2000 impact study and was an © UCLES 2012 – The contents of this publication may not be reproduced without the written permission of the copyright holder.i c r M o c o n t e x t a n d c u l t u r e CAMBRIDGE ESOL : RESEARCH NOTES : issue 50 / november 2012 5 application of the approach within a single macro educational • test features (constructs and delivery systems) context. These two projects are described in detail by Hawkey • contexts (2006). Project 3 was the Florence Language Learning Gains • outcomes over time – the timeline Project (FLLGP). Still within Italy, this project was an extension • research methods and roles of researchers. and re-application of the model within a single school context (i.e. at the micro level). It focused on individual stakeholders Test features (constructs and delivery systems) in one language teaching institution, namely teachers Impact by design builds on Messick’s (1996) idea of achieving and learners preparing for a range of English language ‘validity by design as a basis for washback’. The importance of examinations at a prestigious language school in Florence. the rational model of test development and validation with its The complex relationships between assessment and learning/ iterative cycles is a necessary condition for creating construct teaching in a number of language classrooms, including valid tests and for the development of successful systems the influence of the Cambridge English examinations, were to support them (cf. Maxim 1). Adequate specification examined against the wider educational and societal milieu and communication of the focal constructs is crucial for in Italy. The micro level of detail, as well as the longitudinal ensuring that the test is appropriate for its purpose and nature of the project conducted over an academic year, were contexts of use and to counter threats to validity: construct particularly relevant in this case (Saville 2009). underrepresentation and construct irrelevant variance Based on an analysis of these projects, I have proposed a (Messick 1996:252). meta-framework designed to provide a more effective model Insights from socio-cognitive theory underpin for conducting impact research under operational conditions contemporary theories of communicative language ability, (Saville 2009). I suggest that by implementing this framework language acquisition and assessment (cf. the socio-cognitive more systematically, ‘anticipated impacts’ can be achieved model (Cambridge ESOL 2011:25–27, Weir 2005)) and are more effectively and well-motivated improvements to the also helpful in understanding how language learning and examination systems can be identified and put into place. preparation for examinations takes place in formalised Aspects of this approach are represented in the impact learning contexts, such as classrooms. studies reported in this issue and are focused on in the second While appropriate construct representation is a necessary part of this paper under the concept of impact by design. condition for achieving the anticipated outcomes, it is not sufficient and impact by design highlights the importance of designing and implementing assessment systems which The concept of impact by design explicitly incorporate considerations related to the social and educational contexts of learning/teaching and test use. Impact by design is a key feature of the expanded impact This relates to the need for effective communication and model. It starts from the premise that assessment systems collaboration with stakeholders, as noted in the original should be designed from the outset with the potential to Maxims 2 and 3 and incorporated into the Principles of Good achieve positive impacts and takes an ex ante approach to Practice, Section 2 (Cambridge ESOL 2011). anticipating the possible consequences of using the test in particular contexts. Contexts In the final part of this paper, the following four points which Understanding the nature of context within educational are central to the model are discussed: systems and the roles of stakeholders in those contexts Figure 1: Context in education – a complex dynamic system MACRO Individual differences CONTEXT • Demographic • Socio-Psychological Country • Strategic • Culture • Prior knowledge/learning • Politics • L1 • Role of L2 • Model of L2 Learner Region and • Urban/rural Teacher • Wealthy/poor Community • Demographic make-up Group School Sector Class • Public/private Cycle • Primary • Middle • Upper © UCLES 2012 – The contents of this publication may not be reproduced without the written permission of the copyright holder. I n c r e a s i n g v a r i a t i o n6 CAMBRIDGE ESOL : RESEARCH NOTES : issue 50 / november 2012 are clearly important considerations for Cambridge ESOL • the educational system and how it is organised (e.g. – see Saville (2003:60) for a discussion of stakeholders. It compulsory education and the nature of the educational is now widely recognised that educational processes (see cycles; private vs. public schools; role of standardised Figure 1) take place within complex systems with dynamical assessment, etc.) interplay between many sub-systems and ‘cultures’ and so an • broad differences between geographical regions and socio- understanding of the roles of stakeholders as participants is a economic groups. critical factor in bringing about intended changes (e.g. Fullan Collaboration between an international examination 1993, 1999, Thelen and Smith 1994, Van Geert 2007). provider and local users is essential in order to capture In conducting impact research the aim is to understand relevant data and to shed light on such contextual parameters. better the interplay between the macro and micro contexts Many dilemmas which arise in assessment contexts can only within the society where the tests are being used and to be dealt with if a wide range of local stakeholders agree to determine which elements facilitate or hinder the desired manage them in ways which they jointly find acceptable; the outcomes. In general, diversity and variation increases as challenge is to get the relevant stakeholders working together one moves from the general milieu within a country or region effectively to agree what needs to be done to achieve the (the macro context) to specific schools and ultimately to the intended outcomes. individual participants within classrooms (the multiple micro contexts at the local level involving schools, classes/groups outcomes over time – the timeline and individual teachers and learners). It is essential to know what happens when a test is introduced Figure 2 diagrammatically shows a school context into its intended contexts of use and this should constitute a embedded in a wider milieu with a teacher interacting with long-term validation plan (cf. Maxims 1 and 4). Anticipating groups of learners in a particular classroom. The external and managing change over time within specific contexts influences include the general features of the milieu, as well as is therefore central to this concept and it means that specific educational factors such the curriculum and syllabus appropriate consideration of timescales and the timeline for and the need to produce examination results which are used implementation (often involving several phases) are central to outside of the school context. the design of impact studies. In impact research designs there It is therefore important to develop methods to understand is nearly always a fundamental need to collect comparative both the general context as well as specific local cases, data, and therefore to develop research designs which can be including dynamics which affect learning in classrooms. This carried out in several phases over an extended period of time points to the need to use both quantitative and qualitative or replicated in several different contexts. data collection methods (see below). Similarly, effects and consequences – intended and In understanding the macro contexts into which unintended – usually emerge over time given that contexts of international examinations are introduced (e.g. as part of use are not uniform and are subject to change, e.g. as a result educational reforms or innovations), it is important to focus of localised socio-political and other factors. Impact by design on key factors related to the following: is therefore not strictly about prediction; a more appropriate • the political regime and its approach to educational reforms term might be ‘anticipation’. In working with stakeholders, • the role of educational reforms within wider socio- possible impacts on both micro and macro levels can be economic policies anticipated as part of the design and development process, • cultural norms and expectations in relation to education and where potentially negative consequences are anticipated, remedial actions or mitigations can be planned well in generally, and attitudes towards language education (and advance. towards English specifically in the case of Cambridge ESOL) Figure 2: multiple classroom contexts where teaching and learning take place School (Type) Curriculum Class (syllabus e.g. CEFR level) Wider Embedded context Sub-group in class contexts: Country Content Region Town External Influences School district Exam Exam performance results A school with classes and groups Teacher A Learner 2 etc. Learner 1 – individual – individual – individual characteristics characteristics characteristics inc L1 © UCLES 2012 – The contents of this publication may not be reproduced without the written permission of the copyright holder. CAMBRIDGE ESOL : RESEARCH NOTES : issue 50 / november 2012 7 r esearch methods and roles of researchers Conclusion Contemporary theories of knowledge and learning have The ability to change in order to improve educational played a prominent role in developing Cambridge ESOL’s outcomes or mitigate negative consequences associated with model of impact and the search for a ‘paradigm worldview’ the examinations is ultimately the most important dimension (epistemology and ontology) which provides an effective of the impact by design model. Anticipating impacts and conceptualisation and has drawn on relevant theories in the finding out what happens in practice are not enough if social sciences. A ‘realist’ stance now underpins Cambridge improvements do not occur as a result. Being prepared to ESOL’s approach, drawing on ‘critical realism’ (e.g. Sayer 1984, manage change is therefore critical to a theory of action. In 2000) and contemporary views on pragmatism. working closely with the stakeholders in their own contexts, Constructivism is also important for the re-conceptualisation this approach is now providing us with the necessary tools to of impact for two reasons: first because contemporary determine what needs to be done and when/how to do it. approaches to teaching and learning in formal contexts now appeal to constructivist theories; secondly, because it is most appropriate to finding out ‘what goes on’ in contexts of test use. From the learner’s perspective, affective factors are References and further reading vital for motivation and feedback that highlights strengths Bachman, L and Palmer, A (1996) Language Testing in Practice, positively tends to lead to better learning (i.e. learning Cambridge: Cambridge University Press. oriented assessment). These considerations are relevant in Bailey, K M (1996) Working for washback: A review of the washback designing language assessment systems which have learning concept in language testing, Language Testing 13 (3), 257–279. oriented objectives and a concern in impact research is Cambridge ESOL (2011) Principles of Good Practice: Quality Management and Validation in Language Assessment, available online: www. whether these objectives have been met effectively. The current model of impact looks to ‘real world’ research paradigms to provide tools which can shed light on what Cheng, L (1997) The Washback Effect of Public Examination Change on Classroom Teaching: An Impact Study of the 1996 Hong Kong Certificate happens in testing contexts, including mixed methods and of Education in English on the Classroom Teaching of English in Hong Kong quasi-experimental designs. Case studies are especially Secondary Schools, unpublished PhD thesis, University of Hong Kong. useful for investigating impact at the micro level and for Cheng, L (2005) Changing Language Teaching through Language Testing: A understanding the complexities of interaction between macro Washback Study, Studies in Language Testing volume 21, Cambridge: level policies and implementation in local settings. Without UCLES/Cambridge University Press. such methods it is difficult to find out about and understand Cheng, L and Watanabe, Y with Curtis, A (Eds) (2004) Washback how the interaction of differing beliefs and attitudes can lead in Language Testing: Research Contexts and Methods, Mahwah, NJ: to consensus or to divergence and diversity. Lawrence Erlbaum Associates. Mixed method research designs are becoming increasingly Creswell, J W and Plano Clark, V L (2011) Designing and Conducting relevant to addressing impact research questions. Creswell Mixed Methods Research (2nd Edition), Thousand Oaks, CA: Sage. and Plano Clark (2011:69) discuss six prototypical versions Fullan, M (1991) The New Meaning of Educational Change (2nd Edition), of mixed method research designs which seek to integrate London: Cassell. qualitative and quantitative data in parallel and sequential Fullan, M (1993) Change Forces: Probing the Depths of Educational Reform, ways and these are becoming central to the Cambridge ESOL London: The Falmer Press. approach, as illustrated by the studies reported in this issue. Fullan, M (1999) Change Forces: The Sequel, London: The Falmer Press. The Cambridge ESOL ‘impact toolkit’ of methods and Green, A (2003) Test Impact and EAP: A Comparative Sudy in Backwash approaches is now being used to carry out analyses of both Between IELTS Preparation and University Pre-sessional Courses, unpublished PhD thesis, University of Surrey. large-scale aggregated data, as well as micro analyses of views, attitudes and behaviours in local settings (as in the Green, A (2007) IELTS Washback in Context: Preparation for Academic earlier case of the Progetto Lingue 2000 impact study reported Writing in Higher Education, Studies in Language Testing volume 25, Cambridge: UCLES/Cambridge University Press. by Hawkey (2006)). Quantitative analysis of macro level Hawkey, R (2006) Impact Theory and Practice: Studies of the IELTS Test group data allows us to capture overall patterns and trends, and Progetto Lingue 2000, Studies in Language Testing volume 24, while the qualitative analysis of multiple single cases enables Cambridge: UCLES/Cambridge University Press. the research team to monitor variability in local settings Hughes, A (1989) Testing for Language Teachers, Cambridge: Cambridge and to work with the ‘ecological’ features of context. It is University Press. the integration of both analyses to provide the insights and Messick, S (1996) Validity and washback in language testing, Language interpretations which is particularly important. Testing 13 (3), 241–256. Finally it is important to highlight the make-up of the impact Milanovic, M and Saville, N (1996) Considering the Impact of Cambridge research teams; where possible, the team should comprise EFL Examinations, internal report, Cambridge: Cambridge ESOL. both Cambridge-based ESOL staff with appropriate skills in Saville, N (2003) The process of test development and revision within research design and analysis, as well as local researchers who UCLES EFL, in Weir, C and Milanovic, M (2003) (Eds) Continuity and may be ‘participants’ in the teaching/learning context itself Innovation: Revising the Cambridge Proficiency in English Examination and who bring a deeper understanding of the educational 1913–2002, Studies in Language Testing volume 15, Cambridge: and cultural context which is under investigation. Again this UCLES/Cambridge University Press, 57–120. is illustrated in the studies reported in this issue, including Saville, N (2009) Developing a Model for Investigating the Impact Gu and Saville working jointly with other participants in the of Language Assessment within Educational Contexts by a Public Examination Provider, unpublished PhD thesis, University of Chinese context. Bedfordshire. © UCLES 2012 – The contents of this publication may not be reproduced without the written permission of the copyright holder.8 CAMBRIDGE ESOL : RESEARCH NOTES : issue 50 / november 2012 Saville, N (2010) Developing a model for investigating the impact of Wall, D (1999) The Impact of High-stakes Examinations on Classroom language assessment, Research Notes 42, 2–8. Teaching: A Case Study Using Insights From Testing and Innovation Theory, unpublished PhD thesis, Lancaster University. Sayer, A (1984) Method in Social Science: A Realist Approach, London: Routledge. Wall, D (2005) The Impact of High-Stakes Examinations on Classroom Teaching: A Case Study Using Insights from Testing and Innovation Sayer, A (2000) Realism and Social Science, London: Sage. Theory, Studies in Language Testing volume 22, Cambridge: UCLES/ Thelen, E and Smith, L B (1994) A Dynamic Systems Approach to the Cambridge University Press. Development of Cognition and Action, Cambridge, MA: The MIT Press. Weir, C J (2005) Language Testing and Validation: An Evidence-based Van Geert, P (2007) Dynamic systems in second language learning: Approach, Basingstoke: Palgrave Macmillan. Some general methodological reflections, Bilingualism: Language and Cognition 10, 47–49. An investigation into the effect of intensive language provision and external assessment in primary education in Ho Chi Minh City, Vietnam HAnAn KHALiFA RESEARCH AND V ALIDATION, CAMBRIDGE ESOL THuYAnH nGuYen DEPARTMENT OF EDUCATION AND TRAINING, HO CHI MINH, VIETNAM CHrisTine WALKer RESEARCH AND V ALIDATION, CAMBRIDGE ESOL Acknowledgement to be a pioneering initiative within the Vietnamese context. The Intensive English Programme (IEP), one of HCM DOET’s The authors would like to express their gratitude to Ms Uyen Pham initiatives, started in 1998–99 with one school but by 2011–12, and Ms Bui Thi Phuong Lien who facilitated the data collection and a total of 194 schools out of 495 state-funded primary schools data entry process. Without their support, diligence and patience, had joined IEP. The other initiative was the introduction of a this research study would not have taken place. standardised external assessment in 2010–11. HCm DoeT intervention Context The Intensive English Programme Education has always had a central role in Vietnamese culture and In 1998–99, the DOET in HCM initiated the provision of an society. It is seen as the avenue of advancement and families routinely Intensive English Programme for primary students in state- sacrifice much to ensure their children have the required education. The funded schools whereby students are given additional English Vietnamese government has for some time given priority to education in lessons. State schools typically operate on a half-day basis for terms of its budget. Currently, education occupies approximately 20% of students due to high demand, insufficient classroom capacity all state budget expenditures and accounts for 5.5% of GDP (Department and high class density. English lessons, like other lessons, are of Finance and Planning, Ministry of Education and Training 2008 as cited taught in two lessons per week (35 minutes/lesson). However, in Runckel 2008). IEP schools offer a whole-day programme to students which The Ministry of Education and Training (MOET) in Vietnam allows for an additional eight English lessons per week. has a long-standing strategic objective to raise English HCM DOET’s drive for additional classes did not only stem language learning standards by 2020 so that students are from the desire to meet MOET’s strategic objective but also better prepared for the workplace, for studying abroad and for out of a sense of social responsibility. Until the introduction becoming global citizens. The Common European Framework of IEP, only students from financially able families had the of Reference for languages (CEFR) is used to indicate target opportunity of increasing their English proficiency through levels set for the primary stage (CEFR A1 level), junior high attending private language institutes. With the IEP initiative, school (A2), senior high school (B1), university students with it is hoped that all students have the chance to increase their non-English majors (B2) and university students with English English proficiency at affordable fees without the need to majors (C1). go to private institutes or tutors where fees are exorbitant This paper focuses on the implementation of the for families with average incomes. (The GDP per capita per Ministry’s strategic objective by the Department of annum is estimated at 3,400 (CIA – The World Factbook Education and Training (DOET) in Ho Chi Minh City (HCM) 2011)). Students can opt in or out of IEP. If students opt out, and the intended/unintended effects of the implemented they are offered another less intensive programme, referred to interventions. HCM was selected for the study given that it as a selective programme (four additional lessons of English is the largest city in Vietnam in terms of size and population per week) or they can choose the standard programme, which and the fact that HCM DOET interventions in the learning consists of two English lessons per week. and teaching of English to young learners are considered IEP is not mandatory for HCM schools. However, schools © UCLES 2012 – The contents of this publication may not be reproduced without the written permission of the copyright holder. CAMBRIDGE ESOL : RESEARCH NOTES : issue 50 / november 2012 9 who wish to offer or continue offering IEP must adhere English: Young Learners was selected not only because of to guidelines set by HCM DOET in terms of physical its international recognition and use in similar contexts but requirements such as classroom size (maximum 35 students more importantly because it introduces children to everyday per class), layout and suitable chairs as well as resources such written and spoken English in a fun and motivating way. as teaching materials, aids and realia. Where needed, HCM According to a key decision maker in HCM DOET, Mr Nguyen DOET provides support in terms of processes and procedures, Hoai Chuong, DOET Vice Director, Cambridge English: Young teacher training courses, support materials such as book lists Learners ‘is child friendly, takes into account child psychology, for reading, establishing reading circles, provision of lesson is very motivational and covers all skills . . . if the exam is plans, and as of 2010 aligning the curriculum to Cambridge child friendly and encourages learning in a fun way, then the English: Young Learners (YLE) exams and designing textbooks teaching will change accordingly, so it is a win-win situation’ to complement these exams. According to Mr Le Ngoc Diep, (personal communication, March 2012). the Primary Education Division Manager in HCM DOET and HCM DOET decided that students finishing Grade 2 (age one of the initiators of IEP: 7–8) would need to take Cambridge English: Starters, Grade 4 (age 9–10) to take Cambridge English: Movers and Grade 5 the IEP curriculum developed organically; support materials were given (age 10–11) to take Cambridge English: Flyers. This decision was to schools, criteria for joining IEP were formulated and standardised. The based on local expert judgement rather than local empirical initial apprehension of introducing a foreign language at an early age evidence. It was also based on a belief that the earlier a and its potential negative effect started to diffuse within the first year second language (L2) is introduced in school learning years of IEP after close inspection of results, weekly meetings among decision the better the grasp of it. In 2010–11, DOET decided that the makers, regular school visits and classroom observations (personal minimum number of shields required by students to continue communication, March 2012). in IEP would be 10. However, in 2011–12, post discussion with IEP schools are committed to further develop their teaching Cambridge ESOL during the conduct of this study and given staff and to seek support from the local community. For the motivational nature of Cambridge English: Young Learners, example, some schools arrange a flexible schedule for their DOET decided to use the number of shields received not for teachers so that they are able to enhance their language gate keeping purposes but to place students into levels within proficiency via preparing for a B2-level test, i.e. Cambridge Grade 3 for homogeneity purposes (DOET document 1355/ English: First (FCE). Since 2010, teachers wishing to teach in GDĐT-TH dated 28 May 2012). IEP must pass a three-step recruitment process: 1. candidates Schools which offer Cambridge English: Young Learners are short-listed based on professional qualifications (e.g. have exams have organised awareness-raising events for parents to obtained at least a BA in English Language and Literature or explain the rationale behind introducing external assessment in English language teaching and methodology), 2. candidates and to familiarise parents with the Cambridge English: Young take a written test and make a voice recording (to check Learners curriculum, learning objectives and outcomes. Some pronunciation) and 3. candidates are interviewed by a schools also have offered free test preparation courses for the native speaker. children. Other schools have invited qualified native speakers of English to teach once a week. Cambridge English: Young Learners As of 2010–11, HCM DOET introduced an external assessment as mandatory to IEP. There are two main reasons for this. One reason is the high demand on IEP which led Study purpose HCM DOET to need a fair and reliable measure for student Interventions are usually based on the expectation that ‘if’ selection and continuation purposes in IEP. The other a set of activities is undertaken, ‘then’ some set of changes reason is for accountability and quality assurance purposes. or improvements in the situation those activities address External assessment is used as a measure to evaluate the will occur. effectiveness of IEP in terms of students’ learning progression, Thus, two years into the introduction of Cambridge English: to benchmark the level of IEP students to an international Young Learners exams, Cambridge ESOL initiated a research standard and to monitor their progress over the years. Hence, study as part of its impact studies programme to look at the they chose Cambridge English: Young Learners examinations effect of this decision. This is in line with Cambridge ESOL’s (see Cambridge ESOL 2011). The tests have three proficiency concept of impact by design (Saville 2010) which is built on levels beginning with Cambridge English: Starters set at a the organisation’s four maxims for achieving and monitoring Pre-A1 level, followed by Cambridge English: Movers set at impact, namely, PLAN, SUPPORT, COMMUNICATE and CEFR A1 level and ending with Cambridge English: Flyers set MONITOR AND EVALUATE (Milanovic and Saville 1996). at CEFR A2 level. Each test level comprises three papers The findings of this study are intended to inform HCM covering the four language skills. The Cambridge English: DOET of notable changes in learner motivation and Starters Listening paper has four parts containing 20 questions progression as well as notable changes in teaching practices and candidates are given 20 minutes, the Speaking paper has as a result of the intervention. The study also would provide five parts taking between 3 and 5 minutes to complete, the DOET with an insight into stakeholders’ (see Figure 1) Reading and Writing paper has five parts with a total of 25 perceptions of IEP and of Cambridge English: Young Learners questions and lasting 20 minutes. They are designed to make exams. Such information would allow DOET to record success learning fun and children are encouraged by working towards stories, lessons learned and take subsequent actions whether certificates and earning shields that record their progress. A it is sustaining conditions for success or working on areas maximum of five shields is awarded per test paper. Cambridge which warrant improvements. © UCLES 2012 – The contents of this publication may not be reproduced without the written permission of the copyright holder.10 CAMBRIDGE ESOL : RESEARCH NOTES : issue 50 / november 2012 1 Figure 1: ieP and Cambridge English: Young Learners participant and stakeholder community SCHOOL SECTOR POLICY MAKING IMPLEMENTATION MOET DOETs DOETs SCHOOLS PRINCIPALS PRIMARY PRIMARY DISTRICT HEADS TEACHERS SECONDARY SECONDARY PARENTS ELT PROJECT CONTINUING STEERING STUDENTS EDUCATION COMMITTEE The study was conducted over a period of three months The selected sample comprised 24 schools (13 in a (March–June 2012) in collaboration with HCM DOET. It central district area, six in a semi-outskirts area, and five focused on Grade 2 given the interest of HCM DOET in in an outskirts area) for qualitative data collection. Focus tracking the performance of these young learners through their groups were conducted with 5–10 Grade 2 students in each primary school years. As such, this study is seen as Phase I school. Students’ age ranged between 7 and 8 years old. For of the investigation of the effectiveness of IEP and the use of quantitative data collection, the selected sample consisted of Cambridge English: Young Learners exams with the intention to 59 schools where survey data was gathered from 113 teachers start Phase II in 2013–14 when Grade 2 students are in Grade and 2,683 parents of Grade 2 students. The profiles of the 4 and expected to take Cambridge English: Movers. teacher and parent respondents are as follows: Teacher profile: • eLT experience: The highest percentage of respondents 2 Study design and methods (52%) had between four and 10 years of experience Key question followed by 42% who are considered novice teachers (1–3 years of experience) while the remaining 6% had 11 or more The study addressed one broad question: years’ experience. • What is the intended/unintended effect of HCM DOET’s • Academic qualifications: 81% are university graduates strategic decision to increase English language provision (4-year degree) and 19% have a college diploma/degree through IEP and to ensure the quality of the provision (3-year degree). Both degrees offer pedagogic training if through the use of external assessment, i.e. Cambridge students are trained to become English teachers. English: Young Learners? • Teaching qualifications: 52% of the respondents had a local r esearch sample qualification while the remaining 48% had an internationally HCM has 24 geographical districts subdivided as follows: recognised teaching qualification, namely, TKT (Teaching 11 central districts, five on the outskirts of the city, and Knowledge Test), CELTA (Certificate in Teaching English to eight districts referred to as semi-outskirts (see Appendix Speakers of Other Languages) or Delta (Diploma in Teaching 1 for typical characteristics of district areas). Within the 24 English to Speakers of Other Languages). The latter is an districts, there are 194 schools with 446 classes currently interesting fact. It reflects the importance teachers and their enrolled in IEP and Cambridge English: Young Learners. All employment institutions put on international certification. primary schools in HCM are mixed with a gender balance. • Geographical location of teacher schools: 52% of the Sample selection went through two stages. The first stage teachers work in schools located in a central district area, was a stratified random sample to select schools according 35% in a semi-outskirts area and 13% in an outskirts area. to geographical location and a minimum of two years’ Although this distribution is a result of the first stage of involvement in IEP and Cambridge English: Young Learners. The sampling, it is not surprising to find more schools in central second stage was random sample of classes within a single areas than in rural areas. school. 1  Figure supplied by Ms Uyen Pham, Cambridge ESOL Business Development Manager in Vietnam. 2  In conducting this study, ethical guidelines of the British Educational Research Association (2004) were followed. © UCLES 2012 – The contents of this publication may not be reproduced without the written permission of the copyright holder. CAMBRIDGE ESOL : RESEARCH NOTES : ISSUE 50 / NOVEMBER 2012 11 Parent profi le: Figure 2: Convergent parallel design procedural diagram Adapted from Creswell and Plano Clark (2011:118) • Academic qualifi cations: The majority of parent respondents (88%) are educated with 43% of them Quantitative data collection Qualitative data collection Procedures Procedures holding a university degree and 4% holding a postgraduate • Questionnaires • Open-ended comments in degree. This is quite interesting as Ermisch and Pronzato • Tests questionnaires • Semi-structured interviews/focus Products (2010) among other researchers have shown that parental groups • Numerical item scores education generates a positive correlation with children’s educational attainments. • Socio-economic status: This was a self-assessed category. Quantitative data analysis Qualitative data analysis Most parents (67%) stated that they are in the middle Procedures Procedures socio-economic stratum with 21% in the low/low-medium • Descriptive statistics • Thematic analysis • Group comparisons strata and 12% in the high/medium-high strata. This is an Products Products • Major themes important piece of information given that IEP is intended for • Frequency % families who are less fi nancially able. So it is interesting to • Mode note that according to the self-assessment, it is the more fi nancially able families who are taking advantage of IEP. Merge results & • Relationship to child: 71% of those who completed the provide interpretation survey were mothers, 27% were fathers and 2% were Procedure the grandparents. Although the majority of respondents Consider how merged results produce better understanding/ are mothers, it is interesting to note that fathers and confi rm fi ndings grandparents have also responded, which shows their Product Discussion involvement in the child’s education. • Geographical location of parent schools: 63% of the parents have children in a central district area, 23% in a implementation in fi eld work. In addition, minor amendments semi-outskirts area and 14% in an outskirts area. were made on the fi rst day of fi eld implementation to ensure reliable data collection and entry. Table 1 presents an overview We will return to teacher and parent profi les when of the key investigative points and demonstrates how discussing the results of this study. triangulation of data sources was achieved through a variety Research design of data types. Quantitative and qualitative data were simultaneously Qualitative data collection instruments collected in a mixed method research design (MMRD). The Qualitative data collection instruments comprised interviews analysis of each data strand was carried out independent of with policy makers, district heads, principals (or other school the other, but when interpreting the results information was leaders) and focus groups with students. The interviews and drawn from both strands. This approach enabled us to build a focus groups were conducted in both English and Vietnamese rich picture and the triangulation of information derived from and were audio recorded (with participants’ consent) as an multiple data sources enhanced our confi dence in the fi ndings aide memoire in addition to live note taking. The interviewer/ (see Greene, Caracelli and Graham 1989 for a discussion on moderator was supported by a local assistant throughout the reasons for mixing methods). This type of MMRD is referred face-to-face interviews and focus group discussions. to by Creswell and Plano Clark (2011) as a ‘convergent parallel design’ (see Figure 2 for an MMRD procedural diagram). One-to-one interviews with focal persons Instruments used in this study were selected from Focal persons are defi ned here as policy makers at the Cambridge ESOL’s ‘impact toolkit’ (see Saville’s article national level (i.e. MOET), and at the regional level (i.e. in this issue) and adapted for the Vietnamese context HCM DOET); as decision makers at the district level (i.e. where necessary using expert judgement reviews prior to Table 1: Overview of key investigative points and data type/sources Key investigative points Data type Data source 1. Attitude towards assessment, English learning, and Quantitative 1. Questionnaire to parents and teachers teaching Qualitative 2. Focus groups with young learners 3. Semi-structured interviews with focal persons 2. Learner motivation Quantitative 1. Questionnaire to parents and teachers Qualitative 2. Focus groups with young learners 3. Semi-structured interviews with focal persons 3. Learner progression Quantitative 1. Questionnaire to parents and teachers Qualitative 2. Focus groups with young learners 3. Test score data 4. Semi-structured interviews with focal persons 4. Changes in teaching practice Quantitative 1. Questionnaire to parents and teachers Qualitative 2. Focus groups with young learners 3. Semi-structured interviews with focal persons 5. Change in decision making Qualitative 1. Semi-structured interviews with focal persons © UCLES 2012 – The contents of this publication may not be reproduced without the written permission of the copyright holder. 8855 RN50.indd 11 8855 RN50.indd 11 09/11/2012 14:07 09/11/2012 14:0712 CAMBRIDGE ESOL : RESEARCH NOTES : issue 50 / november 2012 district vice heads in HCM) and implementers at the results are positive with nearly all statements having a mode school level (school principal, vice principal or head of of 3. Where the percentage of disagreement was 20% or English). The interviews served as a basis for gathering more, in-depth analysis was performed to check the influence contextual information, carrying out situational analysis, and of variables in teacher and parent profiles as well as school investigating perceived potential effects. Some of the data district area. gathered fed into the ‘Context’ section of this article while investigation point 1a: Attitudes towards assessment the remaining information is reported on in the ‘Results and discussion’ section. This section addressed attitudes towards assessing young learners in general and the use of Cambridge English: Young Student focus group Learners exams in particular. The focus group was designed to take into account the Parents’ perspective young age and cognitive development of the participating students (see Banks 2001, Capello 2005, Morrow and Eighty-seven per cent of parents view the tests to be the Richards 1996). The focus group was conducted in a way that most effective means of assessment, while 90% of them provided a detailed picture of learner motivation and language see continuous assessment as the most effective form of progression. Throughout the focus group, the facilitator assessment. In addition, 88% responded that it is important maintained patience, enthusiasm, understanding and for progress to be assessed using a variety of methods. organisation. She was able to build rapport with the children. Despite the fact that 92.5% of the parents are happy that Children were encouraged to speak freely and spontaneously the school has introduced Cambridge English tests, 41% on five identified topics: (a) why they have joined IEP, (b) of the parents expressed concern that the tests will bring reasons behind their desire to learn English, (c) incidences of additional work and pressure to their children. A typical using English in the classroom, (d) their views on Cambridge comment was: ‘The English programme at school includes so English: Starters, and (e) whether they feel their English has many things: the intensive programme, Starters, Cambridge, improved and why. . . . that it sometimes leads parents to confusion as they lack information of the efficiency of study’. This concern can be Quantitative data collection instruments partially explained by the fact that parents reported a lack Surveys and test score data were used in the quantitative of information from the schools about the introduction of analyses. The surveys were administered to teachers and the Cambridge English tests. Typical comments include: ‘the parents. They were positively worded, provided a 4-point school should provide more information about the Starters Likert scale for each statement and finished with an open- exam so that students can prepare for this exam as well as ended commentary section. The parents’ survey was to achieve the best result. I am looking forward to hearing translated into Vietnamese to ensure reliable data collection. feedback from the teachers’ and ‘we have not received any information on English in schools’. Teacher survey and parent survey Teachers’ perspective The teacher survey sought their views on tests in general and on Cambridge English: Young Learners exams in particular; their In general, the teachers were positive about assessment. At perceptions of the DOET intervention; and their expectations least 96% of the teachers see tests as important and as a tool of learner progression as a result of the intervention. The for them to understand students’ level and ability. Only 18% survey also aimed at gaining an insight into IEP classroom of the teachers worry about their students taking exams at a practices. The survey contained specific statements about young age. commonly used teaching practices described in ELT literature When asked specifically about Cambridge English: Young (Assessment Reform Group 2000, Brown 1993, Nunan Learners exams, 95% of the teachers were pleased with 1999). Similar views were also sought from parents who have their introduction into the school and found the topics enrolled their children in IEP. Parents were asked to express interesting. At least 80% of the respondents indicated that their opinion on the influence of IEP and Cambridge English: their students like the exams and receiving a Cambridge ESOL Young Learners exams on their child’s motivation to learn certificate. One of the teachers from Tan Binh district (central) English and their language learning progression. commented: ‘I find Cambridge English test interesting. It makes me excited in my English teaching. The students like Test score data doing test so much, they are very confident when they speak Cambridge English: Starters test results in consecutive English through colourful pictures’. academic years 2010 and 2011 were investigated to see However, 27% of the respondents perceived the selected whether standards of English have improved over time. To level of Cambridge English: Young Learners to be incompatible obtain comparative information, Cambridge English: Starters with the level of their students and 37% of the teachers test data from other contexts within Vietnam and from the stated that their students will not perform well on the tests. rest of the world was also examined. It is interesting to note that the higher levels of disagreement were by teachers from schools in central areas. Teachers’ experience or qualifications did not influence their comments or viewpoints. Results and discussion One teacher from a centrally located school commented The results have been summarised below according to the that the ‘Cambridge English test is rather difficult for Grade 2 investigation points identified in Table 1. Overall the survey students’. © UCLES 2012 – The contents of this publication may not be reproduced without the written permission of the copyright holder. CAMBRIDGE ESOL : RESEARCH NOTES : issue 50 / november 2012 13 Students’ perspective positive. Two key issues were raised, though. The first one is about parents’ view that they have not received adequate One of the questions in the focus group explored students’ information about Cambridge English: Starters and the second feelings about taking Cambridge English: Starters. The student one is about the suitability of Cambridge English: Starters for responses are grouped under five core themes. Grade 2 students, which was brought up by teachers. • Aectiv ff e: ‘I am not afraid of taking the test’ (outskirts Although parents complained about having insufficient district), ‘the test doesn’t scare me because the teacher information about the Cambridge English: Young Learners prepared me well’ (semi-outskirts district), ‘the test centre exams, a number of focal persons specifically mentioned is so big. It is a lot bigger than my school. That scares me a meeting with parents to explain the exams to them. This lot’ (central district). suggests that information dissemination about the exams may • Fun element: ‘The test is interesting, I can match, colour not have been consistent across all IEP districts. It is therefore and write the words’ (outskirts district), ‘it is fun taking the recommended that a better information dissemination plan test, we all like colouring, matching and moving the picture’ is put in place. The plan should include comprehensive (semi-outskirts district), ‘Speaking is fun because it has lots information about the content of Cambridge English: Young of pictures’ (central district). Learners, its motivational value, how it fits with the overall • Test practice: ‘I can learn by heart 34 over 36 questions for teaching and learning strategy at a classroom level, at a school Starters Speaking’ (outskirts district). level and at a DOET level. Also it is important to clarify and quantify how much extra work, if any, is required on behalf of • Test difficulty: ‘The test is as easy as a piece of cake’, the teacher, student, and parent. It is hoped that with a better ‘listening to spelling and write the name down is difficult information dissemination plan and a better communication but I can do it very well’ (semi-outskirts district), ‘the strategy, teachers’ and parents’ anxiety about the test would test has a lot of difficult words’, ‘I like the writing part just be addressed. It is also essential to ensure that all schools in because I can think of the word by myself’ (central district). IEP have consistently provided information to parents about • oral examiner effect: ‘The oral examiners are kind, sweet new initiatives and interventions. This recommendation is in and always smile’, ‘I like the speaking part . . . when I say line with Cambridge ESOL’s third maxim of impact, which is something right, she said very good’ (semi-outskirts COMMUNICATE (see Saville’s article in this issue). district). With regard to the suitability of Cambridge English: Starters for students in Grade 2, the comparative test score data (see Focal persons’ perspective ‘Investigation point 3: Learner language progression’) should Overall, the focal persons view the introduction of an shed light on the teachers’ concerns. Additionally, it would be internationally recognised external assessment as a quality worth considering a classroom observation exercise in order assurance badge for the efforts made by the school and the to have an external voice assessing level suitability. teaching team. They realise that although it puts pressure on them, it increases motivation in teaching and learning investigation point 1b: Attitude towards learning english English. They see Cambridge English: Young Learners as a fair Parents’ perspective assessment – as one principal said: ‘nothing is fairer because Research suggests that parental attitude towards education it is international, independent and professional institution and learning has an effect on their child’s level of attainment which gives the assessment and results reflect on what we (see Bartram 2006, Gu and Saville’s article in this issue). have done on teaching and learning English’. Another principal Therefore, we asked parents about the value of their children said: ‘it is a motivation for parents and students in IEP to have learning English and whether they are happy about the more focus on learning English. Also it sets the standard for introduction of English in IEP schools from a young age. Even the school to have plans to develop outstanding students though one parent (a father with a postgraduate degree) and to support students who do not get average number of stated that ‘as children are only in Grade 2, they are not good shields’. at Vietnamese, so English should be considered as a foreign Focal persons also commented on the different test parts in language and should not be paid too much attention with relation to students’ ability level: unnecessary pressure’, the survey results showed that at Writing: ‘The writing part of the test seems reasonable – least 92% of parents saw English as a means to better life looking at the given words and rearranging them or copying opportunities whether it is for social, study or work purposes. the given word.’ In addition, 88% of the parents stated that ‘it’s important to Speaking: ‘One of my students has a problem with me that my child learns English even if he/she finds it difficult’. pronunciation. When taking the exam, he got a lot of Interestingly, however, despite the introduction of IEP, 77% encouragement from the oral examiners and that made him of parents continue to enrol their children in English lessons more confident in using English. His shields on Speaking is outside school. The frequency of opting to do so increases quite high – 4 out of 5.’ as the parents’ socio-economic status increases and as their Listening: ‘Listening is the most difficult part of the test, level of education increases. especially listening to names and numbers.’ Teachers’ perspective Discussion All teachers reported that learning English is essential for The above results show that the attitudes of key stakeholders students today. Although the majority of the teachers (94%) (teachers, parents, students, policy makers, policy agreed that grammar, vocabulary, and the four skills have implementers) towards assessment in general and towards equal importance in terms of learning English, 30% of the Cambridge English: Young Learners in particular are very © UCLES 2012 – The contents of this publication may not be reproduced without the written permission of the copyright holder.14 CAMBRIDGE ESOL : RESEARCH NOTES : issue 50 / november 2012 teachers disagreed on spending much of classroom time Starters has on the classroom. A further point that is worth on grammar activities. Once again there was no conclusive noting is the comments provided by the students in the focus evidence from the teachers’ profile to say that it is the groups. Despite their young age (7–8), the comments given teachers’ experience or qualification which is affecting their are insightful and in some cases moving, which could be an views. When asked to prioritise what they would like to see indicator of students’ realisation of the value of English. Some classroom time spent on, the result was as follows in order of of the reasons given by the students could be used when priority: speaking, listening, reading and vocabulary, followed raising parents’ awareness to the value of IEP. by writing and grammar. Teachers’ decision on which skill they should focus on during classroom time and how this decision would affect Students’ perspective students’ performance on external assessment is something When asked why they have joined IEP and the reasons we return to when examining students’ score data later in behind their desire towards learning English, the following are this paper. some of the typical responses students gave. Responses are investigation point 2: Learner motivation grouped under four key themes. Parents’ perspective • Family support: ‘If my English is good, I can save my parent’s money by winning scholarship to study abroad’ Parents were asked to voice their opinion with regard to (central district), ‘my parents want me to’, ‘I study English changes they have perceived in their child’s motivation to learn well so I can teach my younger brother’ (semi-outskirts). English as a result of being part of IEP and taking a Cambridge English examination. The results are discussed below. • Functional purpose: ‘If I don’t know English, I could not The majority of parents (87%) believe that their children communicate with people outside Vietnam’ (central like the English classroom and getting an international district), ‘If I know English, I can show the foreigners how certificate which shows their level of attainment. Most to get to the place they want’ (semi-outskirts district). parents (87%) view the introduction of IEP as having a Students also mentioned for study purposes: ‘When I grow positive effect on their child’s motivation in terms of learning up, I want to go to America to study’, for travel: ‘English the language. However, on three occasions, 20% or more of is a popular language, when you travel or when you go on the respondents indicated that their children dislike the test business you have to use English’ and for work purposes: ‘I and get anxious about it and as a result are not motivated to can get a good job’ (central, outskirts district), ‘I want to be learn English. On further analysis of these three statements, a singer and sing English songs’ (semi-outskirts district). no affecting variable in the parents’ profile was detected. • Knowledge gaining: ‘Learning English helps me enrich my Typical comments once again reflect the fact that some of knowledge’ (central district), ‘in the English class, I can learn the parents in this sample are unaware of the content of many new things such as Egypt, Spain’ (semi-outskirts Cambridge English: Starters. A typical comment was: ‘at the district), ‘I want to get more knowledge’ (outskirts district). moment, parents are not clear about the structure of the exam • Fun element of learning: ‘Funny classroom . . . games, paper’. Respondents also expected frequent test practice. A songs, story’, ‘learning English at school isn’t as fun as typical comment reflecting this is: ‘students have not taken learning English at the Centre’ (central district), ‘in the the trial test’. English class, I have much fun . . . draw picture, play games’ Teachers’ perspective (semi-outskirts district). Teachers’ comments focused mainly on the fact that good Discussion students like taking the test so their motivational factor is In general, the responses given by the three key stakeholders quite high, but ‘average students feel nervous and worried demonstrate a very positive attitude towards learning English about it’, hence a lower motivation. in a Vietnamese context. There are a couple of things to focus on from the results reported above. First of all, despite Students’ perspective the introduction of IEP, 77% of the parents reported that The focus groups with students showed that their motivation they continue sending their child to English lessons outside to join the programme and to subsequently take the test school. Since one of the reasons of introducing IEP is social stems from a desire to gain knowledge, to be able to study responsibility and a desire to alleviate the financial burden abroad and to gain a good job in the future. Motivation from parents, it is worth reflecting on why this is the case is also due to parental involvement in their learning and and attempting to address it. Is it because of the importance encouragement. One student from a central district area said: parents place on learning English irrespective of their socio- ‘I want to take the test so I can go to Grade 3 IEP class, mom economic status or educational level? Is it because IEP is not told me that’. Another student from a semi-outskirts area as effective as it is perhaps thought of? Is it lack of awareness said: ‘My mom said if I do well on the test, she will take me to of what IEP goals and practices are? Is it peer or social ice cream shop’. Teachers’ encouragement is also a factor. A pressure? These questions need to be investigated by HCM student from the same district said: ‘When we speak English DOET. right, the teacher rewards us by giving us candy, pencils’. Another point to focus on is the order of priority which Similar statements are echoed in semi-district areas: ‘When teachers give to skill teaching and learning. Teachers I speak English well, the teacher gives me happy faces, candy prioritised speaking and listening over other skills. According and she says “very good”.’ to focal persons in the one-to-one interviews this shift of priority signifies the positive washback Cambridge English: © UCLES 2012 – The contents of this publication may not be reproduced without the written permission of the copyright holder. CAMBRIDGE ESOL : RESEARCH NOTES : issue 50 / november 2012 15 Focal persons’ perspective in teachers’ responses as far as Cambridge English: Young Learners exams are concerned. They reported improvements When focal persons were asked about how the introduction of as follows: speaking and listening abilities followed by reading IEP and Cambridge English: Young Learners exams have affected and vocabulary knowledge, then by writing and lastly by students, most of the statements revolved around children grammatical knowledge. enjoying the English classroom and being motivated to learn English. One principal of a semi-outskirts district area school Students’ perspective said: ‘The students seem not afraid of taking the test. They During the focus groups, students were asked if they feel that get more chances to speak English’, while another from the their English is better now when compared to the beginning of same area said: ‘The students have no motivation in learning the year. Here are some of the typical responses: English. They show no responsibility for English learning’. • From central district schools: ‘Now I can read the story A principal of a central district area school voiced a typical opinion of other principals from the same area when saying: to my mom and dad’, ‘I can write the words in English ‘They speak more English in the class’. correctly’, ‘I can watch cartoons in English and understand it’, ‘before I speak English a little, now I can speak English to Discussion my teachers and foreigners’ Survey and focus group data shows that student motivation • From semi-outskirts schools: ‘Now I can speak English to to learn English is quite high. A recurring theme here is my parents and can read English on the street’, ‘my mom test anxiety which may affect motivation, which was said now my English becomes better’ voiced by parents and teachers and interestingly enough • From outskirts district schools: ‘In Grade 1, I didn’t know not by students. This could be due to the fact that parents many new words now I know a lot of new words’, ‘now I can lack adequate information about Cambridge English: read more fluently’ Starters (a recurring comment) and are not very clear on DOET’s intended use of the test results. Because of lack of Students’ score data information, parents may have speculated that results would Table 2 shows the average shields obtained by the number be used for gate-keeping purposes. Another recurring theme of HCM DOET student cohorts taking Cambridge English: is parental involvement and teacher encouragement playing Starters over a period of two years. Over the two-year period, a key role in learner motivation. When a school principal students’ performance has been consistently high with 11 as states that students are not motivated to learn English, we an average total number of shields. The reader will note that need to stop and ask why this is the case. Is it because of when schools started using Cambridge English: Young Learners teaching practice? Is the level of English higher than they can exams the highest shield average was that for Reading/ cognitively deal with? Is it too much pressure from parents on Writing in 2010. In 2011, a slight shift occurred towards passing the test? Seeking answers to these questions would Speaking, which showed the highest average shield, indicating inform HCM DOET’s subsequent actions. that perhaps in 2011, classroom practices may have put more emphasis on speaking. What is interesting to note is that investigation point 3: Learner language progression Listening has consistently received the lowest shield average. Parents’ perspective All in all, the results are very encouraging given that in most Parents were asked for their views on their child’s proficiency cases the amount of exposure students have to English inside of English as a result of being part of IEP and taking a the school is only in the English classroom. Cambridge English examination. Ninety per cent of the parents agreed that their child’s English has improved due Table 2: Average number of shields received by HCm DoeT students for to these two interventions. However, 24% of the parents Cambridge English: Starters (mainly parents with a university degree) disagreed that speaking Listening r eading/Writing Total their children know more about their level according to 2010 3.94 3.23 4.03 11.06 international standards. One possible reason could be that at 2011 3.89 3.43 3.85 11.17 the time of administering the survey, some children may not have received their certificate as indicated by the following When looking at comparative test score data, we looked comment: ‘We want our children to get high results in at candidates who took Cambridge English: Starters in other examinations‘. Another reason could be lack of awareness Vietnamese contexts and those who took it in the rest of the of the value of Cambridge English: Young Learners exams world. Before we examine the data, it is important to note as illustrated by this comment: ‘I hope that the quality is the following two facts: (a) data from the other Vietnamese proportionate to the cost to satisfy parents’. context comes from private language institutes where Teachers’ perspective students receive English language training at a more intensive rate; and (b) the ‘Rest of the World’ (ROW) context is a Teachers were asked about improvements they have seen in mixture of mainly fee-paying schools and private language students’ English language ability as a result of being part of institutes and some state schools who use Cambridge IEP and taking a Cambridge English examination. With respect English: Starters. In other words, the comparison is not a to IEP, teachers’ responses showed that improvements are straightforward one. clearly seen in terms of speaking and reading abilities followed Figure 3 provides average shields obtained per test paper by listening and vocabulary acquisition, and then by writing in the three contexts. The figure shows that overall there and grammatical knowledge. A similar picture emerged © UCLES 2012 – The contents of this publication may not be reproduced without the written permission of the copyright holder.16 CAMBRIDGE ESOL : RESEARCH NOTES : issue 50 / november 2012 Figure 3: Comparative test score data – Cambridge English: Starters HCM DOET mean shields for 2010 & 2011 Rest of Vietnam mean shields for 2010 & 2011 ROW mean shields for 2010 & 2011 Speaking Reading-Writing Speaking Reading-Writing Speaking Reading-Writing 3.91 3.92 3.94 4.13 4.44 4.08 Listening Listening Listening 3.36 3.56 3.90 are slight differences in the number of shields obtained paper has the lowest average number of shields – a picture per test paper. It shows that irrespective of the context, which is replicated across all three contexts. Listening receives the lowest number of shields and in investigation point 4: Changes in teaching practice terms of rank ordering the skills per context, HCM DOET has the same profile as the rest of the world with Speaking This section reports mainly on findings from the teachers’ receiving the highest number of shields followed by Reading/ survey and where relevant views were sought from other Writing followed by Listening. Both HCM DOET and the participants in the study. rest of Vietnam have a total average of 11 shields. This is an When teachers were asked whether their teaching practices encouraging picture for HCM DOET given that those who have changed as a result of the intervention, 96% of them take Cambridge English: Starters in the rest of Vietnam come said they had changed as a result of teaching in IEP and from private language schools and not state schools, where 92% said that their practices had changed as a result of the stereotypically the former would have many more resources introduction of Cambridge English: Young Learners exams. The available to them. The rest of the world has a total average of changes which have occurred from the teachers’ perspective 12 shields. are grouped under the following categories: • increased adoption of some Assessment for Learning (AfL) Focal persons’ perspective principles School principals or their deputies agreed that there has • introduction of collaborative teaching been a notable progression in students’ English, especially • improved teacher motivation in speaking, when asked whether they have perceived any change as a result of the intervention. • increased use of target language versus L1 • best practices utilised. Discussion With regard to language progression, survey data and focus Increased adoption of some AfL principles group data indicated that in most cases there is a notable As a result of the intervention, the majority of teachers (as progression as a result of the intervention. The skill which seen in percentage agreement in Table 3) adopted some showed the most observed improvement was speaking, as of the principles of Assessment for Learning (Assessment reported in teacher and student responses. This was also Reform Group 2002, Black, Harrison, Lee, Marshall and substantiated when looking at the score data (see Table 2). William 1990, William 2009). For example, goal sharing with The overall average number of shields obtained is 11 out of a learners and ensuring that they know the standard or level possible 15, which indicates that Cambridge English: Starters is they are aiming at. Similarly, teachers adopted the principle within the ability level of students who have taken it. of working together with the learner to review and reflect on Earlier in the paper there was a discussion concerning the assessment information and giving feedback to learners in suitability of Cambridge English: Starters for Grade 2 students; ways that enable them to improve and plan their next steps. these results show that the level is suitable for HCM Grade 2 students given the high number of shields acquired per skill Introduction of collaborative teaching and overall. The teachers earlier indicated that they would like Ninety-three per cent of the respondents stated that joining to prioritise the teaching of listening in classroom time. Their IEP has allowed teachers in school to work more as a team views are supported by the data in Table 2 where the Listening Table 3: Adoption of some AfL principles (percentage agreement and mode) strongly Agree Disagree strongly mode agree (4) (3) (2) disagree (1) 1 I share success criteria with my students 26 68 5 3 2 I give oral and written feedback to help identify next steps in learning 28 65 6 3 3 I use assessment data to inform the learning and teaching process 28 63 8 3 4 I give opportunities for learners to demonstrate that they have taken feedback 27 67 5 1 3 into account in their learning 5 I discuss learning objectives and outcomes with my students 24 71 5 3 © UCLES 2012 – The contents of this publication may not be reproduced without the written permission of the copyright holder. CAMBRIDGE ESOL : RESEARCH NOTES : issue 50 / november 2012 17 Table 4: use of target language vs L1 (percentage agreement and mode) strongly Agree Disagree strongly mode agree (4) (3) (2) Disagree (1) 1 I teach in English more than in Vietnamese 51 49 3 2 Students talk to other students in English more than in Vietnamese when they do 22 59 19 3 classroom activities 3 Students talk to me in English more than in Vietnamese 25 43 32 3 4 I encourage students to speak to each other in English 43 54 3 3 and share resources and discuss things more. Similarly, 94% • increased use of pair and group work so that students have said that they discuss planning and outcomes with team an opportunity to use the target language. members/colleagues as a result of the programme. • better lesson plan formulation, for example, a plan including aims, methods, stages, timing, aids, anticipated Improved teacher motivation problems, assumptions, and interaction patterns. Eighty-nine per cent of the teachers reported that the use of • increased reflection on how the lesson went and on own Cambridge English: Young Learners exams has increased their teaching. motivation to teach English. Parents’ perspective Increased use of target language versus L1 When asked about perceived improvement in teaching The statements found in Table 4 were designed to find practice as a result of the intervention, 88% of the parents out whether there has been an increased use of the target agreed that English lessons have become more fun language (English) versus L1 (Vietnamese) in the EFL (e.g. through games and communication activities), are classroom as a result of the intervention. intellectually challenging and have provided their children Table 4 shows that the majority of teachers prefer and with clearer learning objectives and outcomes. Parents also practise the use of the target language inside the classroom. perceived change in the way feedback is provided to their When looking at statement 3, a further analysis of the data to children; in the way that their children are encouraged to see whether the result is due to teachers’ experience or school reflect and assess their own progress. The following comment district area revealed that no conclusive finding can be stated. reflects the parental perspective on perceived changes: Similarly, there were no comments by the teachers to shed • ‘although my child’s English level is intermediate, I like the light on the level of disagreement on this statement. intensive English programme as it has outdoor lessons Students’ perspective which make him feel more comfortable and interesting in English’. One of the elements the focus group with students tried to elicit is how frequently the target language and L1 is used in Focal persons’ perspective the classroom. Typical responses are as follows, irrespective When asked about changes perceived in teachers’ attitude of the geographical location of the school: ‘We speak English towards teaching and their teaching practices, the responses to each other and to our teachers’, ‘the teacher speaks English could be summarised under three main categories. The first is a lot’, ‘we speak more Vietnamese in the class’, ‘we don’t a sense of responsibility: ‘The teachers are more responsible often speak English to one another’, ‘the teacher speaks more for the teaching: they prepare more activities in class . . . they Vietnamese in the classroom’, ‘the teacher speaks more pay more attention to the students and are ready to stay English while we speak more Vietnamese’. after school to help out’. The second is status: ‘The test is a Parents’ perspective chance for us to be named “international teachers” because it is international standardised assessment’, ‘I will be famous When considering parents’ comments, we find that they among the parents if my students do well on Starters’, ‘if my tend to be divided between ‘let students use English more students did not get high number of shields, parents will think frequently in place of Vietnamese’ and ‘as children are only in I am not qualified for teaching English’. The third is application Grade 2, they are not good at Vietnamese, so English should of best practices, as outlined above. be considered as a foreign language and should not be paid too much attention with unnecessary pressure’. Discussion Best practices utilised Data collected from surveys and focus group discussions indicated that in general there has been a positive Teachers who have been engaged in the intervention have change in teaching practice as evidenced in improved advocated the following best practices to colleagues within teacher motivation, increased use of the target language, their schools (with at least a 94% acceptance rate): increased adoption of AfL principles and the introduction of • Adaptation of teaching methods so that they are collaborative teaching. One thing to note as an unintended appropriate to students’ learning goals and styles. In effect of the intervention is teachers’ view of themselves the same vein, ensuring that resources are appropriate, as ‘international teachers’ since they are teaching towards accessible, and relevant to students’ learning needs. international standards. • Frequent use of interactive tasks so that students can speak among themselves and with their teachers in English. © UCLES 2012 – The contents of this publication may not be reproduced without the written permission of the copyright holder.18 CAMBRIDGE ESOL : RESEARCH NOTES : issue 50 / november 2012 investigation point 5: Change in decision making• better plan to disseminate information on the intervention The semi-structured one-to-one interviews with focal persons • ensuring information is provided consistently and sought to find out if, during the course of implementing adequately to stakeholders the initiative, changes occurred in decisions which have • level assessment via empirical evidence in addition to been made prior to implementation. Reponses given were classroom observation affirmative and changes have occurred as follows: • further in-depth investigation as to why 77% of the • Some of the schools who were involved in IEP and used sampled parents continue to send their children to private Cambridge English: Young Learners exams decided they language institutes despite the introduction of IEP wanted to opt out of the programme because they could • probe further as to why some principals felt that learners no longer meet criteria set by HCM DOET. For example, are not motivated to learn English. they lacked qualified teachers due to teacher movement or they had to exceed the maximum class size of 35 students Positive effects because of the demand to provide additional student spaces. The study also revealed areas where positive effects have • Prior to this study, HCM DOET had suggested that the been achieved as highlighted below: guiding principle for continuation in IEP is achieving an • The above findings showed some clear effects such as the average of 10 shields in Cambridge English: Starters exams focus on speaking, which is a direct positive effect of the with no fewer than three shields per skill area – a decision introduction of Cambridge English: Starters, which is designed that has been borne out by cohort-consistent results over based on a communicative approach to language learning. a period of two years (as seen in Table 2). During the What is more important is that this focus did not detract course of this study and as a result of extensive discussion from attention being paid to the other skills as evidenced between Cambridge ESOL (test developers) and HCM by test score data. Although there is a notable language DOET about the nature of Cambridge English: Young Learners progression in terms of speaking, students also performed and its intended purposes (not to be used in what can be well on the other skills. perceived as a high-stakes decision making context), a • Another notable effect is the positive change in terms decision was made to waive this condition and leave it to of teaching practice with the adoption of certain AfL individual schools to decide on their minimum requirement. principles, the introduction of collaborative teaching, and As of May 2012, each school stipulates the number of the utilisation of best practices such as teacher reflection shields their students are required to achieve, based on the or adaptation of teaching methods to support students’ Cambridge English: Starters test results, in order to continue learning goals and styles. into the Grade 3 IEP. Students from the selective English programme (non-intensive) can move to the intensive • It might also be deduced that the intervention led to programme if their Cambridge English: Starters results meet increased parental involvement in their child’s learning in the school’s requirements and there are spaces available terms of encouraging them to learn English, taking them in the school. This change in decision is also in response to to extra English classes as provided by IEP and providing recurrent comments made by focal persons on the criteria incentives for better performance as seen from the set and how it may be impossible to meet given certain comments made in the focus group discussions. Parental school conditions. involvement and teacher encouragement were a recurring theme in the findings of this study as playing a key role in • At the time of writing this paper, HCM DOET announced the learner motivation. launch of a project to further enhance English language skills in 2012–13 with an estimated investment of approximately unintended effects 204,000. ‘The project aims at a comprehensive renewal of • The study illustrates that when decentralisation of decision teaching and learning methods in every grade and at every making is well executed, innovative approaches that suit training level, so as to achieve dramatic progress in students’ the local context can lead to positive effects. Although the speaking, listening and reading skills. The project will then strategic objective for improving language standards came stretch over a 10-year period in which English language from MOET, it was up to HCM DOET to decide on how to will be a compulsory subject from third grade onwards in achieve this and it is also up to schools to decide whether to schools’ (Linh 2012). be involved in the initiative or not, which provided a sense of ownership and faith in the intervention. • One of the unintended effects is the change in decision Key findings and recommendations making based on discussions that took place with The key question under investigation was: ‘What is the focal persons during the course of this study. We are intended/unintended effect of HCM DOET’s strategic decision referring here to the decision about the use of Cambridge to increase English language provision through IEP and to English: Starters and the number of shields obtained (see ensure the quality of the provision through the use of external ‘Investigation point 5: Change in decision making’). assessment, i.e., Cambridge English: Young Learners?’. • Another unintended effect is better utilisation of children’s free time. After a half-day of schooling, children are Lessons learned engaged in IEP and Cambridge English: Young Learners study. The study revealed areas where improvements can be made This alleviated parents’ anxiety as to how to engage their such as: children’s free time once the half day of schooling is over. © UCLES 2012 – The contents of this publication may not be reproduced without the written permission of the copyright holder.

Advise: Why You Wasting Money in Costly SEO Tools, Use World's Best Free SEO Tool Ubersuggest.