12 Apr 2024

The evaluation of EDI principles in coursebook evaluation checklist and EAP coursebooks


Introduction

The concepts of Equity, Diversity, and Inclusion (EDI) have received considerable attention in higher education (HE) and have been widely recognized among HE sectors as essential elements within the missions of their institutions (Barnett 2020). However, despite the frequent appearance on universities’ mission statements, Von Robertson et al. (2016) argued that the support provided by many institutions are not sufficient, moreover, based on students’ negative experiences, it seems some universities only appear to be inclusive superficially, whilst not demonstrating a true commitment to EDI. 



In the attempt to develop university students’ intercultural competence throughout their academic study, fruitful resources were produced in terms of comprehensive guidelines, training programmes, long term recruitment and retention plans at the institutional level (Fuentes et al. 2021).  Lots of efforts have been made in increasing EDI awareness and positive changes through curriculum (Tamtik 2019), such as creating more inclusive classrooms, using appropriate language and exploring EDI related topics in courses. However, Fuentes et al. (2021) believed with the focus falling on the generic curricula and pedagogical approaches, how to design an inclusive syllabus remains under-developed. Brown and Nanguy (2021) also pointed out the global coursebooks employed for English language teaching (ELT) have been criticized as ‘one size fits all’- avoiding sensitive topics such as politics, alcohol, religion, sex, narcotics, isms, pork (PARSNIP), ignoring important ethical and educational concerns to achieve commercial purposes. 


Among the extensive provision of coursebooks, a comprehensive evaluation is crucial for deciding the most appropriate one in specific teaching contexts (Demir and Ertas 2014). Therefore, to address the increasing attention of EDI in HE and to assist with its effective application in ELT, this study intends to evaluate the extent EDI concepts are included in existing English language coursebook evaluation criteria, to suggest EDI appropriateness coursebook evaluation criteria, and to evaluate global English for Academic Purposes (EAP) coursebooks. Though being criticized as not practical and containing over simplistic or ambiguous words, checklists are often used as an effective tool to evaluate English language coursebooks (Mukundan et al 2011), which will be employed as the evaluation method in this study. 


Literature Review 

EDI in Higher Education 


Equity, diversity and inclusion are complex and interrelated terms which carry intricate meanings in varying contexts (Tamtik 2019). Equity is defined by the AAC&U (n.d.) as providing equal access to underrepresented populations, so that they can participate and achieve success in education. With the co-existence of different forms such as equality, justice and fairness, multiple and overlapping versions tend to appear in policies interpreting contemporary norms (Gilbert et al 2011). Equity, therefore, with the combination of other words or situations could address different focuses (Savage 2013). Rizvi and Lingard (2011) argued while demonstrating its importance, there is a lack of agreement over the definition of Equity. However, despite its difficulty to understand in a generalized abstract manner, it is apparent when combining and comparing with contrasting values. 


The definition of diversity includes a series of identifying factors such as race, ethnicity, gender, sexual orientation, socio-economical background and disability, with added considerations of religion, education and family/marital status (Tamtik 2019; Loden and Rosener 1991). However, it is more than those naming categories that are used to identify people, it extends to the principles of inclusion, which not only acknowledge but also value differences, and the ability for people to engage equitably in society (Multiculturalism B.C., 2000). 


In addition to removing barriers that limit participation and achievement, UNESCO (2015) addresses that inclusion in education systems should ensure and respect  diverse needs, abilities, and characteristics, and eliminate discrimination in learning environments. To put it in simple terms, it is fundamental for institutions to be safe, caring and engaging places where everybody feels included and in which there is a sense of belonging (Dixon 2011).


Due to the multi-layered connections between equity, diversity and inclusion, universities tend have a slogan-like approach and address all EDI issues together as an integrated concept, rather than looking at the detailed complexities within each term (Tamtik 2019). However, Tamtik (2019) found many universities only stay at a surface level by expressing their institutional principles and matching these with the federal performance criteria, very few aim at identifying challenges and goals designed, and reflective of their particular university. 


Coursebooks and evaluation criteria 


Despite the ongoing debate on whether coursebooks help or hinder the process of teaching and learning, they provide extensive language input and exposure for learners, more importantly, they hold a unique and significant social function, which is to represent generational knowledge and culture of human beings (López-Medina 2016).  Sheldon (1988) proposed that coursebooks play a significant role in any ELT programme. Apart from providing ready-made materials which lessen teachers’ preparation time (Tsiplakides 2011), coursebooks serve as a framework and measure of students’ learning progress. 


At the same time, the use of coursebooks has also drawn negative criticism. Tsiplakides (2011) argued the target language used tends to be unnatural, which does not represent real-life scenarios, hence challenging their suitability for the purpose of communicative language teaching. Renner (1997) believed coursebooks carry a great degree of cultural content as well as the target language, which was unfortunately dominated by the mainstream groups, with little representation of cultural diversity and often contains stereotypes and biases. 


Since coursebooks could be outdated due to the long process between writing and publication (Sheldon 1988), it is critical that materials should meet current needs and objectives (Murphy 1985). What is more, coursebooks could be suitable for some situations or students but might not be suitable in different contexts (Richards 2007). Therefore, a comprehensive evaluation comes as a vital step in selecting the appropriate coursebook for the targeted learners within a specific learning context (Demir and Ertas 2014). Coursebook evaluation also helps with identifying whether the materials facilitate the achievement of the expected language learning outcomes (Mukundan 2006). 


Checklists are a commonly used instrument to evaluate ELT coursebooks, as they provide a set of generalized criteria that allows a sophisticated and systematic evaluation (Mukundan et al 2011). Hutchinson and Waters (1987) take evaluation as a straightforward matching between the needs and solutions, which appears as rating scales or scoring systems (Sheldon 1988). On the other hand, Mukundan et al (2011) pointed out that many evaluative criteria are identical regardless of the teaching and learning situations, places or purposes. What can be sure is, however, the importance of different criteria will change over the years, along with the interpretation of the scoring system (Sheldon 1988). Şahin (2020) pointed out many current concepts and topics are not included in early checklists, therefore, a timely review is needed to keep them in line with the steadily changing methodologies in language teaching. 


EAP courses


English for Academic Purposes (EAP), emerged in the context of academic study and scholarly exchanges, contains linguistic, applied linguistic and educational topics through the medium of English (Hamp-Lyons 2011). In Jordan’s (1997) view, EAP courses include essential study skills such as listening comprehension and note taking, academic writing, discussion, critical thinking, as well as the proficient use of language in a formal academic register. What content would be most appropriate to constitute these EAP course components, however, has drawn a degree of concern, i.e. to read or listen to which piece of materials (Jordan 1997). Various practitioners have criticized EAP for avoiding important ideological issues and that it should be prepared to adapt to the cultures, world views and languages of foreign students (Benesch 1993). Pennycook (1997) argued that limiting the focus to developing students’ academic linguistic skills would miss a crucial opportunity in building their linguistic, social, and cultural criticism, which could be very beneficial for them both within and outside the educational systems. Therefore, EAP should bear the responsibility to expose learners to the broadest context possible, to foster their linguistic and critical awareness (Pennycook 1997).


Content and Language Integrated Learning (CLIL)

As an educational approach that addresses a dual focus on both language and the content, CLIL has attracted lots of attention and its popularity keeps growing (López-Medina 2016). With CLIL, the subject matter is no longer taught in a foreign language, instead, with and through a foreign language (Eurydice 2006). With the evident global demand for English learning, English serves as a vehicular language that presents the extensive content subjects which prepare students for future studies or work (Coyle et al 2010). Apart from promoting linguistic competence, CLIL also stimulates learners’ cognitive flexibility (Marsh 2009). With its promotion on constructivist learning, CLIL provides learners with different thinking perspectives, create positive impact on their conceptualization, and helps them progress towards more autonomous and sophisticated learning (Coyle et al 2010). 


The emergence of English as a lingua franca has had a significant impact on higher education, with its rising importance as the global language, it is arguable that ELT sits as a part of the core academic life (Coyle et al 2010). Despite its considerable potential, Coyle et al (2010) think CLIL is at an exploratory stage in many HE institutions with situational and structural obstacles. 


Research gap

The above analysis of related literature presented the emerging needs and challenges of promoting students awareness of EDI and intercultural competence through coursebooks (Tamtik 2019; Brown and Nanguy 2021), the necessity of updating the coursebook evaluation criteria (Sheldon 1988), the narrow focus of the content scope in EAP (Pennycook 1997; Benesch 1993), and the growing popularity yet exploratory level of CLIL in HE (López-Medina 2016). However, scarce research has been identified regarding the representativeness of EDI in both the coursebook evaluation criteria and the EAP coursebooks. 


Research aims and questions.

To tackle the above issues and identified gap, the aim of this study is to create an EDI appropriateness checklist for EAP coursebook evaluation. This will be achieved by reviewing and comparing current coursebook evaluation checklists regarding the degree of EDI principles, to embed EDI elements with CLIL approach, and thereafter, to evaluate global EAP coursebooks with my developed checklist.


  1. Therefore, it focuses on answering the following two questions:
  2. Do current coursebook evaluation checklists include EDI principles? If not, what changes can be suggested?  
  3. To what extent do EAP coursebooks include EDI related content? And what suggestions could be given? 


Methodology

Research design and process


To decide what criteria are important and suitable for evaluating EAP coursebooks in terms of EDI, an extensive literature review on previous research is required (López-Medina 2016; Lei and Soontornwipast 2020). The related literature includes existing English language coursebook evaluation checklists, guidelines for developing checklists, developed CLIL checklists, and EDI content benchmarks in coursebooks. Essential criteria shall be summarised, which will be used to adapted into an existing checklist or to produce a new checklist. 


This is followed by semi-structured interviews with tutors to collect valuable insights and constructive suggestions regarding the components of the checklists. Semi-structured interviews give greater flexibility in the order of the questions being asked and provides opportunities to gather in-depth information by asking follow-up questions (Zacharias 2011). Selection criteria of the participants will be addressed in the next section. A draft checklist is expected to be developed after the qualitative analysis of the interview data. 


To test the validity of the developed checklist, it will then be used to evaluate two chosen global EAP coursebooks, which will also contribute to the limited research regarding EDI appropriateness in global EAP coursebooks. The global EAP coursebooks will be chosen based on their popularity based on usage and quantum of publication. 


The design and process of this study takes reference from scholars’ work in related research areas (López-Medina 2016; Cruces Rodríguez 2015; Lei and Soontornwipast 2020; Demir and Ertas 2014; López-Medina 2021; Jafarigohar 2013; Brown and Nanguy 2021; Coyle et al 2010). 


Sampling 


At the literature review stage, this study intends to review an exhaustive number of evaluation checklists relating to ELT and CLIL, to cover a wide representation of the features and critiques of existing criteria. This will help with the overall examination and selection of the most important criteria that are suitable for EAP coursebook evaluation in terms of EDI. A potentially suitable checklist will be evaluated in the pilot study; adjustment shall be made based on the pilot result. 


In terms of the interviewing of participants, this study intends to approach tutors who ideally have 1) EAP teaching experience; 2) knowledge relating to EDI in a teaching and learning context; 3) experience in evaluating coursebooks. The selection of the sample thereby follows a homogenous procedure, which focuses on participants who share specific characteristics that are closely related to the topic being researched (Etikan et al 2016). In consideration of the available research opportunities at this current institution, and the challenges of finding participants that meet all mentioned criteria, this study intends to employ both convenience sampling and snowball sampling. Use of convenience sampling means taking advantage of the geographical proximity and easy accessibility, it will firstly approach members of the target population that meet the above criteria and are willing to participate (Etikan et al 2016), after which with the purpose of enriching the sampling pool and reaching saturation, the snowball sampling will be used to recruit more interviewees provided by initial participants (Noy 2008). One drawback of the snowball sampling being it concedes the control of the sampling phase to the participants, in other words, it is the participants’ initiative to provide more interviewees and to drive the sampling process forward (Noy 2008). Therefore, to mitigate the potential negative impact of using snowball sampling I will be acutely aware of the status of the data collection progress ensuring that successful collection of the required information is achieved.


Though convenience sampling has the benefit of being easily accessible, an apparent disadvantage is the potential bias caused by limited representation of the data (Etikan et al 2016). In terms of this study, the foundation of the criteria will be generated from the literature review stage, and the interview is used to add supplementary data. Therefore, it will not become the dominant aspect of representation. Moreover, with the combination of snowball sampling, participants from different institutions and background would appear and present a level of diversity. 


Regarding the choice of coursebooks, due to the space limit, two global coursebooks from different publishers will be selected for analysis. To reflect the wide representation of current EAP classroom teaching and learning, coursebooks that are most popular with the highest levels of publication and the latest editions will be used. Considering the targeted learners are university students, selected coursebooks will be at upper intermediate level. 


Data collection and analysis 


For the convenience of participants and the potential geographical distance, interviews will be conducted either face to face or via MS Teams. Collected data will be manually transcribed and analysed based on relevant themes and topics, which will then be used to enrich the existing data. The selected coursebooks will then be evaluated based on the developed checklist, findings regarding the indicators of EDI elements will be compared and explained in detail. 


Strength and limitation


As mentioned in previous sections, this study distinguishes itself by addressing the importance of EDI in higher education through English language coursebooks, and the emergent needs of including EDI both in coursebooks evaluation criteria and coursebook content. This study also attempts to explore the practicality of using CLIL approach to evaluate the degree of EDI content in coursebooks, which will be useful for future adaptations of coursebook and implementation of lesson planning and classroom teaching. 


However, the apparent limitation of this study is its narrow focus on EAP coursebooks. It further limits the evaluation range into upper intermediate books from only two global coursebooks. Future research could be done relating to English for Specific Purposes coursebooks, broader English language coursebooks, or with the focus in the EFL or ESL context. On another note, although only student books are selected in this study, as the primary source of content for both students and tutors, skipping teacher’s book and other additional resources won’t diminish the impact of the study (Brown 2021). 


Ethical consideration


Ethical approval with participants will be granted before the interview, to comply with information confidentiality requirements, and to provide them with the intended purpose and content of the interview (Rose et al 2019). In this case study, consent forms will be sent to interviewees, including the main focus of the interview, and that it will be recorded and kept in a safe place, which will only be used for research purposes, and any point in time it is possible for the participants to stop or withdraw from the interview (Dornyei 2007). The data and results will be reported truthfully without personal preferences or biases, even if the result was unexpected or conflicts with the original hypothesis, it adds depth to the study findings and its implications (Cohen 2007; Rose et al 2019). 

Reference