Print   |   Save

Towards an Evidence Base for CLIL
How to Integrate Qualitative and Quantitative as well as Process, Product and Participant Perspectives in CLIL Research

Andreas Bonnet1
Universität Hamburg


Over the last two decades, CLIL has become a well-established part of education systems across Europe. Administrators and other stakeholders consider that it fosters important goals such as the acquisition of foreign language competences and higher order thinking skills as well as the support of motivational gains. In order for CLIL to develop sustainably in the future it is crucial that an empirical evidence base for it be established. Whereas the domain of language competence has been covered to a considerable extent, other areas have been less well researched. Moreover, it is very difficult to attribute established outcomes to CLIL practice, because the control of contextual factors is problematic. In the following article, a selection of existing CLIL studies is analysed with respect to their methodology. It will be argued that a twofold integration of research with respect to its approach (qualitative and quantitative) and perspective (product, process and participant) would increase the conclusiveness of findings considerably. Also, an integrative research strategy based on the documentary method is outlined and examples of its existing implementation in the field of CLIL investigation are provided.


Keywords: Bildung; documentary method; empirical CLIL research; integration of qualitative and quantitative approach; integration of product-, process- and participant perspective; research methodology; sociology of knowledge

1. Introduction

In the course of the last 20 years, Content and Language Integrated Learning (CLIL) has become a well-established part of educational systems across Europe (Wolff, 2007). The last two decades have also seen a steep rise in CLIL research, as evidenced by a large number of research articles, as well as by many conferences and their published proceedings (e.g. Wolff, Marsh, 2007). This process has attracted the attention of the EU-commission that is increasingly prepared to consider CLIL an important facilitator of European integration. Concomitantly, a series of educational claims are being made such as the potential of CLIL to foster ‘cross-border-competence’ (Schimak, 2010). Therefore, the powerful metaphors of ’two for the price of one’ and the ‘added value of CLIL’ seem to have become accepted truths in the general CLIL-discourse rather than hypotheses to be tested through evidence-based research. Still, they create a powerful atmosphere of optimism and almost limitless belief in the potential of CLIL. In this light, greater scrutiny of this potential and the limits of CLIL is needed more than ever because a concept associated with unrealistic hopes can be quickly sidetracked. The CLIL community now considers working towards the development of a more substantial evidence base as a major challenge to be addressed (cf. Meyer, 2010).

Whereas the goal is clear, the way to reach it has yet to have been discussed in depth. The following article is meant to be a contribution to identifying landmarks and possibly even raising some signpost. First, the article tries to do so by analyzing existing research findings on CLIL which upon close analysis appear inconclusive or even contradictory. Second, a methodological analysis will show that this is no accident but rather a consequence of the nature of the research undertaken so far. This idea will be elaborated on by discussing the micro-macro-problem in greater depth. Third, an analysis is undertaken of existing research for approaches that lend themselves to integration. On the basis of this analysis a research strategy is outlined that integrates product-, process- and participant-research as well as a qualitative and quantitative approach. This “doubly integrative” strategy uses the documentary method as its core, which is complemented by additional instruments and procedures.

Page 66
Page 67

2. The two tales of CLIL

In the past two decades, empirical research on CLIL has focused on three major areas: language competence, subject matter competence and teacher research. In all three areas, there are findings that support telling a tale of CLIL success. With respect to the first aspect, several individual studies (e.g. Bredenbröker, 2000) and the large-scale national DESI study (DESI, 2008) on the functional language competences of 15-year-olds have shown that CLIL learners have a higher L2-competence than their non-CLIL control groups. There are also studies that come to similar conclusions with respect to subject matter competence. There are indications for both science and humanities subjects that the development of learners’ subject matter competence is generally comparable to that of non-CLIL control-groups. In some cases, CLIL learners even seem to benefit from the CLIL situation with respect to certain aspects of subject matter competence (e.g. Bonnet, 2004; Lamsfuß-Schenk, 2008). Finally, findings from teacher research (e.g. Dirks, 2004; Viebrock, 2007) also indicate positive effects of CLIL. They suggest that CLIL teachers spend more time on lesson and material preparation and demonstrate cultural sensitivity. This would mean that classroom practice would be up to current standards and include crucial methodological elements (Meyer 2010, Bonnet forthcoming) such as explicit work on discourse functions the importance of which has also been demonstrated in empirical studies (Vollmer, 2009).

In all three areas, however, there are aspects that question this tale. With respect to language competence, none of the above studies has established conclusive links between learning outcomes and classroom practice. Also, there are strong indications from evaluation research (Zydatiß, 2007, and in this volume) that social selection plays an important role in German CLIL. Therefore, high outcomes in CLIL groups might well be ascribed to factors other than classroom practice. As far as subject matter competence is concerned, the evidence base is even thinner. There are strong hints that learners struggle considerably during the initial stages of CLIL and that success is heavily dependent on students’ language competence (Vockrodt-Scholz, Zydatiß, 2007) – in other words: the threshold hypothesis does indeed seem to apply, which Zydatiß corroborates with findings from Clapham (1996). These results are in line with recent findings from Scandinavia (Jäppinen, 2005) and question rather than support the idea of the added value of CLIL as it is put forward by Baetens-Beardsmore (2010) or Wolff (2010). While Baetens-Beardsmore’s optimistic interpretation of empirical data would have to be checked for underlying natural bilingualism, Wolff’s idea of a symbiosis between CLIL and learner autonomy (2010) still needs empirical substantiation. Furthermore, research into teacher and classroom practices (Viebrock, 2007; Dalton-Puffer, 2007) suggest that many CLIL classrooms tend to be teacher-centred and rather weak on working with discourse-functions.

In order at least to hint at what this inconclusiveness could mean, I should like to take the liberty of sketching a different story of CLIL. Although it is of course hypothetical, it matches the evidence put forward in the preceding paragraph. This tale would take seriously that classroom research across school types and subjects suggests that a successful negotiation of meaning is rather the exception than the rule. This means that subject matter more often than not remains unclear, leaving clarification to the learners and their families (Gruschka, 2011). Assuming the social selectivity of CLIL schools, suggested by Zydatiß’ results, CLIL learners would have a better chance of getting their questions resolved, because their families have more social, cultural and financial resources to make up for the classroom’s shortcomings. Therefore, further empirical research is needed to check in how far CLIL in the German context – or even elsewhere (Bruton 2011) – might be a successful facilitator of social selection rather than of improved learning.

3. Methodological analysis of empirical CLIL research

On the basis of existing research, it is hard to tell which of the two tales is true. One could easily argue both cases. It is very likely that it is a matter of context (i.e. a school or even a classroom), which determines which tale applies where. This situation calls for substantiating an evidence base for CLIL in Germany.  Taking into account the grassroots character of both CLIL practice and research in Germany, it is hardly surprising to find the existing evidence base regarding CLIL to be highly diverse. Therefore the central issue in future CLIL research is: Which approaches will help to resolve the existing ambiguities and inconclusiveness? A brief methodological analysis of what evidence base is there might point the way. This analysis will concentrate on German CLIL research. It remains to be discussed in how far the results of this analysis apply to CLIL research from other parts of the world. In terms of their social structure, classrooms and schools in Germany have two important features. Because of countless reform measures undertaken by educational authorities or schools themselves, the last two decades have seen many considerable changes after a period of relative stability. This is particularly true for CLIL as it has been a crucial part of this reform movement itself. Also, there has been considerable devolution in the area of education, having led to increased school autonomy, which in turn has remarkably increased heterogeneity within the educational sector.

Page 67
Page 68

These two features, i.e. change after periods of stability and heterogeneity, are typical of middle range theories (Kelle, 2008: 66ff.). On the one hand, these social structures possess some relatively stable characteristics that can be explained in terms of structural determinist ideas that in turn imply that social processes follow universal laws that research needs to identify. This is referred to as the macro-perspective. On the other hand, they possess crucial features that can only be elucidated by taking into account the idea of agency, which means that participants transcend external boundaries and transform structures and systems (ibid.: 73). This is referred to as the micro-perspective. Whereas the first set of characteristics lends itself to quantitative research procedures and hypothesis testing, the second set calls for a qualitative approach that aims at reinterpreting interaction and constructing meaning based on the participants’ point of view. Therefore, middle range theories can only be developed properly, if macro- and micro-perspective are taken into account, which means that quantitative and qualitative research procedures are combined.

This does not only help to elucidate the phenomenon as fully as possible, but it also resolves the two approaches’ respective shortcomings. Quantitative approaches tend to turn a blind eye to the context sensitivity of models and theories as well as to the question, how a certain phenomenon, such as discrimination, is actually created in social practice. Qualitative approaches in turn may underestimate regularities and weak causal relations as well as misjudge a study’s scope (ibid.: 78). Therefore, only if qualitative and quantitative approaches are combined in an integrated way, can they compensate for each other’s blind spots.

Existing CLIL research does indeed provide both kinds of studies. The quantitative bird’s eye view is provided by the DESI (op. cit.) study – and to a certain degree by other language studies (e.g. Bredenbröker, 2000) – that treat competence and classroom structure as aggregate concepts. The drawback of this quantitative approach is limiting its perspective to a functional-pragmatic concept of competence (cf. section 5.1) and not going beyond a predetermined set of categories used for classroom analysis. Therefore, it cannot establish conclusive links between outcomes and classroom practice, because it does not elucidate the construction of meaning or the participant perspective. This is the quantitative bias.

It is exactly these processes that the above-mentioned qualitative studies emphasise. Therefore, the CLIL community possesses insights into teachers’ subjective theories, their biographical resources and even some consequences of these for classroom practice (e.g. Dirks, 2004; Viebrock, 2007). These insights, however, concern only selected subjects and have not been linked to extensive classroom observation or output studies. Also, the above-named studies have not reached theoretical saturation, nor do they include cross-sectional statistical data regarding CLIL teachers. Therefore, we cannot really draw more general conclusions, such as specifying needs for teacher education. The same is true for learner-focused process studies (e.g. Bonnet, 2004; Heine, 2007; Lamsfuß-Schenk, 2008). They have provided important insights concerning processes of negotiation of meaning and their effects and prerequisites. Again, though, these insights are only partially linked to competence models that would create links to existing large-scale research or to classroom research that would in turn create links to classroom practice. This is the qualitative bias.

Table 1. A taxonomy of approaches to process-oriented classroom research.

Quantitative Qual./quant. Qualitative
Interaction analysis Discourse analysis Conversation analysis Variable
Structuralist interactionist integrative

Existing classroom research, particularly the analysis of Austrian CLIL classrooms (Dalton-Puffer, 2007) but also the study of TEFL classrooms in Germany (DESI, 2008), has shown that both CLIL and TEFL classrooms can be very teacher-centred places in which higher order thinking skills and the related discourse functions are less present than expected. This existing research is more focused on surface than deep structures, though. Therefore it would have to be located on the low inferential and mainly quantitative end of the scope of classroom research (Table 1) (cf. Walsh, 2006). These studies generate profiles of classrooms focusing on predetermined and very selective issues, but they do not enlighten the communicative processes of the construction of meaning or institutional aspects such as power relations. They do determine whether a certain phenomenon can be observed but do not really establish how this phenomenon comes into existence in social practice or what it means to the participants. A major step towards an integrative approach is a Swiss study into CLIL (Badertscher, Bieri, 2009), which establishes links between product and process and leads to interesting insights. The study finds that CLIL-students’ (grades 4 to 9) performance in biology, geography and history equals non-CLIL controls, although the CLIL-students’ L2-competence is significantly lower than the L1-competence of the controls (ibid., 105). What the authors find very remarkable is that CLIL-students have difficulties in verbalising concepts in their L2 which was the language of instruction when the concepts in question were discussed in the CLIL-classroom. They use a social constructivist framework to explain their findings and conclude that in the course of the cognitive processes of concept formation the close link between ideas and particular languages breaks up, or as Vygotsky puts it “evaporates” (ibid., 197).

Page 68
Page 69

This methodological analysis of existing CLIL research shows that establishing the missing link between products, processes and participants requires an integrative research strategy which solves the macro-micro-problem. In order to integrate these “3Ps” it is important to collect data on products, processes and participants at the same time, in order to be able to establish their interrelation. During this process, both a qualitative and a quantitative approach have their individual strengths and weaknesses. Therefore, it seems to make sense to combine not just the “3Ps”. It is also necessary to integrate the “2Qs” in a complex way, transcending the well-established consecutive approach of restricting qualitative research to exploration and quantitative research to hypothesis testing. A more flexible approach would allow for turning this upside down or even creating new combinations, also paving the way for reconstructive approaches to learners’ competence (cf. Bonnet, 2009; Martens, Asbrand, 2009).

4. An integrative approach to educational research

Accepting the methodological analysis put forward above, integrative CLIL research needs to determine: (1) the outcomes of CLIL classrooms taking into account learner performance in a manner that satisfies CLIL’s complex educational goals, (2) classroom processes in terms of analysing the classroom’s social practice in order to reconstruct its content and interaction related structure (cf. section 4.4), (3) the classroom participant perspective in order to establish the participants’ attitudes or beliefs that at least partly explain their way of acting in the classroom. The methodological analysis has argued that in order for validity to be maximised, both micro- and macro-perspective need to be taken, which in turn means that qualitative and quantitative research approaches need to be combined. The following section will show how the documentary method can be used as the hub of such an integrative research strategy.

4.1 The perspective of educational science: Theories of school and Bildung

Looking at CLIL classrooms from an integrative perspective means that the complex relationship between products and processes is reconstructed as fully as possible. This means that the institutional nature of classrooms needs to be taken seriously and that students and teachers are not only studied in terms of measurable content input or outcome but as reflexive human beings. Therefore, an integrative perspective on CLIL needs to possess theoretical roots that nurture it with adequate concepts of school and education. A viable understanding of the first is supplied by school theory (e.g. Fend, 2006) that describes school as possessing two fundamental functions. On the one hand it has to pass on successful cultural concepts and techniques from one generation to the next. This is its ‘conservative’ function. On the other hand, it has to allow for new solutions to emerging problems. This is its ‘innovative’ function. Both perspectives are negotiated in classroom interaction and empirical research tools need to be able to detect or rather reconstruct the constructed meanings.

A viable understanding of the second function of education is supplied by a theory of learning that goes beyond knowledge transfer. This is true for a theory of learning that allows for reflexive components and that distinguishes between different levels of learning (Bateson, 1972). In this model, learning on level one is identified as the accumulation of information within existing frames, whereas on level two, learning transforms individual frames as such, and on level three, a person becomes capable of choosing between frames on the basis of his or her contextual fit, thus becoming reflexive. The higher levels of this model describe processes that in the Germanophone tradition have been identified as processes of Bildung (often translated as “formation”) that can be understood as a transformation of a person’s world- and self-concept, a view that has been developed within biographic research (Marotzki, 1990). This idea of education as transformation fits the twofold function of school as a social institution –level one corresponding to the conservative and levels two and three corresponding to the innovative function. And this idea also matches the concept of education as intergenerational communication (e.g. Peukert, 1998) and of learning as accomplishing developmental tasks (Havighurst, 1953, Trautmann, 2004) which are at the heart of research into learner development and educational experience (Bildungsgangforschung) (e.g. Meyer, 2010). Taking very seriously the postmodern experience that biographical and ethical decisions are no longer predetermined by standardised biographies and monolithic moral imperatives provided by grands récits, this approach to educational research considers developmental tasks the interface between individual aspirations and needs on the one hand and societal demands on the other. If school wants to be relevant in this process, it needs to emphasise its innovative function by providing ample space for negotiation of meaning with respect not only to content, but also on the level of syllabus decisions and its own institutional structure – in other words fostering learner autonomy on the levels of classroom and curriculum (Benson, 2001: 151 ff.). Classrooms can be understood as places of complex negotiations of meaning. The meanings that emerge are informed by experience that is made by students and teachers both in- and outside school. An integrative approach to empirical classroom research needs to provide concepts and tools to access and model the processes, the products, and the factors influencing these constructions of meaning.

Page 69
Page 70

4.2 The perspective of social science: Sociology of knowledge

This idea of classrooms as spaces of interactive negotiation of meaning that both transmit existing and create new knowledge conforms with the position of symbolic interactionism, which argues that humans act according to what things or events mean to them. These meanings are not inherent in the objects but they are given to these objects in interaction and can be changed at any time (Blumer, 1969). Many approaches, such as grounded theory work on the basis of this fundamental assumption and try to research the construction of meaning empirically. As far as classroom research is concerned, the sociology of knowledge (Mannheim, 1985, first: 1936) and the documentary method (Bohnsack, 2010) seem to be a very well suited option, because it (a) possesses an enormous methodological clarity based on its total conception of ideology, (b) has developed a methodically both versatile and rigid procedure of analysis that takes the perspective of the social agents very seriously, and (c) it distinguishes clearly between different levels of meaning that are relevant in classroom interaction.

The sociology of knowledge tries to come to terms with the observation that at the beginning of the 20th century, the discourse in the domain of politics but also in other areas of public life, became more and more radical and confrontative with opponents that were less and less able to relate to each other, a constant non-understanding (Aneinandervorbeireden) becoming normal. This led Mannheim to believe that what we know is heavily influenced by the perspective we take. This fact, that all knowledge is contextually embedded and that therefore all knowledge is necessarily ideological is his total conception of ideology. This means that ideology-free interaction is impossible. Instead, the totality of the world is made up of different aspects, each created by a certain perspective.

Were the choice of perspective haphazard, the resulting knowledge would be accidental which would result in a total relativism. Instead, Mannheim argues that what we know is determined by our conscious intentions and more or less subconscious aspirations or emotions that in turn are shaped by the social context we have been raised in. This is what he calls relationism: Knowledge can be related to experience. In order to solve the discursive stalemate of non-understanding sketched above, Mannheim suggests the so-called Zurückfragen. This meansthat Zurückfragen abandons fruitless content-related disputation taking instead two reflexive steps: (a) to reconstruct the perspective from which a certain person speaks (descriptive) e.g. by analysing the metaphors s/he uses, and (b) to establish the limits and blank spots of this perspective in order to determine its epistemological range. This perspective has epistemological repercussions because it rules out the superiority of any objectivist position and highlights that epistemology itself is by no means unequivocal. Therefore, the sociology of knowledge was designed to be an empirical method, and Mannheim particularly drew on arts history to find an empirical method in order to reconstruct the location of thoughts in epoques and social contexts.

4.3 Empirical turn: The Documentary Method

This strive for an evidence base which Mannheim only completed on a macro level, is the point of departure of the documentary method. It aims at reconstructing the ‘implicit or ‘atheoretical’ knowledge underlying and orientating habitualised social action (…).’ (Bohnsack, Pfaff, Weller, 2010: 21). First devised by Mannheim and then used by ethnomethodology, the documentary method has been decisively advanced since the 1980s, when it was first applied to group discussions. Meanwhile, it has been adapted to investigate interviews, pictures and videos. Taking a constructivist position and aiming at second order observations, the method analyses interactional performance in order to reconstruct from this a generative deep structure that causes peoples’ actions. There are four concepts that can be considered the cornerstones of the method: (1) its distinction of three levels of meaning, (2) frames of orientation, (3) focusing metaphors, and (4) conjunctive spaces of experience. These four concepts will be explained in turn.

Page 70
Page 71

What the method aims at most of all is the reconstruction of the so-called documentary meaning. It is the textual representation of the implicit, atheoretical knowledge held in common by a given social group, because they share a so-called conjunctive space of experience, which means that they have been in structurally similar situations, such as war, migration or poverty. On the basis of their shared worldview these people can relate to each other without making their perspective explicit. This tacit mutual comprehension, based on shared tacit knowledge is called “understanding” and the underlying connection between these people is called “conjunctive relationship”. If there is no shared tacit knowledge, which is the case between researcher and participants, comprehension is based on “interpretative” reconstruction – i.e. “the theoretical explication of the mutual implicit or intuitive understanding of the participants” (Bohnsack, 2010: 104) – and the resulting relationship is called “communicative”.

When people with a conjunctive relationship come together for group discussions, their interaction tends to show various culmination points in which the discussion becomes remarkably dense. This can reveal itself through common metaphors that the participants use (metaphorical density) and/or through accelerating turn taking in which participants rapidly complement and complete each others’ contributions (interactive density). These passages are referred to as ‘focusing metaphors’. These are the points of departure for analysis, because this is where conjunctive atheoretic knowledge reaches the performative surface. The underlying generative deep structure that governs the participants’ actions consists of bits of information, emotions, strategies, beliefs, etc. that add up to a person’s tacit or habitual knowledge, which is referred to as a person’s ‘frame(s) of orientation’. Apart from this documentary meaning, texts also comprise intentional meaning, i.e. the consciously pursued goals of a participant, and immanent or objective meaning, i.e. the generally accepted significance of an action such as the gesture of a bow that expresses respect for somebody.

Figure 1. Schematic flowchart of the steps of analysis required by the documentary method. Social practice (top left) is the point of departure which becomes the object of analysis which aims at attributing social practice to conjunctive spaces of experience (bottom left).

Figure 1

It is this clear distinction between different levels of meaning, which is particularly relevant in institutional contexts, that qualifies the documentary method for school and classroom research. Moreover, this method has also been formalised to a great extent and emphasises the principles of sequential analysis and constant comparison within and across cases. This methodological and methodical rigidity ensures intersubjective results. The analysis takes four steps: (1) The formulating interpretation reconstructs the “what”: sequence and paraphrase of the steps of interaction. (2) The reflecting interpretation reconstructs the “how”: Passages that are of significance on the level of content or that show metaphorical or interactive density are analysed for their content (e.g. metaphorical analysis) and structure (e.g. types of discourse) in order to disclose frames of orientation. (3) The discourse description presents the structure of each case by explaining its frames of orientation. (4) Two levels of cross-case-analysis can then yield a typology and lead to findings with a certain generalisability.

Page 71
Page 72

4.4 Connecting Documentary Method and classroom research

Having described the documentary method and the sociology of knowledge and having linked them to underlying theories on learning and the institutional framework of schools, the question remains what the concrete interface of these theories with classroom research is supposed to be. It has become clear that the rigid differentiation between and methodological grasp of content and interactional structure, of the “what” and the “how” of the analysed discourse is the central strength of the documentary method. It is exactly this duality that is at the heart of a very creative model of classroom research (Erickson, 1982). This model considers the non-determinist relationship between planning and classroom reality and the resulting uncertainty a necessary quality of any teaching and learning and conceptualises this using the musical metaphor of improvisation: Teachers and students bring goals, beliefs and even determinations to the classroom, but all these cannot be more than themes over which the class has to improvise.

Figure 2. Schematic representation of the interface between documentary method and classroom research: the frame of orientation from the documentary method is combined with the structural elements ATS and SPS from Erickson’s (1982) model of classroom research.

Figure 1

The theme comprises both content and interaction structure. Whereas the content aspect, which influences the cognitive activities of the participants, is referred to as ‘academic task structure’ (ATS), the interactional patterns, which influence the emerging discourse, are called “social participation structure” (SPS). If this planning rationale is turned upside down it becomes an analytical approach. Then the nature of a given classroom can be established by reconstructing its ATS and SPS. As the differentiation of ATS and SPS fits the documentary method’s differentiation of content and structure very well, the two approaches are an almost perfect match (figure 2). Whereas elements of the ATS become already visible in the formulating interpretation, the SPS can only be reconstructed in the course of the reflecting interpretation. This combination requires some adjustments of the concepts of the documentary method. They do not change the basic approach or conceptual fundament. Therefore, these alterations need not to be discussed here (cf. Bonnet 2009).

5. Making CLIL research integrative

Integrative research as outlined above needs to bring together products, processes and the participant perspectives. Each of these areas will be discussed separately, analysing in how far CLIL research has put this integration into practice yet, what role the documentary method plays, and what needs to be done in order to further this process.

5.1 Product

As far as competence is concerned, the available models mainly draw on two traditions (cf. Pfaff, Krüger, 2009). On the one hand, there is a functional-pragmatic approach that conceptualises competence as the performance in problem-solving situations, thus reducing it to an observable surface phenomenon. This perspective is taken by the quantitative approach. On the other hand, there is a reflexive-emancipatory approach that models competence as a deep structure that affects a person’s actions across situations and that is linked to a person’s emotional, volitional and reflexive dispositions. This understanding would also fit qualitative approaches. If one combined the two, competence would comprise at least three components. (1) It would be the ability to detect, define and solve domain-specific problems. (2) It would also include the students’ reflection of the epistemological grounds of the related discipline, in order to work out the relevance of the subject to themselves. Much research – e.g. that into science education – has shown that this notion of relevance or irrelevance is crucial to students’ competence acquisition and heavily influenced by their socialisation. (3) And it would include subject related dispositions for cognitive or physical activities, that are embedded in a person’s habitus (e.g. Martens and Asbrand, 2009), such as complex notions of intercultural competence that satisfy current ideas of culture as a transcultural or hybrid phenomenon (e.g. Welsch, 1999; Bhaba, 1994).

Page 72
Page 73

As far as CLIL research on competences is concerned, there are already visible traces of this integrative perspective. Whereas research into language competence is almost entirely done from a functional-pragmatic point of view, in the area of subject matter competence the attempt has been made to combine a functional-pragmatic and a reflexive-emancipatory perspective (Bonnet, 2004). The documentary method with its concept of the frame of orientation was used to determine students’ chemical competence as a complex, reflexive and interactive phenomenon. This was combined with competence-scales from quantitative studies in order to create a link between this set of holistic interaction data and existing large scale results. As far as students’ subject matter competence is concerned, the distribution of competence levels of the CLIL students was fairly similar to that of non-CLIL students analysed in comparable studies. A second very interesting approach (Zydatiß, 2007) looks at competence from an entirely quantitative point of view. Nevertheless, this approach also succeeded in creating a complex notion of competence, integrating the domains of language and subject matter knowledge.

Therefore, the documentary method can indeed be seen as a hub for integrating a functional-pragmatic and a reflexive-emancipatory approach to competence, because it provides two points of convergence. First, the idea of the frame of orientation is similar to the reflexive-emancipatory concept of competence in that it is also located on a deep structural level and as it includes ideas, beliefs, procedures, etc. that guide individuals’ actions. Second, in the same way as competence, this structure is brought about by social practice in conjunctive spaces of experience. Therefore, reconstructing the students’ frames of orientation – particularly its ATS part – elucidates students’ competence and shows the context in which it is situated. The punchline for CLIL research is that this context is nothing less than the relevant cultural background. This is very relevant to CLIL, because recent articles (e.g. Breidbach, 2007; Hallet, 2004; Marsh, 2009) have argued that these approaches still wait to be included in empirical research on CLIL. Accepting that content, cognition, communication, and culture as described in the model of the 4Cs (e.g. Coyle, Hood, Marsh, 41 ff) are crucial categories to model CLIL, the documentary method can make a major contribution to CLIL research: On its own, it can be used to analyse content, cognition and culture. If related instruments to analyse language competence (cf. above) are added, communication can also be analysed. Therefore, a combination of documentary method and additional instruments as specified above does not only integrate the “2Qs” (qualitative and quantitative approach). It is also suitable for researching teaching and learning practices as well as student performance in relation to all 4Cs (content, cognition, communication and culture).

5.2 Process

Whereas the integration of the 2Qs seems rather promising in the area of CLIL product research, the situation with respect to process studies is less advanced. Whereas there are both quantitative (e.g. DESI, 2008) and qualitative (e.g. Lamsfuß-Schenk, 2008) as well as intermediate (e.g. Dalton-Puffer, 2007) studies, these studies have yet to have been integrated. Therefore, there are either overviews from a macro-perspective or deep insights from a micro point of view. Nevertheless, there are also studies that explore the integration of process and product.

On the one hand, the documentary method was used for an interpretative sequential analysis of how small groups construct content meaning in chemistry (Bonnet 2004). The outcome of this strand of analysis was fourfold. (1) Although the documentary analysis was backed up by a criteria referenced tool focusing on reflexivity with respect to language and subject matter, traces of reflexivity were scarce. Therefore, the commonly held idea of CLIL classrooms automatically being more reflexive and therefore more intercultural places could not be substantiated.  (2) The second finding concerns the role of a common lingua franca such as a majority language. On the basis of participant observation of an individual lesson, it has been suggested that this lingua franca can be used to bridge content gaps and work as a “conversational lubricant” (Butzkamm, 1998). In contrast to this, the documentary analysis, doubled by an analytical tool to identify any sort of code mixing or switching, showed that the lubrication does not work on a conceptual level. Code mixing or switching from English to the common first language German often indicated problems in understanding and using content. A change in the language of communication did not help the students to find a solution. This was rather brought about by approaching the problem from a different content or interactional angle, which was always done in English again. Therefore, it seems to be the case that a common lingua franca such as a majority language seems to have the potential to keep classroom interaction going. As far as serious content problems – particularly on a conceptual level – are concerned, however, other problem-solving and scaffolding strategies are necessary. (3) This leads to the third outcome, a model of factors leading to successful competence acquisition in CLIL classrooms, particularly if they are organised in a cooperative way (Bonnet, forthcoming). It is based on the notion that successful negotiation of meaning is the key element of CLIL classrooms. Conceptualising this from both a constructivist (viability check) and interactionist (emergence) perspective suggests that students need three core competences to succeed in CLIL classrooms: foreign language competence, subject matter competence and interactional competence. The model elaborates these competences and describes teaching strategies to foster their acquisition.

Page 73
Page 74

A different qualitative approach was taken by Lamsfuß-Schenk (2008). She used an inductive coding procedure to relate students’ interaction and work on material to their cognitive processes of Fremdverstehen. She found that during classroom discussions in a bilingual history group the reconstruction of self and otherness was more profound and complex than in a non-bilingual control group. She also concluded that this depth of elaboration was caused by the linguistic and cognitive difficulty of the sources that necessitated more intense reading, paraphrasing and cognitive processing rather than by an assumed authenticity of the texts. She also found that students needed thorough scaffolding and developed a greater repertoire of learning strategies. Together with the findings from Bonnet (2004), this suggests that an added value of CLIL in the area of subject matter competence is by no means a given. Particularly, the reflexive dimension of competence which reflects personality-relevant processes is not automatically achieved. Students only reach this level of competence if appropriate teaching methods are used and learning environments are created that provide linguistic and interactional scaffolding.

Although, these findings provide interesting insights, the potential of interaction research has by no means been exhausted. What CLIL research in this area is lacking is the reconstruction of the social participation structure and academic task structure of classrooms in a comprehensive way that overcomes the restrictions of limited perspectives and foci. Therefore, the combination of the existing approaches seems to be promising. The documentary method could again serve as the basis of analysis, because its tackling of both interaction and content has the potential to deliver deep insights into classroom structure. Those studies applying a coding or even quantitative approach (Lamsfuß-Schenk, Dalton-Puffer, DESI), however, contribute valuable tools that focus on CLIL-specific phenomena, such as discourse functions or scaffolding. Combining the two would not just show the distribution of these phenomena but also allow for insights into which kinds of classroom structure support or suppress them.

5.3 Participant perspective

With respect to the participant perspective, German CLIL research has traditionally focused on teachers and predominantly taken a qualitative point of view (Dirks, 2004; Viebrock, 2007). The study by Dirks (2004) has first used the agency-oriented reconstructive approach outlined above for participant research in CLIL and therefore takes a thoroughly socio-cultural perspective. The study by Viebrock is taking a more psychological stance trying to reconstruct teachers’ theories and beliefs. These studies draw a complex and partly contradictory picture. On the one hand, CLIL is seen as a very dynamic and innovative field that attracts teachers with a thoroughly cultural outlook on teaching, often resulting from their own experience abroad. Another positive feature is that teachers seem to spend more time on lesson planning and use content and language scaffolding much more extensively than in non-CLIL classrooms. On the other hand, however, teachers themselves utter doubts as to how much the students actually learn. One teacher from the sample of Viebrock (2007) openly questions that his students really acquire problem solving competence. He has the impression that thorough language support leads to the students reproducing chunks rather than developing cognitive concepts in the relevant domains. Also, the studies identify two strategies that threaten success. Dirks identifies a teacher type whose culture driven individual orientation borders on idiosyncrasy and creates incoherence. Viebrock finds that quite some CLIL teachers seem to lean towards more instructivist teaching strategies, particularly if content difficulty is perceived to rise.

Where there is student research, it has been done quantitatively (Zydatiß, 2007). The findings suggest that CLIL schools’ greater catchment areas provide them with a considerably higher percentage of students with high socio-economic backgrounds. These students profit from this in a twofold way. On the one hand they possess higher levels of self-confidence on entrance. On the other hand, their self-efficacy is further enhanced by their CLIL experience: Although CLIL classrooms tend to be competitive places they entitle their participants to considerable social esteem as successful CLIL students are perceived as achievers.

Page 74
Page 75

The look at these findings shows a complex result. On the one hand the reconstructive approach outlined above has already been present in participant teacher research. This means that this area is well prepared for integration using the documentary method. In a current study (cf. Apsel in this volume), trying to shed light on the case of CLIL dropouts, this is also applied to the student perspective and combined with quantitative data on the schools they attend. The case of quantitative student research, however, shows that using the tools of pedagogical psychology and quantitative professionalisation research could contribute considerably to providing an idea of cross-sectional distributions of features that are very relevant to CLIL. This quantitative approach would also help to monitor the sampling and to establish some idea of the distribution of the cases across the population.

5.4 Integrating Ps and Qs in CLIL research

The documentary method has been successfully used in product-, process-, and participant-research by different researchers. Therefore, it can be considered a powerful tool to integrate research into these three areas, because it can potentially elucidate all three phenomena. The interesting connections between findings from existing participant and process research are a mouth-watering hint of the potential of a really coherent integration of the “3Ps”. In this full-fledged integration, the analysis of classroom structures (Bonnet, 2009) would be the hub around which documentary reconstructions of the participants’ perspective and of their relevant competences are grouped and that serves as a point of reference – i.e. the tertium comparationis.

The discussion of relevant CLIL research suggests that in all three areas of research, quantitative or at least coding approaches considerably contribute to understanding CLIL classrooms. As far as competences are concerned, the careful matching of qualitative determinations of student competence with quantitative scales creates links to existing large-scale studies. They can be used to locate students’ competences determined from qualitative samples within the overall population of large-scale studies. As far as participant research is concerned, quantitative tools help to shed light on crucial areas of CLIL research, such as the role of socio-economic or emotional factors. In process research, qualitative coding approaches have helped to classify task-based learner activities.

Creating links between qualitative and quantitative data is problematic, because both may draw on different theoretical frameworks. Therefore, a closer look at competence research will show problems and possible solutions for this. The documentary method, including subject matter competence research, has demonstrated considerable potential for the integration of quantitative and qualitative research. To move forward researchers need to combine carefully quantitative and qualitative tools so as to link the micro and macro levels of research. The full potential of this integrative approach can only be realised, if the quantitative tests are based on the complex concepts of competence mentioned above. Only then can their results be fully cross-checked and related to the frames of orientation which have been reconstructed based on classroom-interaction data. Unfortunately, the competence models used in existing large-scale studies are still limited in this respect.

As far as language competence is concerned, the situation is potentially similar, because the reconstruction of the SPS part of the frames of orientation includes linguistic features. Nevertheless, the documentary analysis concentrates on those features and passages that can be connected to constructions of meaning or that reveal the structure of the discourse of the analysed group. This means that neither step of the documentary method includes a comprehensive analysis of individual language performance across all relevant categories. This analysis requires additional steps, such as the following procedure: If classroom data are rich enough (i.e. long phases including student-student interaction), they can be analysed for lexical, syntactic and other features using established methods from applied linguistics. As is true for subject matter competence, this would provide an interface to link classroom interaction to competence data, gathered in individual tests.

All in all, the attempt to use the documentary method in order to create integrative approaches to CLIL research has been most successful with respect to subject matter competence. As many quantitative participant studies look at teachers from a point of view of their competence, this approach seems transferable to participant research. This view is corroborated by a look at quantitative student research: the emotional concepts (e.g. self-efficacy) used there also appear in the frames of orientation reconstructed by qualitative studies which means that links can be created. What remains to be established, though, is how or even whether at all existing criteria referenced tools for analysing classroom processes bear significant potential for integration.

Page 75
Page 76

6. Conclusion

The discussion of current German studies on CLIL has come to the conclusion that creating an evidence base is currently the paramount goal of CLIL research. The methodological analysis has argued that this requires a twofold integration. This means bringing together the “3Ps”, i.e. product, process and participant perspective, which in turn implies combining the “2Qs”, i.e. quantitative and qualitative approaches. A closer look at the sociology of knowledge and the documentary method has revealed that this methodological programme is a suitable framework for integrative CLIL research, because it can yield relevant and innovative insights in key areas of CLIL classroom practice. Not only does it allow for the pursuit of crucial issues such as the link between competence and classroom structure, it is also suitable for taking into account the concept of culture which is an essential issue in CLIL classrooms.

Furthermore, it has been argued that quantitative instruments should also be used to widen the scope of research and to create links to existing large-scale qualitative studies. This provides an important second perspective in order to judge the scope of qualitative samples adequately and to place the reconstructed findings within the overall population. These quantitative instruments need themselves to be integrative (Zydatiß, 2007), they should be open to inductive category building (Lamsfuss-Schenk, 2008) and have the potential of including expertise from other fields, such as pedagogical psychology (Zydatiß, 2007).

Furthermore, this approach may also provide valuable concepts for integrating the existing evidence base. Because of disparities among the studies involved, the integration of existing research findings requires a thorough analysis of the respective methodologies. The methodological analysis carried out here with its distinctions of the “Ps” and “Qs” as well as of the two fundamental approaches to competence (cf. section 5.1) and the taxonomy of classroom research (cf. section 3) could serve as a point of departure for the required ‘meta-analysis’ of CLIL research, because it provides an instrument to map and group existing approaches also taking into account their methodological basis.

Page 76
Page 77


Baetens-Beardsmore, H.: 2010, The Brain – An Educational Perspective. Presentation, given on CLIL 2010 – In Pursuit of Excellence, Eichstätt, Germany, 30 September – 2 October 2010.

Badertscher, H., Bieri, T.: 2009 Wissenserwerb im Content and Language Integrated Learning: empirische Befunde und Interpretationen. Haupt, Bern/Stuttgart/Wien.
Bateson, G.: 1972, Steps to an Ecology of Mind: Collected Essays on Anthropology, Psychiatry, Evolution, and Epistemology. University of Chicago Press, Chicago.

Benson, P.: 2001, Teaching and Researching Autonomy in Language Learning. Pearson Education, Harlow.

Bhabha, H. K.: 1994, The location of culture. Routledge, London.

Blumer, H.: 1969, Symbolic Interactionism – Perspective and Method. University of California Press, Berkeley.

Bohnsack, R.: 2010, Documentary Method and Group Discussions, in R. Bohnsack, N. Pfaff and W. Weller (eds.), Qualitative Analysis and Documentary Method in International Educational Research, (99-124). Barbara Budrich, Opladen & Farmington Hills.

Bohnsack, R., Pfaff, N. and Weller, W.: 2010, Reconstructive Research and Documentary Method in Brazilian and German Educational Science – An Introduction, in R. Bohnsack, N. Pfaff and M. Weller (eds.), Qualitative Research and Documentary Method in Educational Science – Results from Brazilian-German Cooperations, (7-40). Barbara Budrich, Opladen & Farmington Hills.

Bonnet, A.: forthcoming, Language, Content, Interaction – How to make CLIL classrooms work, H. Böttger, D. Marsh and O. Meyer (eds.), Cutting Edge – the interface of quality issues in CLIL – insights into key questions and issues – selected contributions from CLIL 2010: Towards Excellence.

Bonnet, A.: 2009, Die dokumentarische Methode in der empirischen Unterrichtsforschung – Ein integratives Forschungsinstrument für Strukturrekonstruktion und Kompetenzanalyse, ZQF – Zeitschrift für qualitative Forschung 10/2, 223-240.

Bonnet, A.: 2004, Chemie im bilingualen Unterricht – Kompetenzerwerb durch Interaktion. Leske und Budrich, Opladen.

Bredenbröker, W.: 2000, Förderung der fremdsprachlichen Kompetenz durch bilingualen Unterricht. Peter Lang, Frankfurt am Main.

Breidbach, S.: 2007, Bildung, Kultur, Wissenschaft. Reflexive Didaktik für den bilingualen Sachfachunterricht. Waxmann, Münster.

Bruton, A.: 2011. Are the differences between CLIL and non-CLIL groups in Andalusia due to CLIL? A reply to Lorenzo, Casal and Moore (2010) Applied Linguistics 32/2, 236-241

Butzkamm, W.: 1998, Code-switching in a bilingual history lesson: The mother tongue as conversational lubricant, International Journal of Bilingual Education and Bilingualism 1/2, 81-99.

Clapham, C. M.: 1996, The Development of IELTS: A Study of the Effect of Background Knowledge on Reading Comprehension. CUP, Cambridge.

Coyle, D., Hood, P., Marsh, D.: 2009, CLIL-Content and Language Integrated Learning. CUP, Cambridge.

Dalton-Puffer, C.: 2007, Discourse in Content-and-Language-Integrated Learning (CLIL) classrooms. Benjamins, New York.

DESI-Konsortium (ed.): 2008, Unterricht und Kompetenzerwerb in Deutsch und Englisch: Ergebnisse der DESI-Studie. Beltz, Weinheim.

Dirks, U.: 2004, „Kulturhüter” oder „Weltenwanderer”? – Zwei „ideale“ Realtypen bilingualen Sachfachunterrichts, in A. Bonnet and S. Breidbach (eds.), Didaktiken im Dialog – Konzepte des Lehrens und Wege des Lernens im bilingualen Sachfachunterricht, (129-140). Peter Lang, Frankfurt am Main.

Erickson, F.: 1982, Classroom Discourse as Improvisation: Relationships between Academic Task Structure and Social Participation Structure in Lessons, in L. C. Wilkinson (ed.), Communicating in the Classroom, (153-181). Academic Press, New York.

Fend, H.: 2006, Neue Theorie der Schule. Einführung in das Verstehen von Bildungssystemen.  VS Verlag für Sozialwissenschaften, Wiesbaden.

Gruschka, A.: 2011, Was der Gegenstand fachdidaktischer Bildungsforschung sein könnte. Presentation, given on Qualitative Unterrichtsforschung in den Fachdidaktiken, Mainz, Germany, 19 – 21 May 2011.

Hallet, W.: 2004, Bilingualer Unterricht als interkultureller Diskursraum, in A. Bonnet and S. Breidbach (eds.), Didaktiken im Dialog – Konzepte des Lehrens und Wege des Lernens im bilingualen Sachfachunterricht, (141-152). Peter Lang, Frankfurt am Main.

Havighurst, R. J.: 1953, Human Development and Education. Longmans, New York.

Heine, L.: 2007, Kognitive Prozesse bilingualer Lerner bei der fremdsprachlichen Verarbeitung von Fachinhalten. Dissertation, Universität Osnabrück.

Page 77
Page 78

Jäppinen, A.-K.: 2005, Thinking and Content Learning of Mathematics and Science as Cognitional Development in Content and Language Integrated Learning (CLIL): Teaching Through a Foreign Language in Finland, Language and Education 19/2, 147-168.

Kelle, U.: 2008, Die Integration qualitativer und quantitativer Methoden in der empirischen Sozialforschung. 2, Auflage. VS-Verlag, Wiesbaden.

Lamsfuß-Schenk, S.: 2008, Fremdverstehen im bilingualen Geschichtsunterricht. Eine Fallstudie. Peter Lang, Frankfurt am Main.

Mannheim, K.: 1985, first 1936, Ideology and Utopia. An Introduction to the Sociology of Knowledge. Harvest Book-Harcourt, San Diego.

Marotzki, W.: 1990, Entwurf einer strukturalen Bildungstheorie. Biographietheoretische Auslegung von Bildungsprozessen in hochkomplexen Gesellschaften. Deutscher Studien Verlag, Weinheim.

Marsh, D.: 2009, Introduction: Culture, Education & Content and Language Integrated Learning, in M. L. Carrió-Pastor (ed.), Content and Language Integrated Learning: Cultural Diversity, (11-30). Peter Lang, Frankfurt am Main.

Martens, M. and Asbrand, B.: 2009, Rekonstruktion von Handlungswissen und Handlungskompetenz – auf dem Weg zu einer qualitativen Kompetenzforschung, ZQF – Zeitschrift für qualitative Forschung 10/2, 201-217.

Meyer, O.: 2010, Towards quality-CLIL: successful planning and teaching strategies, Pulso: Revista de Educación 33, 11-29.

Peukert, H.: 1998, Zur Neubestimmung des Bildungsbegriffs, in M. Meyer and A. Reinartz (eds.), Bildungsgangdidaktik. Denkanstöße für pädagogische Forschung und schulische Praxis, (17-29). Leske und Budrich, Opladen.

Pfaff, N. and Krüger, H.-H.: 2009, Rekonstruktive Forschung zum Kompetenzerwerb in der schulischen und außerschulischen Bildung – Einführung in den Themenschwerpunkt, ZQF – Zeitschrift für qualitative Forschung 10/2, 191-199.

Schimak, F.: 2010, CLIL as a means to create cross-border-competence. Presentation, given on CLIL 2010 – In Pursuit of Excellence, Eichstätt, Germany, 30 September – 2 October 2010.

Trautmann, M. (ed.): 2004, Entwicklungsaufgaben im Bildungsgang. VS Verlag für Sozialwissenschaften, Wiesbaden.

Viebrock, B.: 2007, Bilingualer Erdkundeunterricht – Subjektive didaktische Theorien von Lehrerinnen und Lehrern. Peter Lang, Frankfurt am Main.

Vockrodt-Scholz, V. and Zydatiß, W.: 2007, Zur Interdependenz von Englisch- und Sachfachkompetenzen im bilingualen Unterricht, Zeitschrift für Fremdsprachenforschung 18/2, 209-236.

Vollmer, H. J.: 2009, Diskursfunktionen und fachliche Diskurskompetenz bei bilingualen und monolingualen Geographielernern, in S.-A. Ditze and A. Halbach (eds.), Bilingualer Sachfachunterricht (CLIL) im Kontext von Sprache, Kultur und Multiliteralität (165-185). Peter Lang, Frankfurt am Main.

Walsh, S.: 2006, Investigating Classroom Discourse. Routledge, London.

Welsch, W.: 1999, Transculturality – the Puzzling Form of Cultures Today, in M. Featherstone and S. Lash (eds.), Spaces of Culture: City, Nation, World, (194-213). Sage, London.

Wolff, D.: 2007, Bilingualer Sachfachunterricht in Europa: Versuch eines systematischen Überblicks, Fremdsprachen lehren und lernen 36, 13-29.

Wolff, D. and Marsh, D.: 2007, Diverse Contexts – Converging Goals. CLIL in Europe. Peter Lang, Frankfurt am Main.

Wolff, D.: 2010, Learner Autonomy. Presentation, given on CLIL 2010 – In Pursuit of Excellence, Eichstätt, Germany, 30 September – 2 October 2010.

Zydatiß, W.: 2007, Deutsch-Englische Züge in Berlin (DEZIBEL) – Eine Evaluation des bilingualen Sachfachunterrichts an Gymnasien.Kontext, Kompetenzen, Konsequenzen. Peter Lang, Frankfurt am Main.


1 Contact: Prof. Dr. Andreas Bonnet, Universität Hamburg, Fakultät EPB, Fachbereich Erziehungswissenschaft 4, Arbeitsbereich Englischdidaktik, Von-Melle-Park 8, 20146 Hamburg; andreas.bonnet (at)
return to text