Some thoughts on our aims and ‘orienting issues’

Myself and others at the LTRI are refining our approach to the evaluation by considering a design based research (DBR) perspective (e.g. Joseph, 2004). So I will present some of the current ‘orienting issues’ I’ve been considering based on our work so far that we can then collectively consider and elaborate towards the production of a refined work-plan that is more specific in terms of WP details, proposed technologies and developments, and specific evaluation methods etc. Specifically, the aim is to co-develop this approach to present a refined rationale and proposal at the next project meeting (in Insbruck, in 6 weeks). Obviously, this is work in progress and we anticipate it will be refined considerably as we progress and get a deeper understanding of the key conceptual ideas underpinning knowledge maturing and the candidate tools for realising it.

Design based research (DBR)

During the kick off meeting at Karlsruhe we suggested that the evaluation aspects could play a greater role in the design and development process, and hopefully improve it, rather than being seen as something that is performed ostensibly on delivered designs and implementations. Or ‘evaluate to design better’ as well as ‘evaluate the designed systems’. With this in mind we are considering starting the evaluation earlier and maybe re-thinking it. The proposed DBR approach utilises our expertise in ‘cognitive science’ approaches to learning technology design and development, which synthesise theoretical, empirical and system design work within a holistic tri-partite approach. Work I’ve been doing within the “Learning interaction and dialogue design (LIDD)” (http://www.londonmet.ac.uk/ltri/research/interaction.htm) research group at the LTRI epitomizes this approach, that I have been working on for over ten years, and across a broad range of research and development projects in interactive learning technology, see http://homepages.north.londonmet.ac.uk/~ravensca/.

At the meeting I argued for the relevance of this approach to address the fast changing and evolving nature of technology and related human practices, which necessitates more agile ways to evaluate and improve designs in an ongoing and continuous way. I suggested, half-jokingly, that what we need is a “kamikaze cognitive science” approach to design, implementation and evaluation in the digital age, so that evaluative insights influence technology design and development in a suitably timely (i.e. fast) manner.

According to the Design Based Research Collective, in a seminal issue of Educational Researcher (2003).

“The challenge of design-based research is in flexibly developing research trajectories that meet our dual goals of refining locally valuable innovations and developing more globally usable knowledge for the field.”

Which is what I understand is critical in MATURE. We want to develop specific tools such as the PLME and OLME that are situated and valuable in the contexts that they are being used whilst also developing more generic frameworks, such as knowledge maturing model and other aspects of the SoTA WP. We have recently demonstrated the application of a DBR approach to develop digital dialogue games for learning (Ravenscroft et al, 2008, www.interloc.org), although myself and others like John Cook have also applied the approach over ten years ago to develop intelligent dialogue systems (Ravenscroft and Pilkington, 2000; Cook, 2001). These initiatives seem to have strong parallels with MATURE, as we formally modelled effective dialogue processes to then design tools that supported and promoted its practice. And with MATURE we want to identify and model – technology mediated - social learning processes and behaviours in order to design tools that support these processes and practices in ways that correspond with knowledge maturing.

Theoretical foundations

The emphasis on social learning in MATURE, implies the relevance of socio-cultural approaches to learning that fully incorporate the role of digital technologies. Given the increasingly pervasive nature of technology, an approach which considers design-based research within a sociocultural scientific frame (e.g. Vygotsky, 1978; Wertsch, 1991; Engestrom, 1987) is particularly powerful for investigating and promoting contemporary learning, that is usually social and can often rely heavily on ‘learning dialogues’ conducted in contexts with varying degrees of formality. This stance emphasises the mediating role of technological tools and social relationships that these tools may operate within or give rise to in the context of contemporary learning practices. So this perspective is particularly relevant to the development of dialogue and social software and the ways these are articulated within the learning landscape. In particular, myself and others have conducted a lot of previous research about the role of digital dialogue in learning (e.g. Ravenscroft 2007; Cook and Oliver, 2002) and how we need to re-conceptualise theory (Ravenscroft, Wegerif and Hartley, 2007; Cook 2002) to improve our understanding of developing dialogue practices. It will be important and interesting to see how this maps onto the Distributed Cognition underpinning that is currently being applied to and informing the development of the knowledge maturing framework.

Knowledge maturing and social learning

Is knowledge maturing and social learning the same thing? After attending the first WP2 meeting in Bremen and reading the latest article submitted by Andreas and others, I have been thinking about the relationship between knowledge maturing and meaningful learning and the role of social practices and contexts in the digital domain. To link knowledge maturing to meaningful learning do we need to elaborate the KM model to include practices and contexts as well as contents, processes and semantics? Is marshalling contents, processes and semantics in terms of practices and contexts what amounts to meaningful learning? I am aware, if I’ve got this right, that this actually raises a somewhat philosophical and yet crucial question. In MATURE the aim is, I think - If I'm understanding the paper correctly, to decontextualise situated activities through abstractions in terms of content, semantics and processes - will this always be wise? Or in some cases will the process of abstraction remove the essence of what we are trying to represent? A related question is do our ontologies as they are currently defined need to be elaborated beyond their current form? e.g. to represent context and practices (as well as content, processes and semantics)?

Knowledge maturing and advanced learning design

We have already done some work in this respect in a book Chapter called “New Horizons in Learning Design” (Ravenscroft and Cook, 2007), that appeared in "Rethinking pedagogy for the digital age: Designing and delivering e-learning" and in another recent article (Ravenscroft 2008). Here we pointed out how current approaches to learning design fail to address the changing landscape of learning, that is becoming more personalised, informal, social and emergent - rather than the outcome of highly structured institutional practices. We've been tackling this in UK HE, and referred to our work that is located in this more progressive space as ‘advanced learning design’. But this challenge and the work we’ve done to address it seems particularly appropriate to work-based learning, that by its nature can be more event-driven, emergent and organic. In brief: What would learning designs for knowledge maturing look like? How would they relate to the current knowledge maturing framework? Can they be easily understood and reused across technical platforms and implementations?
A key related question here is: What sort of language or representations are suitable for advanced learning design and its evaluation? This is arguably key for MATURE, as a fundamental question is how do we ensure the knowledge maturing does correspond with learning, or in other words, how do we ensure that knowledge maturing corresponds, gives rise to, or represents 'real learning'. A corollary of the above is that if IMS-LD is too brittle for our purposes, or perhaps inappropriate, what are the alternatives and can we re-use a 'language of design' from elsewhere or do we need to develop our own? One possible way forward is based in recent work into ‘design patterns’, that capture identifiable learning designs based on ‘what people do’ rather than how an instructional organization manages things. Our work on dialogue games is an initial attempt to tackle this in the context of ’reasoned dialogues’.

Methodological considerations and proposals

A next step is to develop these orienting issues into a more specific and refined proposal for the evaluation WP and related work. E.g. we will:

• Take a closer look at the current thinking and papers on knowledge maturing;
• Consider how our cognitive science/DBR modelling methods (some of which are quite formal) and delivered technologies 'shape up' to the challenge of realizing and evaluating learning as knowledge maturing;
• Assess the suitability of the evaluation methods we have used in previous projects;
• Consider the particular technologies that are currently under consideration in the context of the PLME and OLME.

…so, to be continued

References

Cook, J. (2001). Bridging the Gap Between Empirical Data on Open-Ended Tutorial
Interactions and Computational Models. International Journal of Artificial Intelligence in Education, 12, 85–99. Abstract online: http://www.cogs.susx.ac.uk/ijaied/

Cook, J. and Oliver, M. (2002). Designing a Toolkit to Support Dialogue in Learning.
Computers and Education, 38 (1-3), 151–164.

Cook, J. (2002). The Role of Dialogue in Computer-Based Learning and Observing
Learning: An Evolutionary Approach to Theory. Journal of Interactive Media in Education http://www-jime.open.ac.uk/2002/5/cook-02-5.pdf

Digital Dialogue Games for Learning website, http://www.interloc.org/, accessed 16 May 2008.

Design-based research collective (2002). Design-Based Research: An Emerging Paradigm for Educational Inquiry, Educational Researcher, Vol 32, No. 1, pp 5-8.

Engstrom, Y. (1987) Learning By Expanding: An Activity Theory Approach To Developmental Research. Helsinki, Orienta-Konsultit.

Joseph, D. (2004). The Practice of Design-Based Research: Uncovering the Interplay Between Design, Research, and the Real-World Context. Educational Psychologist, 39(4), 235-242.

Ravenscroft, A. (2001). Designing e-learning interactions in 21C: Revisiting and re-thinking the role of theory, European Journal of Education: Special edition on On-line Learning, Vol. 36, No. 2, pp. 133-156.

Ravenscroft, A. (2007). Towards pervasive and personalised machine-mediated learning interactions for the digital age: Design, multimodality and mobility, Invited Paper for Special Issue of International Journal of Interactive Learning and Smart Education: Machine-Mediated Multimodal Communication (In Press).

Ravenscroft, A. (2007). Promoting Thinking and Conceptual Change with Digital Dialogue Games, Journal of Computer Assisted Learning (JCAL), Vol. 23, No 6, pp 453-465.

Ravenscroft, A. & Cook, J. (2007). New Horizons in Learning Design, Chapter 16 in Rethinking pedagogy for the digital age: Designing and delivering e-learning, Beetham, H & Sharpe, R. (Eds), Routledge, pp 207-218.

Ravenscroft, A. and Pilkington, R.M. (2000). Investigation by Design: Developing Dialogue Models to Support Reasoning and Conceptual Change, International Journal of Artificial Intelligence in Education: Special Issue on Analysing Educational Dialogue Interaction: From Analysis to Models that Support Learning, 11/1, 273-298.

Vygotsky, L. (1978). Mind and society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.

Walton, D. (1984). Logical Dialogue-Games and Fallacies, Lanham: University Press
America.

Wertsch, J.V. (1991). Voices of the mind: a sociocultural approach to mediated action’ London: Harvester Wheatsheaf.