User:Andy Roberts/defence
From Dar
This is a forked copy of User:Andy_Roberts/unh3602
Module unh3602
Student id 0264114
Validation and Defence
Contents |
Context
The research which is being validated and defended here is technically that which has already been formally written up as:
"An Action Enquiry into improving the practice of online research, investigating online communities and using internet technology (provisionally termed Distributed Action Research)"
Methodology, Literature review and pilot exhibition
And the online Exhibitions Main Exhibition, Barn raising, Pilot Exhibition.
The task is also to defend the conceptual proposition and practice of Distributed Action research itself.
Why this is being written on a wiki
As explained in Thesis on a wiki by Andy on the April 6th, 2006:
The next module which I have to complete as part of a year long dissertation through research is called “Validation and Defense”. One of the conclusions stemming from the online exhibition, and taking some of the comments into consideration, is that if the subject is supposed to be related to web2.0 in any way, then it isn’t really appropriate to offer a set of read-only pages with one-way podcasts and diagrams. Provision for engagement and feedback was via the comments functionality of a blog post, and while that did capture sufficient critical commentary, it isn’t fully appropriate. For example, Derek said:
- "the exhibition seems to be content only, no tagging, no intereaction (I like content and intereaction to be close)"
and Stephen Powell wrote
- "a downside … the ‘broadcast’ nature of it - that is I wanted to ask a ‘real time’ questions when I listened to the audio.”
After some thought, I entirely agree. In the past I did fleetingly toy with the idea of staging the exhibition on a wiki. Linda [L M Hartley, peer review partner] and I discussed it as well. Indeed, embracing the barn raising idea as part of the exhibition went some way towards that, but I would have held back from planning the entire thing to be hosted on a wiki for fear of pushing the innovative side of things just a little too far. My experience with institutional attitudes towards wiki has not been positive on the whole, and at first thought there would seem to be problems with attribution, drawing the line between collaboration and plagiarism, and presentation. I now believe that none of these drawbacks is insurmountable so my defense against the above criticisms is to respond by writing the whole of my next assignment, (the work for assessment that is, unlike the exhibition itself which wasn’t formally assessed) as a page or pages on a wiki.
The page is initialised on the DARwiki here:
http://distributedresearch.net/wiki/index.php/User:Andy_Roberts/unh3602
And at midnight on May 8th, 2006 the page will be locked and handed in. That means it will no longer be editable ever, but will remain as a snapshot of that which was marked. A 'forked' copy version of the page will remain open for editing indefinitely, thus fully allowing for divergent voices to be traced into the future .
So with that done, now I’m off to seek any precedents I can find online.
precedents
I'm interested to learn how many theses or undergraduate modules have been submitted as wikis already, and any conclusions from the process.
From a connectivist point of view, the first step (after a quick google search) is to ask the people I think most likely to know, so to that end I have edited my page on Meatball Wiki, throwing in the potential pattern language element "ThesisOnaWiki"
Found A wiki about which kind of bindings to use for your thesis! [1] another one: [2] This is more like it, part of the batchelor's thesis of Tobias Müller as a wiki - in german. [3] Asked mediawiki users forum [4]
Introduction
The research undertaken was an example of meta research (research about research) which investigated through action, the difficulties and advantages of conducting Action Research entirely online and began to create a resource about the practice which other internet researchers have found useful.
The original concept for this Action Research project stemmed from Roberts (2004), my year 2 Action Research project. At that time, I identified a distinct type of AR which I called Distributed Action Research (DAR). DAR was employed in that module itself and I defined it then as: “Research into or using online communities and digital tools.” The definition has been tightened during the course of my research to now read as "Action Research with distributed Communities of Practice", concisely explained within the Venn diagram below.
This project took an essentially emergent approach in terms of the exact nature of the cycles, actions and areas of focus, and the overall structure comprised of two main phases. The first phase was a series of small Action Enquiries, each of which improved a particular situation in an online community, improved an online research technique or improved the use a new technology, and combinations of the three.
Only a small part of the first phase had been pre-planned. I ended up working again with the ukcider CoP but then moved on to work with very different types of groups using different technologies in order to broaden my experience and make progress on the bigger picture, the making of the resource for distributed researchers. The choices were made by selecting from a dynamic list of possible future cycles maintained in a journal, and from new ideas as they arose, with the decision being made quite late as the separate cycles progressed, in order to take advantage of timely opportunities such as the invitation to the CPsquare workshop. Previous findings fed into subsequent decisions.
Through taking actions, observing and reflecting on results I managed to bring to the surface some insights and new knowledge which were then submitted onto the DARwiki, thus beginning the construction of a comprehensive online reference for the subject. Reflections on the process of conducting the individual small AE exercises informed and improved subsequent cycles, leveraging the improvement and honing an increasingly effective method of conducting cycles and generating content for the developing resource.
The second phase consisted of collaboration with others working in similar fields. This began with the pilot exhibition, and then turned in a direction determined by an analysis of the experience and feedback obtained. I sought out people and groups of people who are involved in some way with online research, and asked them to view a special Wordpress blog page which exhibited the facilities for the Wiki, solicited responses and began building a community of collaborating researchers around the resource.
While it is likely that this stage will only reach an established maturity long after the the lifetime of the requirements of the degree course, I am able to demonstrate that the project is well under way and beginning to take on a life of its own already. That is to say, this is a worthwhile real life project which was not ended after the 'main exhibition' phase was completed but continues indefinitely afterwards, just as the level two project continues to grow and develop from strength to strength now, and with it, my own learning and capabilities.
This AR adds to a small body of knowledge which was begun by Lave and Wenger, Feenberg, White, Rheingold, Hildreth and Kimble and others. It includes a capacity for ongoing maintenance and development which keeps pace with advancing technology and understanding. It had an immediate impact on my own practice as a self-employed Information Technology worker and on others working with or studying online research. There is also much relevance for remote workers in general, distributed COP facilitators and anybody interested in online communities and distributed research.
Very importantly, the work undertaken for all of my year 3 modules fell within the requirements of justifying my desired Degree Title Suffix of “Information Technology” by focusing strongly on the qualities of various internet tools and developing technologies which are in use and impact upon distributed communities.
Impact Study
Narrative
The challenge of an impact study concerning online environments is that much of the impact remains invisible to the researcher, as lamented in "Hidden from our Gaze" (Terrel 2006), it "brings us back to looking at hits and contributions rather than being able to see effects". Let's see if I can manage a little more than that.
The actions taken have been to declare the new form of Distributed Action Research, to set up a domain with blog and wiki, to conduct some distributed action research, to join and engage in conversation with practitioners and students of related fields worldwide, to hold three types of exhibition, and to set up two groups (one in email and one at Flickr) where community may eventually begin to crystallise.
The impact of all of this on myself has been substantial, but again is not easily gazed upon let alone measured. I feel I have gained in stature, in understanding and in competence but these are all intangible. By taking ownership of my own domain and publishing my ideas I have developed a higher level of responsibility and self management, and this appears to be reflected in the responses of others encountered in the wider world, which indicate a clear inclination to take me seriously.
I have refined and developed the practice of regularly adding little pieces into the DARwiki,at the same time as widening and improving my networking and filtering, such that it is slowly but surely gaining quantitatively, and moving towards the point where it will become qualitatively more significant to others.
So now let's take a look first at some of the data.
Quantitative data
The total feedback to exhibitions as collated at the the time of handing in module 3 is made available online here:
http://frankieroberto.com/dad/ultrastudents/andyroberts/year3/mod3-exhibit/feedback/
Stating that 36 individuals have made a deliberate response to the exhibitions in some way.
22 people registered and attended my presentation at Wiki Wednesday live event.
10 people responded to the Four Dichotomies by adding their "score" or comment to the parallel discussion page on the DARwiki. (barn raisers)
at least 4 people took part in Internet Relay Chat (IRC) at the darnet irc channel during the barn raising.
10 people responded to the online exhibition and podcasts by leaving comments there.
But the exhibitions are not the only means of impact
At the CPsquare conference I initiated a discussion area all about Action Research within the context of Communities of Practice.
7 people took part in the synchronous chat, plus 2 joined the ensuing discussion
over 129 contributions to the AR discussion were contributed apart from my own, by a total of 16 people.
9 members of the DARnet flickr group http://www.flickr.com/groups_members.gne?id=74989511@N00
12 members of the DAR mailing list http://groups.google.com/group/DARnet/about
11 registered for COPS group at AREOL "Thanks, Andy. They have all subscribed. Cheers -- Bob"
9 people have bookmarked the DAR wiki through the social bookmarking tool deli.cio.us
Qualitative data
Quotes
I looked through all of the qualitative data available, seeking out the best instances where somebody makes a comment which appears to me to hint at some kind of impact from what I have done.
- "Thanks for the chat notes. I really like the way these notes are recorded. A summary of ideas going off to wikis and conversations. Thanks for the notes and for the process idea." Beverley Trayner
- "BTW, I love the way you keep the whole chaos humming over in AR by tracking all the emerging ideas and open threads..." Ueli
- "Great song!! .. yeah that's all I wanted to say. Great song! :)" Dorine
- "I thought it would be helpful if you had host priviledges in the AR area." John Smith
- "I haven't had a chance to say that although I'm not doing much in your Wiki, I'm loving the idea and spinning some of my own thoughts from it. Thanks a big bunch - I can see it's an ENORMOUS amount of work and love on your part." Beverley Trayner
- "I find this very interesting and somewhat related to what I am doing for my dissertation on knowledge work in teams through virtual and physical interaction." Niclas
- "what a great idea an exhibition, didn’t know what it was actually, but inspires me for a product of an e-coaching trajectory with a group of development organisations!" Joitske
- "I applaud you for making so much of the process transparent to us. It feels like I’ve gone on the journey with you.....The Spinning Plates table is marvelous. I will likely borrow that idea for a project or two of my own.." Barb Mc Donald
- "I appreciate the openness of Andy and his project." Giorgio Bertini
- "I have added a link to your fascinating blog plus a link within your wiki (reflection)"
- "I was fascinated by your “journey.” I planned to stay for about 20 mins and nearly an hour later I’ve got here! DAR makes so much sense." Angela Souter
- "I would be interested to know if you think my AR project comes under the definition of DAR." Julie Leek
- "I found the glossary very helpful particularly the entry for distributed action research. I always feel I have faded in and out on my understanding of this concept but it made complete sense in one sentence." Sarah Hackett
- "I'll forward your stuff to the SIG eds. Yes - looking at your stuff, we have much to talk about" edmittance
- "I am a PhD Student I cannot help you with action research, but I am studying communities of practice. So in that area perhaps we can share/exchange some information. Nice to meet you" Maria Elisa, Argentina.
- "Our research contains some of the elements of DAR...your website has good “look and feel.” I admire such work." Dr. W. Reid Cornwell
- "I think it might be particularly helpful in developing your idea/concept of DAR (distributed action research) and its relation to the need for a group of people to get together to build somebody's new barn (and more) because you can't do it by yourself" Homero Gil de Zuniga
- "Really fascinating... I'm fascinated (admittedly, initially for work reasons) by the origins and growth of online communities and their structure, and I think this is an interesting tool." Jared
- "Andy Roberts' wiki barn raising event is focused on building some research based materials on Action Research - which is a fundamental building block of most Communities of Practice. Join in and add your weight to the discussion and see what you can learn and what you can contribute...you've got nothing to lose" John Barben
- "This strikes me as eminently worth discussing and reflecting upon: http://distributedresearch.net/exhibition/Spinningplates.html" John Smith
- "Cher membre de la communauté, Vous êtes invité à un événement "lève de grange" pour aider à lancer le Distributed Action Reseach du 10 au 15 mars 2006." http://www.communitywiki.org/odd/F%C3%AAteInternet/DarNet
- "interesantisimo sitio sobre investigacion distribuida y comunidades de practica"
DARwiki bookmarked
A new comment yesterday, reminded me of another source of impact data, in the form of evidence that 9 people have bookmarked the DAR wiki through the social bookmarking tool deli.cio.us :
http://del.icio.us/url/9c0d51903b0de58f0ebe454ce23d076d
Main Page - Dar http://distributedresearch.net/wiki/index.php/Main_Page this url has been saved by 10 people.
1. Main_Page edit / delete by aroberts to actionresearch wiki dar ... on april 5
Apr ‘06 ¡Cósmico! Action Research with distributed Communities of Practice - ananias
- interesantisimo sitio sobre investigacion distribuida y comunidades de practica - poliorcetes
Mar ‘06 Welcome to the DARwiki, a resource and a meeting place for distributed action researchers. - sridgway
- distributedresearch.net - tnt_thomas
Feb ‘06 Andy Roberts wiki on Distributed Action Research - kanter
posting history
first posted by kanter to action-learning wiki Apr ‘06
- by ananias to CoP DEA communities_of_practice wiki
- by poliorcetes to actionresearch community collaboration collectiveintelligence
- by pab to communication community research wiki distributed actionresearch
- by aroberts to actionresearch wiki dar
- by digitaldust to wiki research community actionresearch academic
Mar ‘06
- by cued100prof to wiki ActionResearch MyResearch
- by jsclarkdotnet to wiki activism research community communication
- by sridgway to wiki networkedlearnin
- by tnt_thomas to COP
Feb ‘06 by kanter to action-learning wiki
Analysis
My coding
impact study
evidence of impact derived from feedback data
categorised
- Engaged with Ideas
- DAR understood 6
- own thoughts inspired 3
- related to own work 5
- DAR understood 6
- Appreciation
- facilitation appreciated 3
- own thoughts inspired 3
- participation appreciated 2
- process idea 1
- transparency appreciated 2
- facilitation appreciated 3
Peer coding
- Appreciation
- general 17
- of the process 10
- general 17
- Engagement with ideas
- moves to collaborate 11
- identification with DAR 9
- moves to collaborate 11
process analysis
I anticipated that, with two different people analysing the same data, different categories might emerge but was surprised at the wide disparity in our tallies for the same categories. Linda Hartley (peer review partner) suggested that this may be because of slightly different techniques employed, herself using a more time consuming process of exhausting each code serially, resulting in multiple passes of the data. I had developed a slightly different technique originating from a chat analysis, performing multiple code application with a single pass. The literature eg Dick(2006) suggests Linda's technique and a detailed comparison of individual coding decisions would confirm if this were the cause, but such collaboration was not really practical at the time so I decided to adopt the longer method for my next coding analysis, of the challenges.
Additional impact
- An invitation to present DAR to an event feeding into a large conference in Florence.
- An invitation to contribute a chapter for a book a friend is editing about CoPs.
These opportunities, arising from the research in question provide avenues for widening impact in the future.
I could also dig deeper into the research data itself to find more indirect evidence of impact, such as appreciation for the contributions of others brought about through facilitating the action research discussion, even where it doesn't directly concern DAR.
Negative impact
I'm intending to use the negatives for the defence and validation, and only looked for positive impact in the first section. I don't feel that instances of lack of impact, lack of participation or failure to communicate are really types of impact in this instance. The negative impact on myself is just that I could have been doing something else with my time rather than contriving to make my work fit into the degree requirements.
Downstream impact
The concept of attributable downstream impact is contentious. On the one hand I can look back at work which I initiated in the past and see that unanticipated repercussions are continuing into the present and beyond. For example, it was pointed out to me recently that the report in which the name 'distributed action research' first appeared was appearing as the first result out of 12.5 million on Google for the search "action enquiry" [5]
Work done at that time on the ukcider CoP and wiki continues to reap rewards as the resource improves in both scope and quality, with key individuals benefiting from the community, as feedback testifies
- "the most important things that I have learnt about the craft had come from ukcider , over the last few years." Rose Grant.
From this it may be reasonable to predict that impact from the present project may well be similarly forthcoming over the months and years ahead, although as time goes on it becomes more and more difficult to untangle the threads of cause and effect through action by self and others in order to claim any reasonable attribution, as Smutylo(2004) explains
- "impact is the result of what many people do"
Information for Defence
method
Most of the available feedback and responses to the exhibitions and research is collated in one document which starts here http://tinyurl.com/fun7y
I found there, 28 challenges, then added 8 more from learning set peers Eve Thirkle and Linda Hartley, making a total of 36 challenges.
To these I applied the Emergent Coding Analysis technique which has been refined during the course of my practice.
findings
codes listed alphabetically with instances counted
data analysis 1 definitional confusion 7 evidence 2 findings 3 fuzziness 7 interactivity 7 literature 2 measurement 2 methodology 4 ownership 3 planning 1 reflection 2 scope 2 technology 1 unclear cycles 5 understatement 2 website experience 13
refinement: codes clustered into related groups
A) website experience 13 technology 1 interactivity 7 ownership 3
B) methodology 4 unclear cycles 5 planning 1 literature 2 reflection 2
C) evidence 2 findings 3 measurement 2 understatement 2 data analysis 1
D) definitional confusion 7 fuzziness 7 scope 2
Responses
Cluster A - website experience, technology, interactivity, ownership
website experience:
These are mainly usability issues some of which have already been addressed. Some are part of the continuing cycles of refinement, and some simply come down to differing personal preferences which may not be resolvable. As such this is not really a suitable topic upon which to mount an academic defence.
technology:
The technical requirements for accessing the websites should have been well within the competencies of my intended audience, but some adjustments were made in response to evident difficulties. In particular I concluded that the blog/wiki integration, although aesthetically pleasing, may be inappropriate given that the blog is a personal platform for the individual researcher whilst the wiki is intended to develop as a community resource.
interactivity:
I refer the reader back to Why_this_is_being_written_on_a_wiki
ownership
I find the issue of ownership and control with respect to wikis quite contradictory because you can have an open, democratic, anarchic group-mind wiki only to discover that ultimately it is owned and controlled by the one person who administers the domain name, or perhaps a 'foundation' heavily influenced by one person. Is a genuinely decentralised implementation conceivable, a peer-to-peer hosted wiki which really would be owned by the contributors? A system with multiple redundancy, decentralised, no single point of ownership propagated on a flood-fill basis perhaps.
Mark Dilley responded to my idea:Excellent observation, PeerToPeerWiki is part of that discussion. As far as WikiCommunity is concerned. There are many of us here who have been active for a long time in some general communities. Perhaps when entering into different WikiCommunities? we could talk about our other wiki stuff? Best, MarkDilley
As far as the DARnet community here, on the mailing list and on Flickr is concerned, well these steps forward have been accomplished already without retreat, and Rome wasn't built in a day. In some respects, getting acceptance for new ideas is like pushing at an open door, but in other ways it is a long road with a thousand small tasks in order to launch anything which transcends the personal.
Cluster B - methodology, unclear cycles, planning, literature, reflection
Some of these do not critique the research as executed, so much as to fundamentally misunderstand the research proposal. I did not set out to perform an exemplar two cycles of typical action reseach about a very small project solely for the purposes of assessment. I needed to rapidly acquire enough experience and learning in order to be able to justify my proposition of distributed action research as a new category within the AR family of methodologies. The structure of the planned project was this deliberately distorted from the 'norm' by both self-reference and the need for early dissemination. This is a kind of bootstrapping technique also described as 'building the aeroplane whilst flying it' ! If you think about it (planning) too much then it's impossible, so you have to just do it.
( view movie] )
The Spinning Plates diagram as exhibited and included in subsequent reports needs to be understood not as a finished document, but as a snapshot of the documented research at a particular time in February. It served to unpick most of the 'nested cycles' as described by Dick(2004)
- "action research as consisting of cycles within cycles within cycles"
and helped to show some of the relationships between findings and further actions. Perhaps surprisingly, a full documentation of all of the research carried out is not provided for within the modular assessment products, of which this is a part.
Reflection
Asynchronous conversation is by its nature, highly inclined towards a reflective mode of writing, and I have practised the art of consciously harvesting my own reflective conversations since discovering the work of Pask(1975) two years ago.
unrelated discrete cycles
The accusation that unrelated discrete cycles were undertaken instead of what was planned is false on two accounts. Firstly, it was exactly a series of unrelated small cycles which was indeed planned a means of generating a variety of content for starting off the DARwiki. It may not be the most common way to structure an action research project but for a meta research project it is entirely appropriate, since we are concerned more in this special case, with the process rather than the subject of the small cycles themselves. Since the process has been made into the topic, then improvements in research process from one cycle to the next, which should be a feature of any AR, can be deemed to form the relatedness of the otherwise diverse activities. And secondly, because the research proposal was optimised for serendipity it so happened that most of the small research cycles were closely related to the situation of facilitating a fringe conference at the Web2.0 event in any case.
Lastly, it needs to be understood that the most important and higher level of identified cycles within the nested schema, that of presenting ideas to peers, refining definitions, learning through connectivism and reflection, improving participation and initiating generative action has been through two or three cycles already with this validation and defence forming a further piece of the same jigsaw.
literature
The decision to present four dichotomies as a finding was not directly informed by any specific piece of literature but probably emerged as an original sounding idea which in reality owes something to a wider reading over an extended period. It's similar to the process by which a composer writes an original work but the listener musicologist can hear clear influence from other artists, in some cases almost to the point of imitation.
I will quote two examples which have come to light since, but surely share a common memetic ancestry.
From Reason
- "Dialectics involves a recognition of the inseparability of two apparent opposites, and an exploration of the interplay between these interdependent poles"
And Hildreth and Kimble (2002)
- "Knowledge is not made up of opposites; regarding knowledge in these terms is a false dichotomy. Rather than seeing knowledge as opposites, perhaps we should think of it as consisting of two complementary facets: a duality consisting simultaneously and inextricably of both [hard and soft] knowledge."
So the subtle purpose of the tool was primarily to stimulate discussion, but also to do so in a way which elevated the thinking above the kind which merely pits one person's experience against another's, as had been happening in the email group and workshop discussions. The comments on the Talk:Four_dichotomies show that this was accomplished to a large degree.
Cluster C evidence, findings, measurement, understatement, data analysis
measurement
I plead guilty of failing to measure the subtle, qualitative improvements in practice, theory and understanding which have been brought about through beginning to conduct some participatory distributed action research. I don't even have measurable evidence of 1st person research, and I reject the notion that nothing is worthwhile without evidence. As a self-employed student-researcher I am my own sole stakeholder and I do not feel the need to provide a pseudo scientific veneer of contrived rigour in order to compensate for the low esteem which action research seems to suffer from, according to Reason(2005)
- the family of practices called action research has inhabited the margins of academia for many years. As Argyris (2003) points out, the pursuit of knowledge in the service of justice and effectiveness has often been held in disrepute by management scholars. ( my emphasis)
findings
The exhibition of my 'Four Dichotomies and the presentation of a Taxonomy of Cops as findings have been questioned, so I shall attempt to explain my interpretation of what occurs during the process of 'finding'. I do not believe that finding in this type of research is an inhuman, mechanistic, number crunching exercise but that it involves a more subtle process of absorbing large quantities of different kinds of qualitative data, engaging in both private reflection and public conversation, allowing ideas to settle and brew for a period, after which some apparently new ideas suddenly emerge without necessarily indicating any clear trail backwards as to the inspirations and exact origins of those thoughts. Perhaps this generation of original thought and the recombination of 'found ideas' owes more as a human process to art than it does to science, or perhaps traditional science has merely failed to acknowledge it.
data analysis
The specific suggestion that it would have been better to analyse the chat for social dynamics rather than content deserves a direct response. In that particular situation I wasn't trying to learn about the social dynamics of how people interact in a chat, but attempting to unravel the different strands of conversation in order to clarify the next actions which could be taken. I also think that if I had been intending to use the transcript of a chat to formally analyse the way in which different people interact then ethically, specific informed consent would need to be obtained beforehand from all participants, with the purpose of the analysis declared upfront. I'll leave the the conundrum of how to obtain consent without then distorting the data through self consciousness of the actors, to be solved by some other kinds of social scientists.
Cluster D - definitional confusion, fuzziness, scope
My first reaction is wonder whether the definitional confusion is related to 'website experience' since a clear and concise definition of DAR should be quite easy to find, right there on the front page of the DARwiki where it says "DAR = Action Research with distributed Communities of Practice" closely followed by the Venn diagram. A Venn diagram shows a hierarchical relationship between concepts such that it should be easy to deduce that not all Communities of Practice are distributed and that not all Action Reseach takes place online but that DAR is the area of intersection where Action Research is taking place, on the internet, with communities of practice.
Perhaps it isn't really a definition which people are asking for, but a deeper understanding of the nature of the concept, or perhaps they need better definitions of the constituent parts. Through engaging in focussed discussion on dedicated email groups such as com-prac and areol, I have discovered that not only is there little agreement as to what constitutes Action Research, but that the terminology of CoPs suffers from even more definitional dissensus and nobody can even agree what is meant by "community" by "practice" or even by "knowledge". This kind of epistemological diversity would appear to make the possibility of an accepted and understood definition of almost anything recede into an academic distance, about which many practitioners are past caring.
So we are left with pragmatism, and DAR has to become simply something which works. A methodical, reflective and cyclical approach to working with online CoPs, whatever you think they are, to be valued according to the sense of 'improvement' held by the practitioner, the group or the sponsor.
Developing my Taxonomy of CoPs
This is part of a continuing conversation about the widely differing concepts which participants hold about what constitutes a community of practice. From "com-prac" the public yahoogroup owned by John Smith
I wrote:
Pete and all,
Thanks for your explanation of "definitional dissensus" which we have been seeing quite a bit of around here lately.
I have been reading "Reembedding Situatedness - The importance of power relations in learning theory" by Contu and Wilmott, and also Andrew Cox's " What are communities of practice ? A comparative review of four seminal works".
My own work is leading me to conclude that current stark differences in perspective which keep surfacing in the discussions around the nature and application of concepts relating to CoPs are an indirect reflection of existing class relations in society. From what I can gather, Contu and Willmott seem to have begun heading in a similar direction, from within the educationalists point of view.
My reading of Cox also suggests that the definitional dissensus stems from the original literature, and points to the contradictory definitions In Wenger's earlier and later works. The definition in Wenger, McDermott and Snyder is described as "a much vaguer definition than that used in the 1998 book... The purpose is specifically to learn and share knowledge, not to get the job done.... This is genuinely a different concept from that proposed in (3), not just a change of tone or position; it is simply a different idea"
So we would appear to have at least two quite different ideas going by the name of CoP. Does one need to "win out" over the other or can the terminology be stretched to allow both to happily coexist? If the latter, then surely at the very least we need to be more aware of the different ideas so that when somebody new starts to talk about CoP as if we all know what they mean, we can ask "which kind do you mean?" rather than try to create a false consensus based on extreme fuzziness and unconscious ambiguity.
I think my contribution will be to offer a simple tool for analysis of the diverging structures which are being called CoPs, a tool based on power relations, economics, and class. By this method I will suggest that the theory which Lave and Wenger derived from observing proletarian organisations, when applied from the later managerialist perspective will produce quite different structures and outcomes, replicating the material and philosophical interests of the bourgeois corporations and institutions.
As Fred wrote here a few weeks ago:
"Well, FWIW, I think most of us in the Western World are serving the same master: "the ruling class" -- which consists mainly of a network of the very, very wealthy (a.k.a. "the swells who run the show") and their paid hirelings and lackeys (mostly second-tier execs and second-rate politicians). The rest of us subsist and succeed on whatever trickles down. (If you think that's "a stretch," take a look at income distribution statistics.)"
By raising the issue of class, Fred inspired me to apply a class analysis approach to the taxonomy of COPS, which I feel helps to peel away the fuzzy thinking, compromise and dilution of theory. Lave and Wenger(1991) suggests that CoPs have existed since at least feudal times, when the main classes would have been the aristocratic Lords as landowners, and the serfs, with a fledgling mercantile and craft class just beginning to emerge. But what has happened is that the principles observed have been grafted straight onto 'modern' society by looking at practices within enterprises, in some ways echoing the Peter Senge myth of "whole organisational learning". But this only looks at one side of the equation for a capitalist society, the side of the bosses. A more accurate class analysis would look at the two main classes in society, and classify CoPS accordingly.
Hence my simple but fundamentally essential starting point for any taxonomy, having only two top level categories
Proletarian CoPS
( self organised groups of fellow workers )
Bourgeois CoPS
Management controlled, profit orientated, having "experts" respected not for their expertise, but demanded through their position.
Which are perhaps more likely to become known in common parlance as
"Workers CoPS" and "Corporate CoPs".
This is not a simplistic distinction based upon some kind of average class composition of the individual members, but on the perceived and real sense of ownership, control and purpose which have been present or gained the upper-hand in the CoP.
As you can no doubt tell, this post is not a fully thought out or properly written up account of my proposition, just a very early version which I am prematurely sharing here because I felt the current direction of the conversation warranted it. So I'm sorry for taking that liberty, and nonetheless welcome comment, looking forward to further debate leading to eventual clarification.
further discussion
On the com-prac yahoogroup at http://groups.yahoo.com/group/com-prac one academic authority on Wenger replied "an interesting approach - I look forward to reading more about your marxist approach to looking at communities of practice" and Pete Bond, author of "Communities of Practice and Complexity : Conversation and Culture" 2004 responded:
- Andy, I can appreciate where you are coming from. As you are thinking about it, you might be interested in reading the following title, if you have not done so already. The Regulation of Capitalism: A US experience, written by Michel Aglietta (1978??) ( French Marxist writing from Harvard University).
- This is about the self-regulatory dynamics of capitalism as a system of 'economics', Its depressing, as its about how impossible it is to take any initiatives that are against the logical imperatives of capitalism. The 'boss' the 'ruling class' is Financial Capital, The myriad of very powerful financial investment institutions at the 'head' of the capital investment cycle.
- In particular you might be interested in the 'canalization' of working class initiatives, such as the trades unions (proletarian CoPs). Essentially, this means that the working class initiatives were 'guided' toward the boss's objectives by the logic of the capitalist system of production and consumption.
- The trades unions are an obvious example of CoP but I'm not sure whether they even get a mention in the literature (I wonder why not, could it be that they are examples of subversive CoPs?) . The point of mentioning the book is that your workers CoPs will be subject to the imperatives of profit making and capital accumulation.
I've not read Professor Aglietta's book but the theory sounds remarkably similar to the pessimistic revisionism of Antonio Gramsci.
I would agree that any organistions such as trades unions or workers CoPs and the individuals within them are in no way immune from the influences of the economic system that predominates in any particular society. As such the aims, objectives and repertoire of workers CoPs will be found to be pragmatically grounded in seeking ways to further their own interests within the current system, and not to try and create some kind of utopian microcosm of a possible future society. This will not always be a progressive thing, since the interests of any one particular group of workers may not always coincide with the interests of the proletariat as a whole. I suspect that the extent to which different CoPs cooperate in solidarity with each other, perhaps through Wenger's model of 'constellations of CoPs' will depend more on the events playing out in the wider society than on any processes occurring inside the CoPs themselves. This looks a promising area for further research and action.
A response from one of the attendees at AREOL23 says:
- In your Trigger 11 posting, you say:
- > What's demanded in the final analysis by the change process is
- > democratic control and ownership, not just participation...
- I can agree that participation should be 'democratic control and ownership', but I'm interested that, in : the CoP quote, the change process is only about furthering "their own interests within the current system" and not about making fundamental change.
- Or is this peculiar to CoPs of trades unions and workers? And if so, was this a reason for the events of Paris in May 1968 failing to achieve the Situationists' destruction of the manufactured spectacle and commodity economy, for example?
Learning Set Peer Questions
On Connectivism
- Andy, you use this term connectivism and I feel a bit unsure on what exactly it means. I know it's a buzz word and I keep thinking I've understood it then it slips away again. So what is it and why is it important to your research? Linda Hartley
Just for a start, this is what I wrote in report 3:
Connectivism
I decided that the most appropriate way to appraise my learning from the four months duration of this module is by applying the new theory of learning called "connectivism". Connectivism maintains that the type of learning which is most appropriate for the internet age is not so much the accumulation of facts or the development of specialised capabilities which may quickly be rendered obsolete, but the development of more and better quality connections in the personal and professional network.
and the reference:
Siemens(2005) Connectivism: A Learning Theory for the Digital Age, elearnspace URL: http://www.elearnspace.org/Articles/connectivism.htm accessed 3/4/2006
- But unless you've got specialised capabilities who'd want you in their network? Or is the idea that everybody has their own area? or maybe that you look the info up as you need it? Is it in danger of getting too shallow a form of knowledge? Sarah Hackett
I think there are quite a few areas where connectivism needs to be criticised, and Sarah has highlighted a couple of them here.
By emphasising the value of connections, this is neither egalitarian nor generally meritocratic. But then perhaps neither are the other learning theories and models?
Is the memorisation of lots of facts any deeper than knowing where to look them up?
- But it's not just facts is it? It's ideas and surely to really have a deep understanding of an idea one must know enough about the thing itself to think about it and engage with it, not just 'know a man who can'. LH
I agree that the understanding of ideas is essential and I think a connectivist understanding would see strong ideas as attracting a substantial network of thinkers around those ideas. The connectivist learner then, seeks to plug into that network in a bidirectional way, and maintain an ongoing dialogue with the living development of ideas by and within the network.
So it's not just a question of knowing where to look things up, but of using, adapting and adding to your connections to zoom in closer to the centre of whichever subnet of ideas or topics you want to be focussing upon. AR
- "strong ideas as attracting a substantial network of thinkers around those ideas" Strong ideas or strong personalities? In this connected world isn't it more about your abilities as an influential blogger than the strength of your ideas? LH
Maybe by defining strong ideas as those which are better at getting people to promulgate them, I have moved more towards a discussion of memetics than connectivism. AR
From Eve
Hi Andy, No red ink in sight! Please don't take any of these questions personally! I struggled to find much to criticise in yours – it's such a new approach ... hence the last question! Eve
- "The methodology employed is that of emergent action research, adapted to the online environment and geared towards pragmatic goals of improving practice, building a resource and developing theory."
1. How exactly have you adapted the methodology of emergent action research to the online environment? What are the differences in your methodology?
2. Whose practice have you improved? Yours? Others? By what criteria do you measure this?
3. How would you answer the challenge that this is not the planned emergent research that you proposed ...
- "The second phase will be necessarily relying on a very emergent approach, in that the direction of the research will be entirely dictated by events and results from initial cycles of research which have not yet begun."
but merely a series of online undertakings loosely linked by a common theme. (Ouch! ö)
From Linda
Andy
You know how highly I rate your work Andy so please take this as nit-picking. My criticality is not much in evidence at the moment so take this all simply as questions to provide you with opportunities to show the bits that got cut out in trying to squeeze it into the module:
- You make no mention of the theoretical model of action research you've chosen except:
- "The methodology employed is that of emergent action research, adapted to the online environment and geared towards pragmatic goals of improving practice, building a resource and developing theory"
How do you counter the suggestion that what you are doing is not research at all but pragmatic, opportunistic actions without any underpinning theoretical structure?
- You say in your introduction: the mechanism has been found to be sound such that further progress is most likely assured. What is the evidence for this assertion?
- one of your findings is a taxonomy of CoPs - a division into workers and bourgeois owned. In what sense is this a finding rather than a conclusion?
- How would you counter the suggestion that your actions do not seem to be clearly identified in specific cycles.
- What evidence is there that learning from cycles fed forward to fuel future action?
Conclusions about Andy's Action Research
Andy and I just had an interesting conversation. I suggested we might be better at writing other people's impact studies than our own. I thought that if we handed over data etc. another person might be able to see things more objectively than we could ourselves. He started to draw out what he thought my conclusions should be and so I returned the favour.
I think Andy set out with his question about Distributed Action Research and sought to answer it by constructing for himself a person learning environment. He took the stuff he's learned about using 'small pieces loosely joined', about situated learning and learning from practitioners and he's used all this to engage in reflective conversations (as described by Pask - see Andy's yr1 work). These conversations form Andy's data. Analysing and refining during the discussions has left him with the sort of analysis Dick recommends where meanings have been explored and agreed during the course of the interaction. Much of what he's found doesn't need further analysis as it's already been worked through at the time. It's concentrated and interesting stuff. He's not shied away from divergent voices - he has as Dick suggests actively sought them out. When he's found them then he's engaged with them and worked to see and encompass their viewpoints in the data.
By engaging these practitioners he's begun to answer his initial question in a positive way. The concept of Distributed Action Research seems to be a useful one for looking at working with distributed CoPs. Other people are happy to latch on to it, to help define and refine the concept, to contribute their experience and expertise.
It's moved him nearer to the goals he started yr3 with. He's a node in a network of practitioners with an interest in working with distributed communities of practice and he's managed to draw together people with an interest in both CoPs and action research on his wiki and also within CP2. He's learnt a great deal about the technical aspects of wiki management and worked with others in CP2 on the implications for distributed CoPs of web 2.0 technology. The cider wiki and CoP go from strength to strength benefiting from these new skills and expertise.
To Eve
Dear Eve,
I enjoyed reading through your research again, this time with the requirement to produce some critical questions for you to defend against. So I hope you'll find something amongst my comments which might help in some small constructive way with your ongoing research, or at least something worth responding to, for the sake of the exercise. Andy R 11/4/2006 [6]
- "How do I, as a parent/carer of an autistic child, improve my parenting skills and adapt the home environment to my child's needs?"
1)
- "table manners (or rather the lack of them) have been forefront in our parenting difficulties, mainly due to the problems this causes when visiting friends and relatives........This action research concentrates on table manners. "
Is it possible that the research outcome is geared towards placating the concerns of friends and relatives who may hold narrow and impractical values as to what constitutes good table manners, rather than more productively helping the family to overcome obstacles which are of more direct concern to the child and family themselves?
2) Cycle 3 seems to consist of an action - the drawing up some behaviour maps, an immediate reflection-in-action that the maps were too complicated for the child to take in, followed by the abandonment of the cycle before any data was collected. This could either be criticised as the researcher's own prejudgement interfering to override the true action research process, or as a false cycle.
3) During the account and conclusion there are many references to published literature and to family discussions but I didn't see anything about self help groups, online autism discussion lists, parent support groups or similar. Is this because such potentially valuable resources of knowledge and support were left untapped or felt not worth mentioning?
To Linda
Dear Linda,
I appreciated reading research section 1 and tried to do so from an objective point of view, despite having seen earlier versions. I have included some technical criticisms as well as ones about the research because for me, the structure of the presentation does make it a little harder to follow the account of your research, which may detract slightly from otherwise rigorously executed and very worthwhile work. Andy R 11/4/2006 http://distributedresearch.net/blog/
1)
- “How can I as a resource provider use the 'read-write' web to improve communication between users?”
The intended scope of the research is difficult to ascertain without any description of either the 'read-write web' or the 'users' in this case. Why did you not attempt to define your terms?
2) This is just a technical comment:
- "This diagram provides an overview of my research:"
I cannot read the small writing on the overview diagram and there is no link to a larger version, so the overview is effectively missing.
3) In Cycle 2, you have taken some actions to improve the resource and also discovered a whole new potential audience (the TA forum). But then to collect data, you ask the same questions as in Cycle 1. Was there no scope for improving the research process by asking better questions?
4) Another presentation problem: Cycle 3 then seems to begin around the time of the illustration of Elmer, but without any heading to mark the event, so I'm losing the sense of structure, as I'm reading the report.
5) The postcards cycle is a significant outcome and seems to contradict the initial assumption of lack of ability to have an impact in the workplace. Did the idea not then occur to radically change the research question in the light of this finding?
My conclusions about Linda's research
Linda's research has yielded tangible and lasting results of a very real nature. This is no abstracted or hypothetical achievement, she does not say "this shows what might be possible if only..." but "This is what I have made, and made happen".
Setting out to explore the potential of using simple but powerful new web based services and tools for communications, the Classroom Displays blog exceeded all expectations by becoming one of the best examples which leading edge commentators provide when they wish to demonstrate an inspirational case of new educational technology already being applied in practice.
TA Forum
When Linda sought out and discovered the TA Forum, she had the theoretical understanding to appreciate that despite initial appearances, this was no idle chat-room but the beginnings of a self-organised community of practice. What happens when substantial numbers of people who could all answer the question "I am a....(Teaching Assistant)" gather together on a regular basis? Why, they will chat about their work, exchange stories, form bonds of solidarity and begin to exchange tips ideas and resources to help each other solve some of the daily challenges. (Incidentally, this is a great example of the first type of CoP - the workers' CoP as opposed to the corporate CoP defined in my proposed proletarian/bourgeois taxonomy.)
Postcards
And so it was possible to take the opportunities presented and skillfully relate them to immediate needs in the classroom with the postcards cycle, which used the internet to give children who often grow up in a very small world, a very real glimpse of other places and other lives. This is the kind of thing which may have been in the minds of policy makers when the funding was provided to increase the ratio of computers per child, and to install broadband "always on" connectivity. The National Curriculum even suggests building links with other schools and in no way prohibits the kind of inspirational lesson designs which Linda manages to construe, and yet very very few primary class teachers ever manage to build these exact kind of links which the TA forum people have managed. Instead, they seem to have been trained to view the web as a place from which you can download ready-made and "safe" lesson plans for printing out.
impact
Within the data, there is evidence that a functioning many-to-many link has been set up: "Great to hear other people's feedback" thus providing a powerful resource which has a widely disseminating impact, not just an immediate or local one. And the impact very quickly multiplies by reaching the recipients' students as well: "After I saw your poster, I gave my online students a link to a multicultural diversity test"
The fact that the classroom displays site is well optimised for search engines means that this collective resource does not languish unnoticed in some cupboard or online backwater, but is eminently findable by people worldwide who are actively seeking just this kind of thing, and the statistics show that they do so.
blocked
One negative finding which Linda importantly highlights, is the tragic side effect from badly executed internet access policies, namely the false positives or blanket blocking which results from over zealous filtering systems. "What a great use of a blog and flickr, I just wish we could see flickr in school"
Perhaps another impact from this pioneering project will be to provide one more compelling reason for why these blocking policies need to be revised. The publishing of this research should help to bring that day nearer, so that children and adults can benefit from the positive aspects of internet communications, armed with the knowledge and confidence to manage their own safety without being deluged by exaggerated fears and bureaucratic interference.
Conclusions
With the nature and size of the project, the researcher succeeded in soliciting sufficient genuine critical feedback from peers to mount a defence without having to fall back on the safety net of asking a learning Facilitator to play the role.
Some extended discussions on pertinent topics were achieved but this as often is the case, is a source of disappointment as well.
learning during the course of the module
- that taking time out for 'writing up' can detract from the ongoing research objectives.
- Writing each others conclusions is a much better form of peer review than contriving criticisms.
- That I have in fact created a Personal Leaning Environment for myself.
- That I have actively sought 'divergent voices' and this has not caused my work to fall apart in any way.
- That one or two of the questions I have asked of others have been responded to in a way which to me, doesn't really address the nub of the issue, and I then held back from further probing. Therefore, by deduction it's quite possible that in turn, my defence as provided here, although convincing to myself, may not be fully adequate as far as the questioners are concerned. But I won't know unless unless they are forthcoming. It would be so much easier to debate with one's enemies.
wiki
The novel process of writing the report on a live public mediawiki has served its purpose but without really being tested in terms of multiple voice editing. Two people have each corrected one spelling typo and the rest is all my own doing, with the talk page being used for one suggestion (audio interview) which was attempted but abandoned due to audio quality problems with the technology (Skype + garageband)
No precedents having found, it was difficult to estimate the innovative worth of the exercise, it seemed fairly normal to me. Writing copy direct into a mediawiki is a different experience to writing into Voodoopad, the private wiki I have used for some previous reports. The automatic generation of the Table of Contents is significant guiding feature, giving a growing overview as the work progresses, which I think encourages a more global and less linear writing style, although combined with the way the topic is structured it has perhaps mitigated towards an overly systematic construction method.
further research
The next stage of my research will veer back from the immersion in Communities of Practice and CoP theory, returning to some of the unanswered questions about the differences between DAR and other AR, refactoring the wiki for further simplicity, and studying further cases with reference to the taxonomy. The wiki will be enabled for RSS stream rendering and this will allow for a study of community through social bookmarks and tag clouds, blog aggregation, reputation systems and predictions markets. On the other hand, I could plant an orchard with perry pear trees.
on the dissemination process
An Anonymous Senior Researcher on Friday, May 5, 2006 at 9:28 am +0000 wrote:
- Research is generally written to be published
Where is all of yours published, Anonymous Senior Researcher?
(no response forthcoming yet)
Linda Hartley on Saturday, May 6, 2006 at 10:57 am +0000 wrote:
- On the contrary - undergraduate research usually remains unpublished.
A lot of other research is published only in academic journals with a very limited circulation and an extortionate cover price, and another branch of research is destined to remain a commercial secret owned by the sponsors.
So all in all it looks to me as if the systems for assembling and disseminating human knowledge for the problems facing humanity in a changing world are going to have to be built outside of the academies rather than there being any chance of innovation emerging from the inside.
References
Bond, Peter (2004) "Communities of Practice and Complexity : Conversation and Culture" http://www.leader-values.com/Content/detail.asp?ContentDetailID=984 Accessed 22/04/2006
Cox, Andrew (2005) "What are communities of practice ? A comparative review of four seminal works". Journal of Info Science, 31 (6) pp527-540
Dick, Bob (2002) Action research as meta-research. A paper prepared for the International Sociological Association Conference, Brisbane, 7-13 July 2002. available http://www.uq.net.au/~zzbdick/dlitt/DLitt_P47meta.pdf last accessed 4/5/2006
Dick, Bob (2006) "What can grounded theorists and action researchers learn from one another?" as yet unpublished
Hildreth, P.J. & Kimble, C. (2002). "The duality of knowledge" Information Research, 8(1), paper no. 142 Available at http://InformationR.net/ir/8-1/paper142.html accessed 8/5/2006
Johnson, C.M. (2001) Survey of current research on online communities of practice available http://dissertation.martinaspeli.net/papers/communities-of-practice/johnson-2001-a-survey-of-current-research-on-online-communities-of-practice/
Kimble C & Hildreth P (2004) "Communities of Practice: Going One Step Too Far?" http://www.aim2004.int-evry.fr/pdf/Aim04_Kimble_Hildreth.pdf Accessed 11/04/2006
Pask, G. (1975). Conversation, Cognition, and Learning as described in The Theory Into Practice (TIP) database URL http://tip.psychology.org/pask.html Accessed 11/04/2006
Reason, Peter (1993) "REFLECTIONS ON SACRED EXPERIENCE AND SACRED SCIENCE" Journal of Management Inquiry, 2(3), 273-283. available online at http://www.scu.edu.au/schools/gcm/ar/w/Reason3.pdf accessed 8/5/2006
Reason, Peter (2004) Choice and Quality in Action Research Practice. (forthcoming) Journal of Management Inquiry. (developed from Keynote Address, World Congress of Participatory Action Research, Pretoria, South Africa, September 2003) available from 'Papers and Chapters by Peter Reason' http://www.bath.ac.uk/~mnspwr/paperslist.htm viewed 4/5/2006
Smutylo, Terry (2004) "The Output Outcome Downstream Impact Blues." (multimedia) Available at http://www.idrc.ca/uploads/user-S/10960530301karaoke.swf accessed 3/5/2006
Willmott, Hugh and Contu, Alessia (2003) "Reembedding Situatedness - The importance of power relations in learning theory" . JOURNAL:. Organization Science (OSc), 14(3), 283 - 295.
See also:-
AR_literature and COPs_Literature pages on this wiki.
Appendix
hotseat
Impact
Hi Ian, In my case, the online environment has been the focus of my research, although I have needed to venture outside of Ultraversity to meet many of my needs.
The wider internet has provided me with context, environment, both scholarly and business conversation, data, access to experts, colleagues, professors, peers, authors, technical advice and, dare I say it, 'content'.
As an internet researcher, working with geographically distributed communities I didn't think I was going to be able to hold a physical exhibition, but then at the last minute, an opportunity arose to speak at 'London Wiki Wednesday' to an audience of interested community builders, social software developers and new media people. A few were interested enough to hang around and talk to me afterwards, and there are brief reports of the evening in a couple of places but apart from that I cannot see any way to measure impact from it. If there is any, it could take a long time to show itself.
The main impact of my research will be felt by individuals in the communities affected by my interventions and learning. But measuring this seems to me quite a challenge as it is largely invisible to me. The convening of a group of CoPs people and others to attend an online AR course together is perhaps a usable example.
The main exhibition was an online website attached to a blog, which received comments enough to count as feedback, and some indication that ideas seen there might be taken up and tried in visitors own projects. So that's evidence of impact I suppose, and then there was a participatory side as well, with visitors being asked not just to view and to comment, but to get involved, contribute, communicate with each other and build the beginnings of a new community.
The proposal to turn the 'barn raising' event into the launch of an 'Action Research Coffee Shop' for critical friends and peer validation will potentially have a wide reaching impact, though again, probably not in time for the pressing task of writing up.
So I'm very open to suggestions about the measurement of impact in an online environment. Thanks
http://distributedresearch.net
Hidden (from our gaze) Effects
Posted by Ian Terrell at March 18, 2006 10:20 AM
Thanks Andy. You remind us that the Ultraversity activities have to be set in context of activities elsewhere in life. Clearly for some like yourself with such a widespread web presence, and activities beyond, the UV can barely be more than a stimulus at best, but is likely merely to be a means of gaining academic award, for activities that you would have been doing anyway.
I also thank you for returning to the "hidden" impacts. Those that we know little about because they are unseen influences, and nudges. (Regrettably in brings us back to looking at hits and contributions rather than being able to see effects, but thats the life of researcher/evaluators for you. Indeed these unknown effects are the real meat of "community", are they not?
(Reminds me of Johari's window and those blind spots. Things we cannot see ourselves. The unknown. Even though others might see them. )
Good luck with your barn and coffee shop community work.
