Hive Research Lab at AERA 2016

AERA 2016 bannerWhile the blog has a bit quiet in the past months as the Hive Research Lab team has been engaged in data analysis and writing (including two dissertations!), we’ll be presenting a good bit of what we’ve been working on at the upcoming American Educational Research Association (AERA) annual meeting. If you’re coming to the conference, we invite you to check out our sessions (details below). The presentations cover all three of our core research areas, including better understanding the relationships between organizational networks and educational innovation, youth pathways and trajectories around digital making and the design of research-practice partnerships in educational networks.


Saturday April 9th

Enabling Equitable Knowledge Building in Research-Practice Partnerships through Community White Paper Routines

 

Authors: Rafi Santo (Indiana University), Dixie Ching (New York University), Kylie Peppler (Indiana University), Chris Hoadley (New York University)

Time: 4:05-5:35

Structured Poster Session: Strategies for Promoting and Studying Equity in Design-Oriented Research-Practice Partnerships

Chairs: William Penuel (CU-Boulder), Jean Ryoo (Exploratorium), Molly Shea (CU-Boulder)

Location: Convention Center, Level One, Room 102 B


Monday April 11th

“A City-wide Laboratory”: Scaling Digital Learning through Inter-organizational Collaboration in the Hive NYC Learning Network

 

Authors: Rafi Santo (Indiana University), Kylie Peppler (Indiana University), Dixie Ching (New York University), Chris Hoadley (New York University).

Time: 11:45-1:15

Symposium: Scaling Up Digital Media Innovations

Chair: Cynthia Coburn (Northwestern University)

Location: Marriott Marquis, Level Four, Capitol


Tuesday April 12th

Brokering Learning Opportunities within an Out-of-School Network: A Conceptual Model for Supporting Youth Interest-Driven Learning

 

Authors: Dixie Ching (New York University), Rafi Santo (Indiana University),  Chris Hoadley (New York University), Kylie Peppler (Indiana University).

Time: 8:15-9:45

Structured Poster Session: Brokering Future Learning Opportunities: Theoretical and Practical Considerations for Linking Youth to Out-of-School Time Opportunities

Chairs: Denise Nacu (Depaul University), Katie Van Horn (CU-Boulder)

Location: Convention Center, Level One, Room 103 A


Tuesday April 12th

Maybe a Maker Space? How an Out-of-School Center Engaged in Organizational Learning around Maker Education

 

Authors: Rafi Santo (Indiana University), Dixie Ching (New York University), Kylie Peppler (Indiana University), Chris Hoadley (New York University)

Time: 10:35-12:05

Roundtable: Media, Culture and Learning SIG Roundtable – Gaming, Community, and Civic Engagement

Chair: Christopher Carl Blakesley

Location: Convention Center, Level Three, Ballroom A

[report] Leveraging Hive’s Collective Intelligence To Better Measure Connected Learning

Educators at the January 2015 Hive NYC meet-up brainstorm indicators for Connected Learning

Educators at the January 2015 Hive NYC meet-up brainstorm indicators for Connected Learning

This past winter, Hive Research Lab had the opportunity to collaborate with folks from the Connected Learning Research Network (CLRN) Longitudinal Study of Connected Learning team at the University of Colorado-Boulder and SRI International on a process they were going through to improve a survey instrument around Connected Learning. The team, led by Bill Penuel at CU-Boulder, had been working to figure out how the survey could do two things. First, it aimed to measure which Connected Learning design principles a young person experienced in a program or activity. And second, it sought to understand whether that young person was experiencing valued outcomes in four areas: deepening civic engagement, advancing career goals and goal discovery, increasing academic success and bonding to school, and generally a deepening in satisfaction, fulfillment and joy.Screen Shot 2015-05-18 at 3.30.58 PM

But measurement instruments can only be effective if there’s clarity around what’s being measured –  similar to how curricula can only be judged as effective once there’s a actually good sense of what you’re trying to teach. At the time they came to us, the survey team’s understanding of outcomes of related to Connected Learning was evolving along with the larger network of scholars that were part of the CLRN. As a result, parts of the instrument weren’t working perfectly, particularly some of the questions related to measuring the valued outcomes mentioned above. So the team was iterating on various parts of the survey, figuring out what was and wasn’t working from a measurement standpoint, what wasn’t matching with qualitative findings and grappling how to address those issues.

The challenge was determining what a young person could say or do that would be a good indicator that certain outcomes were happening for them, while also not creating a situation where every young person taking the survey reported that those outcomes are occurring when it’s not possible for that to be the case (a statistical phenomenon called ‘ceiling effects’).

The approach the CLRN team wanted to take to solving this issue and to improving the survey was to bring in the views of both young people and on the ground educators that are closest to these issues. Rather than going back to the literature, or talking more to statisticians and psychometricians, they took a community-oriented and participatory approach – talk directly to educators and young people about what they’ve heard (or said themselves) that have been indicators that things are going well, or not so well, around a particular area for a young person like being civically involved or having a sense of future career goals. The purpose was both to help clarify the outcomes to be measured and to help design new survey items around those outcomes.

Hive educators share out things they're heard from youth that could be indicators around career outcomes

Hive educators share out things they’re heard from youth that could be indicators around career outcomes

So in a series of workshops that Hive Research Lab helped to design and facilitate in early 2015, youth and adults at the Colorado Alliance for Creative Youth Development and educators from Hive NYC shared perspectives and accounts from their experience. From there, the survey development team, and the broader Connected Learning Research Network, built on what was shared in order to generate new, more valid survey items. The team just finished putting touches on a report that goes into more detail about the process of taking a participatory, community-based approach like this. It also includes the outcomes of the workshops: new survey items that educators can use to measure Connected Learning at their sites. You can check it out here.

Hive Research Lab at the AERA 2015 Annual Meeting

AERA 2015 imageHive Research Lab will be heading to Chicago this week for the American Educational Research Association’s 2015  Annual Meeting.

The theme of the meeting this year is “Toward Justice: Culture, Language and Heritage in Education Research and Praxis”. Check out the official meeting page.

Here’s a run-down of where you can find our work at the conference:

Friday, April 17th

4:05 to 5:35pm, Sheraton, Second Level, Superior A

Symposium: Learning as Transformation: Examining How Youth Author New Learning Pathways/Ecologies in Science, Engineering, and Technology

Chair: Daniel Birmingham, Loyola University Chicago

Presentation: “From Half-Pipe to Full-fillment: Leveraging Interest-Driven Identities as a Strategy for Technology Learning”, Dixie Ching (NYU, presenting author), Rafi Santo (IU), Tal Bar-Zemer (Citylore), Jessica Forsyth (Harold Hunter Foundation), Chris Hoadley (NYU).

 

Saturday, April 18th

Sheraton, Ballroom Level, Sheraton V

Structured Poster Session: New Tools, New Voices: Innovations in Understanding and Analyzing Life-Wide Ecologies for Youth Interest-Driven Learning

Chairs: Kylie Peppler (IU), Chris Hoadley (NYU)

Poster: Affordances of Social Learning Ecology Maps for Examining the Importance of Social Support in the Pursuit of Digital Media Making Activities Dixie Ching, NYU (presenting author); Rafi Santo, IU; Kylie Peppler, IU; Christopher Hoadley, NYU

HRL@Mozfest2014: A Network of Networks, a Hive of Hives

Global Hive Meetup @ MozFest 2014
Photo by Mozilla Hive NYC Learning Network

As we’ve studied Hive NYC, one of the big achievements we’ve seen is what can happen when previously unconnected educational organizations come together from across a given city. Organizations just miles apart geographically may have previously seen themselves as being in different sub-fields, but in Hive created a shared community platform where they can exchange ideas, find the places they have common values and organize around collective goals. One of the biggest value propositions of Hive, as contrasted with, say, broader field-level structures like conferences or online communities, is its locality. From an organizational perspective, it’s an ongoing field-level community located right in the backyard of your city. This core feature, and value, of locality is what sets Hive apart in many ways in an increasingly ‘flat’ virtual world.

And so the idea of having a ‘global’ Hive movement can seem antithetical to this local commitment. The whole point of a Hive is rooted in the idea of educators and organizations being able to physically come together with regularity, easily collaborate and design solutions to local problems – wouldn’t some sort of ‘Hive global’ be missing the point that the model made in the first place?

What I saw at Mozfest made me think about this issue in a new way. It made me realize just how powerful it is to have interaction among the growing number of cities that have Hives, be they full fledged Hive Learning Networks (in NYC, Chicago, Pittsburgh and Toronto), emerging Hive Learning Communities (in Kansas City, Chattanooga, San Francisco-Bay Area, India) or bunches of others in places that are exploring like Denver, Seattle, Vancouver, Manchester, Berlin and elsewhere. What I saw at Mozfest was not some sort of unitary ‘Hive Global’, some singular über-community that subsumes the commitment to local urban community. Rather, the Hive track represented a ‘Hive of Hives’, with Hive members and leaders sharing, discussing and creating solutions to shared problems together, using common language and based in shared values.

Kevin Miklasz, a Hive NYC member, reflected beautifully on the power of Hives coming together in his post about the festival:

It’s like that feeling you have when you meet up with an old friend and pick up right where you left off, as if no time had gone by. Except we had never met before, but we were able to just pick things up and start going anyway. We talked about the same things, using the same language and ideas, and were comfortable using the same kinds of open-ended design-thinking techniques to work through problems.

The rich interactions amongst ‘Hivers’ at Mozfest was what has convinced me of the value of having a ‘network of networks’. People were not only able to talk about issues that were relevant to actors in every city (e.g., what are effective strategies for making learning relevant for youth?), but also for sharing ideas about how a Hive might work, and how it might organize itself (e.g., what are the ways that your Hive designs opportunities for members to share ideas?). The former example, sharing thoughts on common issues like pedagogical approaches, is certainly powerful. But it is also the kind of thing that happens at many conferences and workshops. But that latter issue – on how Hives might facilitate and organize themselves – this is the place where a ‘network of networks’ can act as a context to innovate on the Hive model, to share thoughts on what works and doesn’t when it comes to building a Hive in a city. A Hive of Hives acts as an infrastructure for innovation on the idea of Hive itself.

As more cities explore organizing Hives in their backyards, it will be critical that there is a common space where the ‘theory of Hive’ and the ideas that Hives care about can be discussed, debated, learned from and about, and most importantly, advanced. Just as a local Hive is a great opportunity for educational organizations and actors that are committed to exploring and inventing around ideas of Connected Learning and web literacy, a Hive of Hives presents a critical opportunity for those committed to such city-based models to share knowledge and collectively level up what Hives around the world can be.

[white paper] What does it mean to Work Open in Hive NYC? New Hive community whitepaper

Working Open Whitepaper front page imageIn the summer of 2014, a group of Hive NYC members and stakeholders came together to think, talk and design around the idea of ‘working open’ in the Hive. The context was the Network that Learns design charrette. Led by Hive Research Lab, we created this two day sprint as a space to address key knowledge management issues that were brought up State of the Hive meeting in March 2014. Members had voiced needs around making it easier to locate expertise in the network, continually capture best practices from ongoing projects, and figure out how the network could be a context for accumulating collective wisdom. In our ongoing work studying innovation in a ‘networked’ context, we’ve see how just such issues can be critical in supporting organizations to effectively leverage Hive as they dive into new areas of work and strengthen existing ones, and so the charrette acted as a space where we could collectively think them through.

We used the idea of ‘working open’ during the charrette as somewhat of a grounding theme that cut across these knowledge management issues. As a way of working that values collaboration, failing early and often, ongoing storytelling, community building and an experimental and flexible spirit, ‘working open’ felt like it might have something to offer these issues that were raised at the State of the Hive. If understood and supported in a way that speaks to the Hive community’s distinct context, working open could be considered a mode of engagement that allows Hive members to progress both individually and collectively.

The white paper we’re sharing now [pdf] is the result of those discussions, and aims to synthesize many views about what constitutes ‘working open’, what it looks like in practice, what tensions are involved and what this all might mean for the Hive NYC community. Drawing on the voices of participating Hive members, as well as our field research in Hive NYC, the white paper offers somewhat of a vision for collective organizational learning within the network. We’d love to hear what you think, and how you could see these ideas applying to the way you do your everyday work.

[brief] Networked Innovation Interim Brief #3 – How does a learning network spread and scale innovations?

spread widelyHow to spread and scale work coming out of Hive NYC has always been a central question for the network. We’ve regularly referenced [pdf] how Mark Surman, CEO of the Mozilla Foundation, early on envisioned Hives as “both R&D and retail”; simultaneously incubators for new approaches to learning as well as a means through which those innovations might travel. More recently, Cynthia Coburn has been writing [pdf] and speaking with the network about how we might begin to conceptualize these issues of spread and scale in a digital age. While the focus of our Networked Innovation research has generally been on the practices associated with early stage innovation design, Hive Research Lab naturally comes across the strategies and practices that member organizations utilize to spread and scale the innovations they design, and in this brief aims to share some of what we’ve seen in our fieldwork in that area. Through naming the approaches we’ve seen, our hope is that the network, as a collective, might better think about how it might accomplish more together than any individual organization might accomplish alone.

NI Brief 3 Cover FinalIn this interim brief [pdf], which we excerpt below and can be downloaded by clicking the image to the left, we give a very brief overview of some of Cynthia Coburn’s framework on spread and scale, then use to bulk of the brief to describe some of what we’ve seen in Hive both in terms of what’s spreading as well as strategies organizations employ to spread work, and finally close with some questions that organizations might ask themselves as they consider these issues internally.

From the brief:

Strategies for Achieving Scale by Hive Member Organizations

As we’ve researched Hive NYC, we’ve come to see a number of distinct strategies that organizations take to spreading their work. In each, different forms of innovations are being spread, different target groups are envisioned (sometimes youth directly, sometimes educators broadly, sometimes specific types of “adopter” organizations, etc.), and different levels and types of capacity are needed to pull off the approach. The following typology is by no means exhaustive, but covers many of the approaches we’ve encountered.

  1. “Physical” Footprint Expansion – this strategy involves increasing the number of physical sites where an organization’s youth-facing programs or pedagogies are implemented. This approach can be achieved through internal growth of an organization in terms of the number of front-line educators it employs, and sites where it implements its work. This strategy often involves intensive partnership and training of “adopter” organizations that implement the model or curricula that was developed by a “base” organization. Such a strategy might also involve the development of online platforms that support multi-site implementation.
  2. “Virtual” Footprint Expansion – in this strategy, we refer specifically to online educational experiences that are aimed directly at young people. Educational video games, online communities, any type of targeted online “content” meant for uptake by youth can be put under this umbrella. This approach is distinct in that it likely means that an organization increasingly develops capacity in areas such as web development, interactive content development, digital design and online community management.
  3. Open Education Resources (OER) Distribution – while it’s an older term, “Open Educational Resources” well describes the kinds of things that many organizations aim to spread for usage by other individual educators or organizations. Well structured curricula, “teaching kits”, digital design tools or games that can be incorporated into an existing curricula, activity templates or even educational design principles all might be considered under this umbrella.
  4. Face-to-Face Professional Development and Consulting PD and consulting are well established approaches taken by specialist educational organizations to capitalize on and spread their distinct capacities and resources. Professional development events often focus on more generalizable innovation forms so as to be more widely applicable and attended by educators from a variety of contexts. PD offerings might combine sharing pedagogical approaches and design principles, with exposing trainees to existing knowledge channels relating to a particular area. Consulting, naturally, is often more intensive and tailored to the needs of a client organization.
  5. Thought Leadership – many Hive organizations actively play a role contributing to and even shaping the discourse within various communities and fields as “thought leaders” that are looked to around particular areas of expertise. Such an approach might leverage public speaking in a range of venues such as conferences as well as regular writing and publication whether it be through white papers, on widely read blogs or in various media outlets. A thought leadership approach leverages some core expertise with facility at communication and framing in order to spread ideas and practices.
  6. Working Open – We’re hesitant to position working open simply as a strategy for spreading innovation, as in many ways it can be seen as a particular configuration of innovation practices (coming from the open source software movement) that values iterative co-development of innovations with a range of stakeholders in a transparent way. At the same time, this approach, which values cultivating community during the design process, could be seen as a strategy that simultaneously develops and spreads innovation.

From an ecosystem perspective, organizations can of course play different roles in relation to these strategies. Some might be looking for other organizations to implement programs developed in-house, others might be looking for online distribution partners to help spread resources they’re developing, and some might even help others to build capacity towards spread and scale itself, as in cases where an organization with greater curriculum development capacity assists another organization to ready an program from broader uptake. One of the advantages of coming at these issues from a “networked” perspective is that it can allow organizations to ask questions about what role they do or want to play in the larger eco-system, as well as how existing actors in the ecosystem can play roles that allow their own organizations to “punch above their weight”, so to speak.

Five questions Hive organizations can ask about spread and scale

As organizations wrestle with these issues, there are a number of basic questions they can ask and bring into internal strategy conversations regarding spread and scale:

  1. What form(s) of innovation I am trying to spread?
  2. What conception(s) of scale am I aiming to achieve and how do they impact my strategy? Does my organization envision adoption, replication, adaptation, reinvention or some combination thereof as being applicable to spreading its work?
  3. What changes need to be made to the innovation I’m trying to spread, the context I’m trying to spread to or through, and to my own organization in order to make spread viable?
  4. How am I going to learn from past attempts at spreading work, both from my own organization as well as others, as I engage in a scaling strategy? How am I going to learn while I’m engaging in a current or future strategy so that course corrections can be made along the way?
  5. What role(s) can I and do I want to play in the larger Hive ecosystem in terms of spread and scale issues? What are roles I can see other organizations in the ecosystem playing in relation to my own strategy for scale?

As a network, we know that there’s much more that can be done together than alone when it comes to achieving impact. Thinking together about how different organizations might leverage their strengths through strategic partnership is ultimately only the first step – just as we need to prototype, test and refine innovations themselves, we also need to take an experimental approach to achieving scale. Each of the strategies above must leverage distinctive best practices that have been developed both within and outside of the education sector. At the same time, such strategies can only be well achieved in Hive if they’re approached from the same perspective of collective learning and careful observation that’s taken by the network in other areas of its work. Scale brings new challenges, and therefore new things to be learned and shared across the network. Success will likely only be found if Hive continues to be “a network that learns” when it comes to efforts to spread and scale.

 

On Common Language as Hive Network Infrastructure

One of the things I’ve been contemplating recently is the role of language within a network like Hive NYC. Specifically, I’ve been thinking about how common language might actually be considered part of the network’s infrastructure, and how that infrastructure can support achieving a host of collective impact goals. Infrastructure is a notion that I’ve previously used to describe Hive, and I think that it’s apt when it comes to issues of common language’s role in coordinated action.

This has been in the back of my mind for a while, but came to the forefront as Hive Research Lab recently released a number of interim briefs relating to our research within the network, as well as through a number of Hive events that we were involved in – the February meet-up focusing on leveling up project ideas and the recent event on “Unpacking Spread and Scale” with Cynthia Coburn from Northwestern University. In each of these examples, language, and specifically well operationalized and meaning-laden language, played a key role.

In the case of the first “networked innovation” interim brief, we worked with Hive NYC HQ to figure out how ideas we shared in the brief could be infused into a network meet-up focused on issues of innovation. Part of the goal of that meet-up was to help members “level up” their projects through creating scaffolds for articulating their value from number of different perspectives and to contextualize the problems they were solving and for whom. We explicitly used some of the conceptual frameworks from our brief to create activities. Specifically, we talked through how examining different “dimensions” of an innovation — such as value-add, novelty, form, complexity –might allow that innovation to be better understood and described. One positive outcome of this meeting was that people were able to better understand this concept of innovation from the perspective of network facilitators, researchers, and funders. In a context where language around innovation is regularly used in conversations, requests for proposals and community calls, the meeting was a means to give more depth to a communally valued idea in a way that cut across stakeholders.

Similarly, the recent event Cynthia Coburn ran relating to spread and scale was centrally focused on sharing and clearly defining language relating to these issues of organizational reach (you can see a similar talk she gave at the 2014 Digital Media and Learning conference here). A lot of what Cynthia shared was founded on clearly defined constructs which were then built up into a larger dynamic framework. As small example, she describes spread as the means to achieving scale, which describes as a state. In this case spread is the verb that leads to scale, a noun. As she shared her broader framework and the meanings associated with various concepts, I saw Hive members actively use newly introduced language as a means to sort through dilemmas they’re encountering in their work. Having meaningful language was a tool to engage in problem-solving.

An important point that Cynthia alluded to in her talk that’s also good to highlight here is that particular definitions for words are of course contextual and that the idea is not to convince people that a given word must be defined in a give way absolutely. It’s just useful to know that within a given context language is meant by a particular actor in a particular way. This is what allows meaning to emerge when people then ground that language in their experiences, and share across them. On a similar note, part of the conversation around common language also needs to address how common language is negotiated and agreed upon, by whom, and for what purposes. Words and their definitions of course carry values and assumptions, and language can end up silencing and marginalizing, even inadvertently.

In a network like Hive, made up of individuals with diverse backgrounds both culturally as well as professionally, it’s easy for discourse to simply to be experienced as jargon or buzzwords. Innovation. Pathways. Spread. Etc. These words mean nothing if we don’t work out their meaning together. And unless that happens, we fail to truly reach towards collective impact because we talk past each other in a modern tower of Babel. But when we say what we mean and mean what we say through shared language, then we can actually accomplish something in relation to shared goals. I’m only coming to realize now how much of our role at Hive Research Lab is about that. By engaging in basic research on the network, we work out well defined language that actually describes phenomenon in the real world, and by sharing this language we can do a small part in helping the network take steps towards achieving shared vision. We can play a role in co-creating a common linguistic infrastructure for collective problem solving.