OpenIDEO is an open innovation platform. Join our global community to solve big challenges for social good. Sign up, Login or Learn more

MindScribe: Reflective Inquiry through Scaffolded Storytelling for Low-Income and Multilingual Early Childhood Communities

A technology-enhanced stuffed animal that supports young children in crafting & communicating their creative ideas.

Photo of Layne Jackson Hubbard
25 27

Written by

Updates: How has your idea changed or evolved throughout the Prize? What updates have you made to this submission? (1500 characters)

We've learned so much since the beginning of this competition!

OpenIDEO Community
First, we spent a lot of time engaging with the OpenIDEO early childhood innovators and their ideas. Wow! There is a wealth of great ideas and inspired innovators. We were awed by the diversity of their approaches and motivated by their commitments to so many different areas of early childhood development. Our founder has especially loved to see the passion for transforming trauma (ACEs), and she plans to keep in touch to explore opportunities for collaboration.

Multilingual Expansion
Our first big insight came when we realized how our multilingual platform is a game-changer for immigrant and blended families. We knew that we needed to rapidly develop our multilingual support. Since starting the competition, we have increased our support from 3 languages to 11 languages—and counting! We work with local early childhood centers and international housing communities to develop our translations, and prioritize the inclusion of languages spoken by the children and families. Additionally, our founder enrolled in our university's first ever graduate course on Bilingualism. There, she is engaging with a skilled community of Speech Language Pathologists to understand how to better support dual-language learners in early childhood.

Gary Community Investment
Next, we knew that we needed to understand the mission of the competition's funder, Gary Community Investments. We obsessively read every white paper, blog post, article, tweet, data profile, and investment strategy released by GCI. We created a Slack channel to organize our findings and reviewed their content with our team. We knew that, to support GCI's mission, we needed to effectively support School Readiness for children from Colorado's low-income families.

Evidence-based Practices
Thus, we wanted to explore new evidence-based practices to gauge our effectiveness and inform our iterations. First, we met with several experts in language development and language acquisition, and learned ways that we could measure the impact of our intervention on children's language development. With these new tools, we analyzed our database of children's stories, and are now creating programmatic scripts to scale our analysis over time.

Language Development
In our language analyses, we are exploring ways to measure the impact of our intervention. One such measure, Mean Length of Utterance, or MLU, approximates the child's language complexity and grammatical structure. In our analysis, we found that 3-year-old children's MLU during MindScribe storytelling is higher than average. 3-year-old children in our case studies had an average MLU value of 6.26. However, the expected MLU for this age is typically in the 3.75 to 4.5 range (https://www.speechandlanguagekids.com/increasing-sentence-length-mlu/).

Prototyping & Experimentation
Finally, we wanted to expand our prototyping abilities beyond software development. Three new members joined our team this Winter, with skills in sewing, tactile fabrication, electronics, and microprocessors. We're now creating new stuffed animal characters and experimenting with customizable patterns to support the diverse needs of the communities we serve. For instance, one of our colleagues is blind, and has been sharing insights on how to make our prototypes more tactilely engaging to support blind and sighted children alike. We are integrating these insights into testable designs by exploring tactile inputs in the stuffed animal's paws.

Toolkits
We've loved exploring OpenIDEO's many toolkits. As we build a team of interdisciplinary students, these toolkits provide accessible ways for our team to unite under human-centered principles (see the prototyping section for scenes from our brainstorms). We especially enjoy the Storytelling Toolkit, and we wonder if our MindScribe Kit is an analogous offering for preliterate explorers. ;)

Name or Organization

University of Colorado Foundation.

https://giving.cu.edu/about-us/university-colorado-foundation

Geography

MindScribe is headquartered in Boulder, Colorado and collaborates nationally.

What is your stage of development?

  • Early Stage Innovator, with at least one-year experience in ECD

Type

  • University

What is the stage of your proposal?

  • Piloting: I have started to implement my solution as a whole with a first set of real users.

Describe how your solution could be a game-changer for your selected Opportunity Area (600 characters)

Young, preliterate children lack diverse opportunities for reflection—an essential aspect of learning.

To support young children in reflecting on their creative play and developing their prowess as innovators and ideators, we leverage the ancient tradition of oral storytelling with the emerging technology of voice interaction.

Using MindScribe's voice scaffold, children express their ideas, play with complex thoughts, develop their language and communication, share their perspective, and are better understood by their community. MindScribe empowers children's own innovation & creation.

Select an Innovation Target

  • Technology-enabled: Existing approach is more effective or scalable with the addition of technology

Tell us more about your innovation (1500 characters)

MindScribe stuffed animals invite young, preliterate children to tell a story about their creative play—and scaffold their construction of complex ideas & cohesive expression. This supports children in (1) reflecting on their creations and learnings, (2) communicating their perspectives, and (3) deepening their engagement with sustained creative play. Through these three components, MindScribe supports children's social-emotional, language, and cognitive development.

Here's how it works.

Scenario
When a child is engaged in creating something—whether a crayon drawing, block building, play dough rendering, or finger painting—a parent, teacher, caregiver, or sibling invites them to "tell a story" about their creation. They open our MindScribe voice interaction application on their mobile device, and then insert the device into one of our affordable, customizable stuffed animals. The stuffed animal then sits with the child and asks them questions about their creation—and helps them construct an original, imaginative story about their ideas.

As the child tells their story, they are practicing their language and communication skills, and are engaging with complex, open-ended questions. Through these skills, they are developing their ability to learn, reflect, and innovate. Not only does this prepare children for school, but it helps prepares them to be leaders in life.

What problem are you aiming to solve? (3 sentences)

We know that reflection is a vital part of learning, but young children lack access to diverse tools for reflection. Books support reading, crayons support making, but what toys or tools support reflecting?

Our investigations have found that many adults underestimate young children's abilities & are not yet comfortable supporting open-ended, reflective inquiry with young children.

We want to model this change & we're creating a tool to support transformational reflective interaction.

Explain your idea (5000 characters)

Reflecting
Reflective inquiry with MindScribe is composed of three elements: (1) An open-ended questioning algorithm that scaffolds young, preliterate children to tell a story about their creative & playful activities; (2) a mobile app that affordably delivers the voice interaction in any spoken language; and (3) a simple stuffed animal that houses the technology for a screen-less, environmentally-aware engagement.

Sharing Stories
We're developing ways for children to share their stories by experimenting with play-back features to support intrapersonal growth, as well as book-bound transcriptions to create family-time and story-hour engagements. Children's stories are rich with their social-emotional explorations, and children and caregivers alike are able to learn from the social-emotional content. We've consistently seen how children's stories can transform their understanding of the challenges they face, and create new opportunities for resolution through social-emotional awareness.

Methods
We employ the methods and techniques of human-centered design and rapid prototyping to iteratively build and test our tools. We collaborate with children, families, therapists, and schools to intelligently scale our efforts, and we aim toward co-designing solutions with our stakeholders utilizing a two-generation approach—meaning we work closely with children and caregivers to catalyze transformative change.

Mission
Our goal is to create a transformative platform that can be meaningfully integrated into school and home early childhood environments. We want to instigate cultural change by supporting open-ended, reflective inquiry with young children. We want to empower young children as innovators and ideators to share their unique, important perspectives with the worlds around them.

Videos
Check out some of our early videos, showing our prototyping process with our young co-designers. These children gave us valuable feedback on our prototypes, and helped us better understand how to support diverse innovators.

Pilot: https://www.wevideo.com/view/802740873
Storytelling in Vietnamese: https://www.wevideo.com/view/910912308
Siblings tell stories together: https://www.wevideo.com/view/1076560486

Who benefits? (1500 characters)

Our primary beneficiaries are young, preliterate children. Our foremost aim is to foster reflective, expressive interactions that are meaningful and valuable to these learners. We want to enrich their lives through a deeper connection with themselves and their perspectives, and strengthen their ability to express.

Our secondary beneficiaries encompass the children's caregivers and peers, including parents, teachers, grandparents, speech therapists, nannies, siblings, and classmates. Our aim is to enrich their interactions by helping the children provide them with rich insights into the child's development, interests, needs, and goals.

Our tertiary beneficiaries include researchers and developers who are creating interactions for young children and who seek insight and feedback on the children's experiences with their curriculum. Our aim is to provide a plug-in that strengthens their interaction through the addition of reflection, as well as channels feedback for their own design iteration.

Our founder taught preschool for four years and created deep relationships with the children and caregivers in the preschool ecosystem. She continues to collaborate with this school, as well as with many other schools and families throughout Colorado and California. Our founder is also active in entrepreneurial and academic communities (and was a data engineer for a tech startup) and understands the deep need for research-based approaches and direct feedback from stakeholders.

What kind of impact will your idea have? (1500 characters)

To date, MindScribe has helped 64 young children craft their own original stories (in 3+ languages) in 8 communities across CO & CA. Our preliminary analyses have shown evidence that these storytelling interactions scaffold rich language development, and support complex thinking through new cognitive opportunities. In one such analysis, we found that 3-year-olds who were telling stories with MindScribe showed higher Mean Length of Utterance (a measure of sentence complexity and grammar) than typical 3-year-olds in everyday talk. In another analysis, we found repeated evidence of computational thinking skills such as sequencing, decomposition, abstraction, & algorithms within children's story construction.

Beyond language & cognitive gains, open-ended storytelling with MindScribe provides a holistic opportunity for young children to explore their unique social-emotional landscapes. While children might all participate in a similar craft, such as watercolor finger painting, they craft stories that are socially-emotionally diverse. From bunnies hiding from monsters, to bees taking care of their babies, children use stories to express their experiences and to be better understood.

Over the next 2 years, we aim to pilot our reflective inquiry with at least 20 of the 5000+ early childcare centers in Colorado. By Spring 2020, we will launch our MindScribe storytelling model at scale, as we collaborate with 50 centers on our MindScribe U.S. tour, and launch online for 1000+.

How does or how could your idea impact low-income children? (1500 characters)

Everyday Hardware
One of our primary foci throughout our R&D process is to create an affordable solution that can be easily transferable, updatable, and fixable. We experimented with custom hardware solutions, including Arduino-based platforms, but quickly recognized the burdens on cost, updates, and repair. Thus, we sought to repurpose everyday and upcycled objects to power our MindScribe interactions.

Activity Extension
Similarly, we didn't want to rely on a solution that requested additional time and energy from resource-strapped caregivers, and instead sought to build an activity plug-in that would both extend and enrich existing interactions while modeling reflective inquiry practices that children and caregivers alike can flexibly adopt and make their own.

Multilingual Interactions
We know that low-income families are more adversely affected by language barriers. We created a multilingual system support the experiences of immigrant families.

Low-Income Collaboration
We know that low-income learners may be those most in need of our solution, as scaffolding reflective storytelling is a resource-intensive process. We strive to ensure our product is meaningful and accessible to those most at-risk, and we do so by continually collaborating with several low-income early learning environments, including our local branch of Head Start.

Through careful design, we've created a multilingual, interactive robot that requires no novel hardware beyond a mobile device.

Innovation: What makes your concept innovative? (5000 characters)

Multilingualism!
Through our novel voice interaction platform, we have already translated our algorithm into 11 languages and dialects, and we have many more languages in the pipeline. 21% of children in the U.S. under the age of 5 have a minority language spoken in the home, yet a fourth of these children do not become bilingual. Our platform will flexibly engage children in all of their emerging languages.

Child-Centered
Many innovations in the early childhood space focus primarily on changing the behaviors of parents and caregivers. Our goal is to make sure that we give children an experience that is valuable in and of itself—not just for the data it provides. Our mission prioritizes the child's experience, and we work to ensure our engagement provides an immediately enriching experience as well as long-term gains. We know that intrinsically satisfying rewards instill deep, lasting motivation.

Technology-Enabled
We're leveraging our experiences in engineering, early childhood, and cognitive sciences to create a novel technological platform with a captivating use case—oral storytelling. Our deep understanding of machine intelligence allows us to both expand the possibilities and recognize the limitations. Our technology-enabled solution produces an intervention that wasn't previously possible at scale (multilingual, judgement-free, extraordinarily patient listener) while simultaneously modeling a complex form of communication with young children (open-ended, reflective inquiry) for widespread cultural integration and customization.

Developmentally Appropriate
Many current technological tools and toys for young children are centered around a screen-based application. In our early interviews with speech-language therapists, we were advised to explore screen-less interventions to promote children's socio-emotional awareness. Thus, we utilize a stuffed animal to house the voice interaction. In videos of the children's MindScribe interactions, we see that their eyes are not fixed to a screen—they move their bodies, they look around the room, and they remain sensorily and environmentally aware.

Empowerment
In one of our first pilots, a child worked with MindScribe to tell a story about her drawing. After telling her story, she said she wanted to create another drawing so that she could tell another story. She wanted to create "chapters" in her own book, and she surmised that she would build it over the years. When she grows up, she said, she would publish the book and win a medal.

After she told her story with MindScribe, she wanted to help her younger brother tell a story too. She suggested that we translate the questioning algorithm into Vietnamese for him. Both children told their stories—in English and in Vietnamese, and then our team created story artifacts to help them share their stories with their families.

Her dad told us, "I'm so thankful for what you have created for my children. It has helped me realize that there's so much more I could do when spending time with them, especially since they're changing each day."

MindScribe is a tool of empowerment. Through engaging deeply with their creations, children develop a long-lasting awareness of their unique perspectives and the communication skills to realize their visions. They are empowered as innovators in their own lives.

By engaging with the MindScribe interaction, children show their caregivers new ways to interact with them and support their development. We envision a systemic change as open-ended, reflective inquiry becomes an everyday tool in our intergenerational and intrapersonal communication.

Scale: Describe how your idea could reach a significant number of end-users. (1500 characters)

Market Size
Day care systems in the U.S. (NAICS Code 624410) have total revenues of $47.8 billion over 755,457 institutions (50.2% day care, 39.9% pre-school). Of total revenue, 18.1% of revenue is used for “other” purchases including curriculum, a total addressable market of $8.65B. In the U.S. 2,371,000 children 0-4 were in either preschools or head-start programs (2011).

MindScribe’s target market is Colorado institutions, of which there are roughly 2,266, with total revenues of $143M, and purchasing budget of $25.95M.

Revenue
Revenue comes from the sale and customization of our MindScribe Kit, Shareables, and Curriculum. A basic kit includes an augmentable stuffed animal, a mobile application download, and an instructional storybook. Shareables include printed storybooks created from transcriptions of children's stories. Curriculum includes workshops and custom packages for early childhood centers.

Scale
We aim to reach end users by connecting primarily with early childhood centers and by scaling our engagement with the center families. MindScribe’s go-to-market strategy involves a pilot of 20 Colorado schools in year 1. The goal of this pilot is to build relationships, test the model, integrate feedback, and develop our plan for robust scaling. In year 2, we will expand to 60 schools and 250 in year 3.

Obstacles
Currently, our biggest obstacle to scaling is meeting demand. We have had more requests for collaboration than we can currently fulfill.

Feasibility: Where are you with understanding the feasibility of your idea? Describe what you’ve done so far and your plans. (3000 characters)

Iterative Design
Since Spring 2016, we have prototyped 7+ iterations of MindScribe with 64 children, 5 schools, and 3 families. Our team has included 5 graduate students and 7 undergraduates in multiple engineering disciplines, and has been advised by 10+ professors throughout computer science and cognitive science.

Team
Our team is currently composed of four paid researchers & developers in engineering and computer science, as well as three independent study students, and a host of scientists, educators, entrepreneurs, and children who actively advise our work. We have multiple ongoing collaborations with local Colorado children's centers—including multilingual and low-income preschools—and continue to partner semi-annually with our California schools and families.

Design Collaborations
For each school, we first conduct human-human interaction to support storytelling, then human-puppet interaction, and then human-robot interaction. This helps us familiarize ourselves with the needs of the community and customize the model to support their success. We then integrate these individual findings into our model following principles of Universal Design.

Technology
We continue to improve the flexibility, stability, scalability of our technology. We are simultaneous developing atop two different application platforms (web and hybrid) in order to exploit the advantages of each platform while seeking and understanding each of their limitations. We live by the motto, "Fail Fast, Fail Early, Fail Often" and employ rapid prototyping to quickly test our assumptions. We push each other to always create Minimum Viable Products before sinking development hours into complex, but potentially unnecessary features.

We have created a lean interaction model that viably supports children's development of imaginative stories. Our robotic model supports the construction of original stories on par with our human-human interaction, and we find that both children and caregivers continue to invite us back for more collaboration. Both of our mobile applications are functional and scalable.

Operations
From an operations perspective, MindScribe's core team is funded through at least Spring 2020. Our founder was awarded a National Science Foundation (NSF) fellowship to fund her research and development on MindScribe, which pays her tuition and living stipend. Her advisor, Dr. Tom Yeh, is also committed to supporting the project and has himself been awarded significant grant funding to support his research activities. MindScribe also supports the professional development of many undergraduate and graduate students, who trade their skills for research credits and internship experiences.

Implementation
Over the next two years, we will advance the impact of MindScribe through collaborations with our current and upcoming community partners. We are currently launching our automated applications, which will enable us to significantly scale our partnerships.

Business Viability: How viable is your business model? (5000 characters)

Finances
The financial viability of the business model is rooted in a our revenue stream and a simple grassroots go-to-market strategy. MindScribe will host on-site implementation workshops at Colorado early childhood centers by co-designing with collaborators, building brand ambassadors, and integrating feedback and insights. Stage two of go-to-market includes building a thriving online user community, and delivering the product through our digital storefront and video learning hub.

Expertise
Beyond the product’s viability, our founder’s personal experience and background give MindScribe credibility and vision. Layne’s 4 years as a preschool educator provide a foundation of understanding children and their caregivers—a two-generation approach. Children's House Preschool served children of diverse backgrounds and supported over 30 spoken languages. At least 50% of the children were identified as at-risk and received government support for their education, which required Layne to develop Individual Teaching Plans to support the children's well-being and school-readiness.

Our founder's work on MindScribe is currently funded by a 3-year fellowship (2017-2020) from the National Science Foundation in Human-Computer Interaction. This prestigious award will pave the way for future grant funding to expand MindScribe's opportunities for research and development.

Our founder recently published two academic research papers on the MindScribe model, in the premier international conferences on Human-Robot Interaction and Intelligent User Interfaces. Connections with these and other scientific communities will accelerate MindScribe exposure to constructive, expert feedback and will inform future grant applications.

Challenges
The market size and budget in early education is small. Competing for available spending means we will have to invest significant time up-front with educators to demonstrate our value. Market testing is needed to understand what pricing the market can bear. Ensuring that low-income and at-risk communities are primary beneficiaries is of utmost concern. MindScribe has a goal to maximize accessibility of the product. We will have to work to find creative funding sources to subsidize costs.

Next Steps
To support our grassroots mission, we are drafting designs for an inclusive MindScribe touring bus. This will enable us to share an accessible storytelling experience with underserved communities, and to ensure that their experiences and needs are integrated into our technology designs. We aim to reach both rural and urban communities with this touring van, to hold intergenerational co-design sessions with community members, and to leave each community with a customized MindScribe package for extended use.

HCD: How have you used human centered design to build or refine your concept? (5000 characters)

Co-design is our passion. We don't underestimate the complexity and arduousness of true empathy and collaboration, and thus we engage in continual professional development to advance our skills and destabilize our assumptions. We are steadily building co-design experiences with our core design stakeholders. We're most inspired by co-design sessions that involve intergenerational teams, and we aim to bridge children and caregiver collaboration through our work with MindScribe.

Read our Graduate Research Plan Statement (see attachments) for our full user centered design plan with MindScribe, which includes methods and tools such as participatory design, ethnography, contextual inquiry, rapid prototyping, minimum viable products, and iterative development.

Our founder has attended several IDEO-inspired Design Thinking workshop throughout her undergraduate and graduate studies, and is deeply thankful for the impact you've had on her professional development and career trajectory. To inspire more HCD work, our founder regularly provides guest lectures for college courses and seminars on user centered design, human-centered computing, and co-design.

(Hint: To see more of our HCD, browse through the Prototyping section photos at the bottom of this page).

Tell us more about you (3000 characters)

We have an incredible team.

Our founder, Layne, and her PhD advisor, Dr. Tom Yeh, have been researching together since Fall 2014—years before MindScribe's inception in early 2016.

The idea for MindScribe came from Layne's experience as a preschool teacher, where she taught for four years using the Storybook Journey curriculum. She witnessed the transformative power of children's stories, and the delight as children became empowered as innovators and ideators. The brilliant impact of children's storytelling on both children and their intergenerational communities left an indelible impact on Layne's mind. When she started her PhD in Computer Science, this experience resurfaced, and she knew that she wanted to use her technological mind to scale these storytelling interactions.

Layne's passion for early childhood arises from her own early experiences. Layne lived in several foster homes and SAFE homes when she was 2, 5, 6, 10, and 12 years old. She understands the impact of trauma and she also recognizes the value of empowering children with opportunities to share their perspectives and grapple with their own challenges. Through her observations as an early childhood educator, Layne saw that stories touch on several areas of development—social, emotional, cognitive, and language.

Through storytelling, Layne has seen how children can express their own needs, interests, and challenges. Not only can children can resolve their conflicts by iterating on the themes of their stories, but caregivers can also gain insight into the children's unique perspective to better meet their individual needs. Check out some of the stories children have told with the MindScribe team: mindscribe.org/stories

Read more about Layne's early experiences in foster care in her Personal Statement (see attachments).

Since developing the mission of MindScribe, Layne and Dr. Yeh have had many other students and professors collaborate with the team. We love how early childhood innovation gives collaborators new opportunities to combine their technological skills with their empathetic caring. We've compiled a diverse team of innovators who are each personally motivated to support the incredible, individual growth of young children.

Our team continues to grow, and includes the children co-designers and storytellers who have influenced our work along the way.

Read more on the background of MindScribe here: mindscribe.org/about

Do you have the people and partners you need to do what you’ve described? (600 characters)

We have an amazing team of technologists, researchers, grant writers, caregivers, and prototypers. (Scroll down to the bottom and click "View More" next to the Team to read about each team member's role).

We're continuing to bring new members onto the team, especially those with experience in robotics, software development, voice interaction, early childhood, curriculum development, language acquisition, tactile fabrication, and business development, finance, & marketing.

We're so grateful to be apart of the University of Colorado Boulder's stimulating, entrepreneurial campus.

As you consider your next steps, what kinds of help could you use? Is there a type of expertise that would be most helpful? (1800 characters)

As we grow, we need to scale our business model with industry experts to build an execution plan that both (1) delivers on our promise to support low-income and at-risk communities, and (2) supports sustainable, funded growth and innovation.

We'd love support on developing grassroots collaborations that embed strong, intergenerational co-design partnerships with children and caregivers.

We want to create equitable development, where children and communities are deeply enriched by their efforts in our Research & Design Partnerships.

Looking forward, we'd love to create collaborations that support Trauma-Informed Care. We're inspired by the work of Sesame Workshop and refugee communities, and wonder how we might catalyze trauma-informed care in our own communities.

https://www.npr.org/sections/goatsandsoda/2017/12/29/574532231/with-100-million-grant-sesame-workshop-reaches-out-to-refugee-kids

Would you like mentoring support?

  • Yes

If so, what type of mentoring support do you think you need? (1200 characters)

We would love mentorship on ethical business development and sustainable, scalable implementation.

Are you willing to share your email contact information submitted on OpenIDEO with Gary Community Investments?

  • Yes, share my contact information

[Optional] Biography: Upload your biography. Please include links to relevant information (portfolio, LinkedIn profile, organization website, etc).

Website:
mindscribe.org

Read recommendations for our founder & mission (bottom of page):
f6s.com/mindscribe
f6s.com/laynejacksonhubbard

Read a news article about MindScribe:
colorado.edu/graduateschool/2017/09/27/nsf-winner-designs-voice-interactions-empower-early-childhood-development

Read interviews with our founder:
colorado.edu/cs/2017/11/16/layne-hubbard
cu.edu/news/qa-cu-boulder-grad-layne-jackson-hubbard

Read our founder's LinkedIn page:
linkedin.com/in/dreamlayne

[Optional] Attachments: Please upload relevant attachments or graphics or show us how you prototyped.

Mentorship: How was your idea supported? (5000 characters)

We were so glad to have been mentor matched with our industry connection! She provided a wealth of experience, and even the act of researching her current and prior work beforehand gave us rich insights on the financial landscape of early childhood innovation and investment.

She encouraged us to refine our message by focusing on our primary user group, rather than on our impacts for the caregiving team. We took her advice to heart and brought our focus "back to the basics". Interestingly, this shift has both inspired new areas of exploration for our team (multimodal interactions) as well as kept us laser-focused on our child-centered experience

Additionally, she encouraged us to refine our understanding of our business model and financial trajectory. We've since spent time researching IBISWorld Industry Research Reports and collaborating with business school alumni to materialize our approach.

Finally, she brainstormed how we might build a stronger network of early childhood innovators in Colorado, and how we might connect with professionals who are in more advanced stages of development. We'd love to meet with more entrepreneurial early childhood experts along the Front Range, and hope we can build a cohort of change-makers.

We're excited to keep in touch and to continue learning from her expertise.
View more

Team (8)

Layne Jackson Hubbard's profile
Tom's profile
Tom Yeh

Role added on team:

"Advisor. Tom is Layne's doctoral advisor, a professor at CU Boulder, and has a PhD in computer science from MIT. Tom has ample experience teaching courses in user centered design and human centered computing, and has made significant research contributions in the fields of computer vision, programming languages, & accessible tech. His flagship project supports the creation of tactile picture books for children with visual impairments. https://mashable.com/2014/07/03/3d-printed-book/"

Chen Hao's profile
Chen Hao Cheng

Role added on team:

"Software Engineer. Chen Hao (Johnny) is a Computer Science student with a penchant for web and mobile development. Chen Hao joined our team in Summer 2017, and is rapid prototyping to create Minimum Viable Products which allow us to test our assumptions and iterate our designs. Chen Hao is developing our web application for accessible interactions. Originally from Taiwan, Chen Hao created the Mandarin and Taiwanese dialect translations for our app."

Dylan's profile
Dylan Cassady

Role added on team:

"Data Scientist. Dylan is a Mechanical Engineering and Computer Science student who brings a systematic, organized approach to building our data representations. He works intimately with transcriptions of the children's stories to calculate statistics which inform our language and interactions models. His calculations feed into the development of our software algorithms. Dylan joined our team in Fall 2017."

melP's profile
melP

Role added on team:

"Tactile and Electronics Fabricator. Mel is a Masters student in the ATLAS engineering program. She has ample experience with sewing, tactile inventing, and upcycling, and has been lending her expertise to the lab since Winter 2018. She is augmenting stuffed animals with internal pockets to insert our mobile devices, and is informing our electronics research using her background in rapid prototyping."

Vi's profile
Vi Nguyen

Role added on team:

"Tactile and Electronics Fabricator. Vi is a Marketing and Technology student who has experience creating reproducible clothing patterns for industrial manufacturing. Since Winter 2018, Vi is working with MindScribe to create customizable stuffed animal patterns, and to research dynamic interaction through sensory input. Originally from Vietnam, Vi is creating a Vietnamese translation for the app."

Boskin's profile
Boskin Erkocevic

Role added on team:

"Software Engineer. Boskin is an Electrical and Computer Engineering student who has experience in signal processing. He has been working to develop our mobile application with dynamic voice interactions since Fall 2017, and he utilizes his skills in Test Driven Development (TDD) to create effective, scalable code."

Andrea's profile
Andrea Chamorro

Role added on team:

"Language and Electronics Researcher. Andrea is a Computer Science student who has a passion for mindful prototyping. Originally from Ecuador, Andrea created the Spanish translation for our app in Fall 2017, and brings her user-centered focus to iterating our interaction model. Now, she is experimenting with multimodal interactions through tactile inputs to support children's agency in their storytelling."

View more

Attachments (4)

Graduate Research Plan Statement.pdf

Part One of our founder's National Science Foundation Fellowship application. This graduate research plan outlines her mission for MindScribe. Based on this application, in 2017 she was awarded a 3-year fellowship from the National Science Foundation (NSF) to fund her PhD studies (2017-2020) and her development of MindScribe. http://nsfgrfp.org/

Personal Statement, Relevant Background, and Future Goals.pdf

Part Two of our founder's National Science Foundation Fellowship application. This personal statement showcases the motivation, skills, and experiences which inform her work with MindScribe. Based on this application, in 2017 she was awarded a 3-year fellowship from the National Science Foundation (NSF) to fund her PhD studies (2017-2020) and her development of MindScribe. http://nsfgrfp.org/

MindScribe_IUI_2018.pdf

"MindScribe: Toward Intelligently Augmented Interactions in Highly Variable Early Childhood Environments." Our research on MindScribe was recently accepted for publication in the upcoming 2018 conference on Intelligent User Interfaces in Tokyo, Japan. Our paper will be published Association for Computing Machinery (ACM) Digital Library in March 2018. iui.acm.org/2018/

MindScribe_HRI_2018.pdf

"MindScribe: Reflective Inquiry through Scaffolded Storytelling for Low-Income and Multilingual Early Childhood Communities." Our research on MindScribe was recently accepted for publication in the upcoming 2018 conference on Human Robot Interaction in Chicago, IL. Our paper will be published Association for Computing Machinery (ACM) Digital Library in March 2018. humanrobotinteraction.org/2018

25 comments

Join the conversation:

Comment
Photo of Sophia Pan
Team

Great idea, and I love including children as co-designers. It's so easy to dismiss the abilities of children, but that is when I think we are the most creative as humans so what a powerful and great way to tap into that. Well done!

Photo of Ryan Fletcher
Team

ENOUGH IS ENOUGH FOR THE WISE,PROSPER LOAN FIRM IS THE ONLY WAY OUT FROM ANY FINANCIAL DIFFICULTIES { prosperloanfirm@yahoo.com }
.Greetings great mind, It will be my greatest pleasure saving Individuals and companies from extortion, I know not everyone will want to take it upon their self to reveal the bitter truth about online Lending because of insecurity, Time of doing it and all that. I Ryan F. Tyler, an outstanding businessman who reside at Oakland, California has taken it upon himself to share this article to whom it may concern so they can learn and educate their self from it . It's bad to the extend where you see testimonies online about getting a loan and it turn out to be fake. For real i have fallen for that tricks countless time to the extend i lost almost 65,000usd in total all in the name of getting a loan to invest in a very lucrative business. After so many failed attempt of securing a loan, I and my Manager went online to do a thorough search and found this company PROSPER LOAN FIRM +1(816) 366-8769 ) but before giving it a try we also went ahead to make sure they are registered under BBB, check their reviews and also go to their headquarter to confirm their existence and capability.We were so careful because we didn't wanted to lose any dime again and to our greatest expectation they deliver according to their review and grant us our desire loan amount of 510,000.00Usd. A word to everyone out there when it comes to Online Lenders contact only PROSPER LOAN FIRM via email: { prosperloanfirm@yahoo.com } or drop a message to them via text at +1(816) 366-8769 and consider all your financial problems tackled and solved.#SHARE, You can rescue someone from been victimized today, Thanks.

Photo of An Old Friend
Team

https://amyshoop.com/cara-menggugurkan-kandungan/

https://amyshoop.com

https://amyshoop.com/jual-obat-aborsi-jakarta/

https://amyshoop.com/jual-obat-aborsi-kediri/

https://amyshoop.com/jual-obat-aborsi-malang/

https://amyshoop.com/jual-obat-aborsi-semarang/

https://amyshoop.com/jual-obat-aborsi-jember/

https://amyshoop.com/jual-obat-aborsi-balikpapan/

https://amyshoop.com/jual-obat-aborsi-bandung/

https://amyshoop.com/jual-obat-aborsi-bekasi/

https://amyshoop.com/jual-obat-aborsi-batam/

https://amyshoop.com/jual-obat-aborsi-makassar/

https://amyshoop.com/jual-obat-aborsi-denpasar-bali/

https://amyshoop.com/jual-obat-aborsi-palembang/
https://amyshoop.com/jual-obat-aborsi-surabaya/

https://amyshoop.com/jual-obat-aborsi-hongkong/

https://amyshoop.com/jual-obat-aborsi-taiwan/

https://amyshoop.com/jual-obat-aborsi-malaysia/

Photo of DeletedUser
Team

DeletedUser

I love your MindScribe idea! Congratulations for winning the prize in the Student Design Competition! I have been following your project and watched your three videos (pilot; storytelling in Vietnamese; and siblings tell stories together). I am curious to know how the mobile app works in terms of when to speak and when to pause. Can the app detect the end of the child’s turn? Are the robot questions pre-set or can they be modified by the child’s carer? I look forward to hearing your reply.

Photo of Layne Jackson Hubbard
Team

Thanks for the kind words, Winnie!

Good questions. In early pilots, we actually used the Wizard-of-Oz style of prototyping in order to quickly gain insights about the interaction experience before sinking time and resources into costly development. This allowed us to test our interaction model and rapidly iterate on our design.

https://en.wikipedia.org/wiki/Wizard_of_Oz_experiment
https://www.youtube.com/watch?v=VPVec5eupTY
https://blog.prototypr.io/wizard-of-oz-prototyping-process-blog-a20ffce8886
http://futureofstuffchallenge.org/download/prototype/bootleg-wizardofoz.pdf

To implement this interaction model in our mobile application technology, we first needed to analyze audio and video recordings of the storytelling interactions to determine the architecture. It was a lot of data!

Now, we've been working on calibrating our voice interaction to handle the turn taking. It's not easy. However, we've been inspired by our interviews with several speech language pathologists (therapists) who work with bilingual children. Our tenets for development is that the robot shouldn't interrupt the child, and should give them enough time to think through their responses.

What challenges do you face in your work?

Best,
Layne

Photo of DeletedUser
Team

DeletedUser

Hi Layne,
Thanks so much for your reply and the clear explanation about the pilot stages of the MindScribe technology. As a postgraduate business student, I can fully understand your strategies in terms of product prototyping. Now I understand why I thought the conversation in the videos took place with a wizard behind the curtain so to speak.
Like you, I also think it is a good idea to leave enough time for bilingual children to respond to the robot. I grew up as a bilingual child myself with Cantonese as my mother tongue. Although I am now fluent in both English and Chinese, I can imagine I would have needed more time to find the right English words to create a story considering the vocabulary in a second language could be quite limited at a young age.
My challenges at the moment are to study how innovative designs on open and collaborative platforms can translate into reality. Therefore, I am following different product designs and their business models such as yours. Thank you again for answering my questions and I send my best wishes to you and your team!

Photo of Dahalia Jackson
Team

I love the ideas for MindScribe and the fact that all children could benefit is a plus. You'll get my support just by including children who are blind and those with other disabilities. Consider the universal impact that this initiative will have. Totally awesome!
Thank you for developing and sharing. I look forward to learning more.

Photo of Layne Jackson Hubbard
Team

Thanks for the encouragement, Dahalia!

We've been collaborating with one of our research colleagues, who is blind, to gain her insight and feedback on how to develop our prototypes for blind and visually impaired learners. She has given us several key takeaways:

(1) Texture is to the blind what color is to the sighted—it helps us distinguish between elements, pieces, and features. So while a stuffed animal might be tactile, if it lacks rich textural contrast then it will feel more like a soft mass than a specific animal or creature. So, we've been working with our sewing interns to design new stuffed animals that exhibit more contrast in their features.

(2) We must support children's agency in their interactions. While this is important for all children, it is especially important for blind and visually impaired learners (and is a principle of inclusive design). Thus, we are working to extend our voice interaction with multimodal tactile inputs to give children more agency in directing the flow of their stories. And yes, we use the tenets of universal and inclusive design to guide our process—meaning we apply the lessons we learn from our diverse populations and integrate those insights into our prototypes for all.

http://universaldesign.ie/What-is-Universal-Design/
http://www.inclusivedesigntoolkit.com/whatis/whatis.html
https://idrc.ocadu.ca/about-the-idrc/49-resources/online-resources/articles-and-papers/443-whatisinclusivedesign
https://inclusivedesignprinciples.org/

Thanks for advocating.

Keep in touch,
Layne

Photo of Hilary null
Team

This is a neat idea! It reminded me of the great work that the LENA Foundation is doing in Boulder. I wonder if there might be opportunities for colloboration!

Photo of Layne Jackson Hubbard
Team

Thanks for the feedback, Hilary! We've been following LENA's work too, and are impressed with how they've scaled their mission. I also checked out your submission for the Mobile Curiosity Lab; keep in touch as it develops, I think there could be an opportunity for us to collaborate as well.

https://challenges.openideo.com/challenge/ecprize/evaluation/st-vrainnovation-the-curiosity-lab-project

Speaking of, I met Dr. Roy Pea last year at the conference on Interaction Design and Children in Palo Alto. He told me about a mobile lab from the past wherein his team travelled with a laser cutting machine and helped children create their own designs. It sounded so intriguing!

http://idc2017.stanford.edu/
https://tltl.stanford.edu/people/roy-pea

Best,
Layne

Photo of Andrea Chamorro
Team

On March 6, Mindscribe presented our paper “MindScribe: Reflective Inquiry through Scaffolded Storytelling for Low-Income and Multilingual Early Childhood Communities” at the ACM Conference on Human Robot Interactions (HRI2018).

Our paper was selectively accepted and is now published in the ACM Media Library, find it here: (https://dl.acm.org/citation.cfm?id=3177823)

Additionally, Mindscribe demoed as one of 19 research finalists at the Student Design Competition at HRI. Layne Hubbard, Ph.D, Andrea Chamorro, B.S and Vi Nguyen, B.A were completely and partially funded to demo at the Conference by the University of Colorado Boulder.

Our demonstration yielded valuable feedback from professors, industry leaders, graduate students, roboticists, and parents, many of whom asked one resounding question: “Is this a business?” The quantity of positive feedback lets us believe people see value in our research and believe it could find valuable in the market.

Layne and Andrea attended the Robots for Learning Workshop (R4L) @ HRI2018 (https://r4l.epfl.ch/HRI2018), where we gained insightful knowledge from leading experts into: methodologies for measuring engagement and interaction of children, specially in the context of learning, the perception children have towards learning companions and learning tools, and how best to accommodate children’s learning and creativity in the context of learning robots.
Overall, this was an insightful and incredibly forwarding conference for our members, who, having been invigorated and having found a new sense of direction, will continue to push our research and strive for our mission.

Photo of Layne Jackson Hubbard
Team

Yes! This was so exciting—thanks for sharing with the OpenIDEO community, Andrea.

And—we won 2nd prize in the Student Design Competition!! This award also gave us an honorarium to put toward our equipment expenses.

HRI SDC: http://humanrobotinteraction.org/2018/sdc/

This was an incredible opportunity for our first showing at the HRI conference, and I'm glad we made the most of it.

And, as a freshman in computer science, you especially rose to the challenge. I know it wasn't easy to juggle your courseload with the intense robotics competition, but I'm so proud of you for seizing the moment.

I'm looking forward to seeing the results of your multimodal experimentation when I return from our MindScribe collaboration in Japan!

Talk soon,
Layne

Photo of Layne Jackson Hubbard
Team

Our latest MindScribe research was published in the ACM Digital Library (Association for Computing Machinery). Check it out:

"MindScribe: Toward Intelligently Augmented Interactions in Highly Variable Early Childhood Environments."

https://dl.acm.org/citation.cfm?id=3180319

Photo of Vi Nguyen
Team

As I worked on MindScribe project, it amaze me when we put effort to incorporate emotions into our tones of conversation we can get very different output of emotional responses.
I worked to acquired multiple translations and recordings of different languages to incorporate into MindScribe and as I was coaching our native speakers, we ran into some difficulties. A couple of people were uncomfortable and felt silly to adapt Matuka's curious and caring tone , one person actually left the recording and did not come back.
I believe that there is a fear factor for people when they are asked to act more like children or child-friendly. Most people don't have a problem with joking around but they are not comfortable with behaving in a child-like manner such as curious or embracing their emotional side. Through this project, I understood the disconnection between adults and children a little bit more, moving forward I hope this knowledge will help me in visualize a solution to bridge the gap.

Photo of Layne Jackson Hubbard
Team

How interesting!

1. Thank you for your diligence in bringing new languages onto our platform—from Dutch to Arabic to Russian to Vietnamese, we're increasingly able to serve the 350+ languages spoken in the U.S. You've helped us scale fast!

2. This is a great reflection on the gap that caregivers often face when tasked with creating rich interactions for their young ones—how do we regain our comfort and connection with childhood? Our goal is to model open-ended, curious, imaginative inquiry, and to scaffold these skills for children and caregivers alike.

Let me know if you have more ideas on how we might better reach this goal. It could be a compelling investigation for your independent study.

Talk soon,
Layne

Photo of Dylan Cassady
Team

I work on language analysis for the MindScribe research team. A large part of my work involves transcribing stories created by young children. I analyze the language that the children use in their stories to identify how the MindScribe tool supports children's agency and promotes meaningful interactions. I also identify ways to improve the tool in future iterations.

One thing which surprised me when I first started working on the project was how much the children came out of their shells while telling stories with Stripes, the stuffed animal that the team brought to the preschools. Often, the children will be quiet and shy around adults but will create intricate stories using incredibly creative language. When I first started working on the project, Layne told me that one of her colleagues was skeptical of the virtues of talking to a stuffed animal. She encouraged him to try talking to Stripes for a bit and he was still going strong 20 minutes later.

Photo of Layne Jackson Hubbard
Team

You remembered that story! It really was a fun MindScribe moment. Did you know that that colleague is also the person who voiced the MindScribe (Wizard-of-Oz) prototype for the Bixby school collaboration? He did an excellent job with the sentiment and timing.

I've loved that unexpected finding with children's openness and the stuffed animals too.

I think we all originally expected that the stuffed animals might be a practical solution to supporting storytelling for young children, but we may have underestimated how stuffed animals can deliver on supporting diverse social-emotional inclinations. It's been amazing to experiment with ways that we can leverage the non-judgmental assets, especially by helping children feel confident in sharing their ideas with others.

Thanks Dylan Cassady for your diligence and patience in working with our growing database and making quantitative sense of the language data. Your analyses have contributed directly to our application development, as well as to our interaction model. Your case studies, response statistics, and MLU analyses have been especially interesting. By lending your organizational and structural mind to our database, you are creating a lasting legacy to support future findings.

See you tomorrow,
Layne

Photo of Boskin Erkocevic
Team

I'm a part of the MindScribe research team, and what drew my interests to the were the support for multilingualism. I come from a family that speaks 3 different languages (English, Cantonese, and Bosnian) in our household, yet I only came to learn English, and that fact bites me whenever I try to communicate with my grandparents who don't speak English on either side. I want children in multilingual families to be able to connect with their extended family and culture rather than leave it behind.

I work on building smarter apps that help make the interaction with the stuffed animal smoother using audio signal processing techniques I have learned in my electrical engineering classes.

This work has helped me apply theories I have learned in class while also posing the challenge of creating an audio processing app that works in a variety of environments (noisy, quiet, etc.). I have also learned a lot about designing products around user-friendliness and not just algorithmic functionality.

Photo of Layne Jackson Hubbard
Team

Thanks for sharing more of your story, Boskin Erkocevic . As we've watched and discussed videos on bilingualism experiences during our MindScribe team meetings, it's been equally insightful to hear your reflections on your own language experiences. They've added a lot to our whole team's understanding of our impact.

In my graduate course on Communication Disorders in Bilingual Children and Adults, I've been learning how not only does bilingualism support impact cognitive and language skills, but it also strongly impacts the social and emotional dynamics of family bonding.

Here are a couple videos of family experiences:

https://www.youtube.com/watch?v=8IBfgUNpRsY

https://www.youtube.com/watch?v=qq72I-t204c

(I love the story at 8:15 in the second video!)

Thank you for all of your efforts to advance MindScribe's support for dynamic listening. We know that non-judgmental, patient *listening* is powerful feature of our platform, and it's been such a thrill to see it come to life. I'm glad that you've been able to learn a lot from our lab's user centered design focus, and we've similarly learned a lot from your electrical engineering background.

Talk soon,
Layne

Photo of Crystal Loose Ed. D.
Team

Dear Layne,

I really like your idea for interactive story telling with toys. Young children are engaged with toys during play and this is a unique opportunity to turn play in to a vocabulary building opportunity. The ability to speak directly to the animals is a unique avenue to engage children in the oral language process. Your connection to families learning language is helpful, as there are many that can benefit from your work.

Great job,

Crystal
planttheseedoflearning.org

Photo of Layne Jackson Hubbard
Team

Dear Crystal,

Thank you so much for reaching out!

This work was inspired by my time teaching preschool—we learn so much about ourselves and each other when we tell stories about our creations! Interestingly, this is also a big theme in professional science communication (#scicomm) right now. It seems as though scientific studies are also more deeply examined and better communicated through the mediums of expressive and creative storytelling. I love how the lessons we relearn as adults can also be found in childhood.

Here are some interesting dissections connecting science, art, and Nobel Prize winners.

https://www.inc.com/nicolas-cole/want-to-win-a-nobel-prize-pick-up-an-artistic-hobby-says-science.html

https://priceonomics.com/the-correlation-between-arts-and-crafts-and-a/

Thanks again for your support, and for seeing the value in our work. We've had a lot of questions along the way (will children want to talk with stuffed animals?) and the tools of co-design, HCD, and rapid prototyping have helped us find insights and keep progressing.

I'd love to hear more about your experiences with family engagement. This Family Creative Learning workshop model (the guide is open source and free to use and remix!) by Dr. Ricarose Roque has deepened my understanding of collaborating across generations.

http://familycreativelearning.org/

http://familycreativelearning.org/guide/

Very best,
Layne

Photo of Chen Hao Cheng
Team

I'm software engineering research assistant for MindScribe and here's something I have learned.

Language is an identity, representing a store of society's culture. It's more than just a communication tool. As you speak a different language, you can realize its culture. We can have started building international values since early childhood to form children's wild imaginative world.

Well-designed reflective inquiry about their artifacts helps children think deeply and answer questions logically. Children's curiosity is huge, thus, we could support them to explore a different language.

Word is, since English has been an international language for decades, American people do not really try to learn a different language. However, there are people that actually want to learn another language, but they just don't have a chance. Our project could offer this opportunity to let them explore a new different language.

There are people exposed in bilingualism family, however, after children get to school, they become monolingual because most students/teachers speak the national language to each other. Our project can help them expand their ability to communicate in multiple languages.

Photo of Layne Jackson Hubbard
Team

Dear Chen Hao Cheng ,

Thank you for sharing your reflections on language, culture, and multilingualism.

I too have been learning a lot on this subject—through our project especially and through a graduate course on Communication Disorders in Bilingual Children and Adults. For instance, research shows that children who are stronger in their first language more easily acquire their second language (the same might be said of programming languages too!). I love that our work can both (1) help children advance their communication with their family in their first language, and (2) better prepare them for learning their second language when they start school.

Thank you so much for your work on MindScribe. Since July 2017, you have constantly been seeking ways to prototype and improve our technological platform. I especially appreciate how you have spearheaded efforts to advance our multilingual offerings. 11 languages and counting!

Talk soon, Layne

Photo of Vi Nguyen
Team

I met an engineering student who is a non-traditional student. He told me his life story coming from the slum, he was not able to do well in school. He failed 7th, 8th and 9th grade, was told repeatedly that he was unintelligent and dropped out of high school to work and support his family. In his heart, there was a yearning to prove himself, so at 33 he came back to school. Looking at him I did not see stupidity, I saw determination, I saw strength and wisdom not seen in many others.
My wish through this project is to allow children to foster their learning abilities, to allow every children, rich or poor the opportunity to push the boundaries of their potentials.

Photo of Layne Jackson Hubbard
Team

Wow, what a beautiful story Vi Nguyen ! Thanks for sharing your motivation for working on MindScribe. Our team is fortunate to have you—you bring a wealth of insight, proactive energy, and a new mix of rapid prototyping skills.