OpenIDEO is an open innovation platform. Join our global community to solve big challenges for social good. Sign up, Login or Learn more

The Talking(Speech technology enabled) Form Services for Blind, Visually Impaired or Agraphic Persons of Nepal

We will allow visually disabled people to access Government Information and Services via audio contents and fill forms using their voices.

Photo of Sagun Dhakhwa
69 14

Written by

What problem does your idea solve?

Visually disabled people are left to live in stigma in Nepal where they are dependent to access government information and services. There is no government support so far to even file a request form for services like citizenship certificate. Privacy is a serious issue, as they have to rely on a mediator to answer even private form-questions. Our idea will solve this problem by giving access to the information and form contents in audio and allowing persons to orally fill and file the forms.

Explain your idea

Government offices of Nepal are yet to support blind or visually impaired citizens. Most of information and request forms for the government services are paper based and have no braille support. There is a growing trend to digitize these forms but still lack accessibility. Filling these forms is extremely time consuming if one relies on accessibility support in the computers; however most of the blinds have no computer skills. In both cases, a disabled person has to rely on someone to get information and fill the forms. Our idea is to solve this problem by building a enclosed kiosk which provides information in a richly formatted (easy to navigate) audio contents and allows users to use voice to fill a form. The kiosk will contain a headphone, a microphone, a special remote controller with a biometric sensor, a printer, an Automatic Speech Recognition (ASR) software and a touch screen (optional for normal users). We will also make a companion mobile-app which has the same functionality and can be connected to a printer. When audio inputs are given to the forms, the ASR software automatically converts the voice into text. Screen readers in the form will help to review the inputs. The user can then confirm the submission by providing a finger print. The system will respond with a confirmation code in audio and braille printouts on a card with a barcarole. The card or the confirmation code can be used to check the status later on the same kiosk or the mobile-app.

Who benefits?

In 2011 Nepal Census, 513,321 individuals reported having “some kind of disability,” of which almost 100,000 were blind. There has been no fresher census in the country but with an annual 1.24% growth in population, we can expect at least 107674 people to have partial or complete blindness by 2017. If we implement the Audio Form Services in all the federal, state and local levels, all of them will directly benefit of which the youth will mostly benefit as they will seek services more.

How is your idea unique?

We have no information on similar technologies being used elsewhere and the nearest competitor would be a company that produces audio books and contents. There are also applications like SIRI in iPhones however they are not available in Nepali. In 2007, I had been working in Access to Computers for Non-Literate People project (ACNLP) which was related to audio enabled User Interfaces and audio content visualization of speech into sentences and paragraphs in Nepali. The proposed speech technology enabled forms will, however, be quite different than ACNLP as it will be dealing with a formal government form and audio inputs. Current speech API support in most of the modern browsers will also make the development faster and implementation straight forward. Speech API or commercial Speech technology is not yet available in the Nepali; so we believe, that after the success of this project, a new paradigm to digital contents and digital forms will globally open.

Tell us more about you

We are an IT research and development company comprising of individuals with a lot of experience in software and mobile app development, Interaction Design and Speech processing. We work a lot with government, universities and tourism sector. Apart from commercial projects, we also do a lot of community-based projects. We had recently been awarded a grant by the World Bank to work on urban poverty alleviation of the Patan City using IT and local Tourism at UNESCO World Heritage sites of Patan.

What are some of your unanswered questions about the idea?

We feel that we will need to test out Automatic Speech Recognition(ASR) in Nepali Language using Sphinx4 library which is the most promising library for ASR. Though we had already done some experiments in Sphinx3 with Nepali a few years ago, there are a lot of modifications in Sphinx4 and we will need to vigorously train and test the ASR system before we can use the speech technology enabled forms to be used by the public. Also, though we work a lot with the Government of Nepal, we will still need to convince a lot of officials before we can deploy the system to the government offices. The system may also be applied to the private sector, however, we are unsure on who will pay the costs in that case.

Where will your idea be implemented?

  • Nepal

Experience in Implementation Country(ies)

  • Yes, for more than one year.

Expertise in Sector

  • I've worked in a sector related to my idea for more than a year.

Organizational Status

  • We are a registered for-profit company (including social enterprises).

Idea Maturity

  • Prototyping: I have done some small tests with prospective users to continue developing my idea.

How has your idea changed based on feedback?

After the feedback from OpenIDEO, we decided to follow the WCAG2.0 guidelines as far as possible. However, we want to focus on usability of the system for users even for those who are new to the digital machines. Hence, we will not use TTS as they don't sound natural and format audio contents in natural speech. After the user feedbacks, we found that one of the biggest concerns the users have was related to privacy when users answer to the machine. They said that they were embarrassed to share their personal details in front of a third person even if he/she is a friend. We have thus decided to enclose the kiosk in a sound proof booth so that the user can feel free to answer the personal questions. We realized that making the users awareness of technology is more important than making a technology. Hence we have decided to promote and advocate about the solution in any ways possible. Inside the offices, we have decided to use Tactile pavements to give directions to the kiosks.

Who will implement this idea?

Sagun Dhakhwa will be responsible for managing team, designing and creating the solution. Tina Rajbhandari (Resource Manager) will be responsible for outreach, management of resources and logistics. There will have 5 supporting staffs : 1 Speech Technology Expert, 2 Software, 1 hardware, 1 UX/ Sound Engineer. The team will be located in the Kathmandu Valley and will travel when needed. The blind association of Nepal will be responsible for advocacy and lobbying with the government.

Using a human-centered design approach, you may uncover insights that lead to small or foundational changes to your organization’s existing strategy or processes in order to unlock the potential of your idea. How would your organization go about making such changes?

In our organization, we focus mainly on the usability of the products and found that human-centered design improves usability. We have seen the potential of human-centered design approach after using it a bit for this challenge. Using the prototypes and User Experience Map, we could unveil many features and design decisions which we wouldn't have found else. Our organization has already decided to adopt HCD in future projects. To implement it, we are going to organize formal trainings for the design team on the new design process in coming months. We have a vibrant design team where people come from multimedia, fine arts, IT and graphic design background. The training will not just be about the process but the benefits and potentials it has. It will be also bring them on board to HCD

What is it that most attracted you to Amplify instead of a more traditional funding model?

We were attracted to Amplify because it gives us is a chance to make a difference while learning new methods, tools and ideas. It also gives us an opportunity to learn from other's solutions and the chance to collaborate with them. It helped us to test, get feedbacks, improve and grow and above all test our potentials. The experience of the whole challenge was engaging, rewarding and fun, that is why we are continuing to participate in the challenge.

What challenges do your end-users face? (1) What is the biggest challenge that your end-users face on a day-to-day, individual level? (2) What is the biggest systems-level challenge that affects your end-users?

On a day-to-day level, visually disabled users have to get help from others for any kind of legal processes. Whether it is a citizenship application, bank transaction or property transfer, legally, they have to be accompanied by a normal person in order to do a transaction. There is no privacy and sense of independence for them. The biggest and critical system-level challenge will be the Nepali Automatic Speech Recognition (ASR). There is no Nepali ASR available and we are trying to customize Sphinx CMU, an open source ASR in Java, for Nepali. A Hindi language model is already available for Sphinx and since Nepali is very close to Hindi, we are confident that we will be able to create a language model for Nepali. ASR will be the most critical part that will affect the end-users.

Tell us about your vision for this project: (1) share one sentence about the impact you would like to see from this project in five years and (2) what is the biggest question you need to answer to get there?

IMPACT: By 2022, we aim to implement Talking Form Application for Citizenship Certificate Application in all the 75 Citizenship Offices in Nepal, which will benefit more than 120000 blind and visually impaired persons in the country. QUESTION: How do we convince and lobby with the government, political parties, civil societies and human right organizations to implement Talking Form for Citizenship Certificate Application and How do we legalize audio inputs as equivalent to inputs in writing?

How long have you and your colleagues been working on this idea together?

  • Less than 6 months

How many of your team’s paid, full-time staff are currently based in the location where the beneficiaries of your proposed idea live?

  • Under 5 paid, full-time staff

Is your organization registered in the country you intend to implement your idea in?

  • We are registered in all countries where we plan to implement.

My organization's operational budget for 2016 was:

  • Between $50,000 and $100,000 USD

If your team/idea/organization has a website, please share the URL below.

http://outlines-rnd.com https://www.facebook.com/OutlinesRND/ https://www.facebook.com/patanheritagewalk/
View more

Team (3)

Sagun's profile
IMemyself's profile
IMemyself Tina

Role added on team:

"She is a resource manager in the team. She will be responsible for collaborating with the Association of Blind Nepal and she will managing test sessions for feedbacks. She will be also managing resources and logistics."

alisha's profile
alisha shakya

Role added on team:

"She is responsible for doing all the graphic design and UI design works. She is also helping the team with audio menu."

Attachments (1)

sketch.pdf

This is a short story board of use case of The Talking Forms.

69 comments

Join the conversation:

Comment
Photo of IMemyself Tina
Team

I feel proud to be a part of such a dedicated and passionate team. It was a great experience of learning new things already while meeting visually impaired and blind people and knowing about their day to day problem during our research for this project. This has already been a unforgettable experience for me and also inspiring to contribute towards community. Thank you Nepal Asscociation of the Blind (NAB) for your cooperation and constructive feedback . I would like to thank everyone from NAB who participated on our interview and feedback session. Thank you Bikesh Shrestha from Balkhu, Kathmandu for your opinion as a user point of view. Your energy, friendliness and involvement inspired our team more. Great work team and all the best! It has always been a pleasure to work with such a positive attitude and yes we are here to make things happen.

Photo of Sagun Dhakhwa
Team

Thanks Tina for corresponding with the Nepal Association of Blind and other potential Blind users. The prototype testing sessions were just the great method to come up with better ideas to solve the problem in this iteration. The people at NAB were just fantastic. We need to do one more round of testing of the new version of prototype with them.

Photo of alisha shakya
Team

The proposal is finally coming to a shape. Really proud to be part of the team!

Photo of Sagun Dhakhwa
Team

thanks Alisha!

Photo of OpenIDEO
Team

Hi Sagun and Team! We’re excited to share with you feedback and questions from the Amplify team and an external set of experts. We encourage you to think about this feedback as you continue to improve and refine your idea. You are welcome to respond in the comments section and/or to incorporate feedback into the text of your idea. Your idea and all associated comments will all be reviewed during the final review process.

General feedback from experts:
• On expert shared, “The idea is bold, in most cases blind people use braille to learn, I am excited about the use of audio enabled forms. This will bring in confidence and more privacy for the target group.”
• Another Expert shared, “The idea presents a new approach using existing technologies. The development of this kind of technology has potential to be implemented in many other cases and government services could be more easily accessed by people with disabilities! Tests with users is essential in this project.”

What experts shared for is this idea desirable, feasible and viable?
• Some areas to consider and potential challenges that experts brought up were: 1) Acceptance of the idea by the government and other authorities (this came up multiple times especially considering Nepal’s signature UNCRPD); 2) How will the users needs and accessibility be considered, for example, for things like offices in which these machines would sit; or how would the proposed beneficiaries be able to reach these offices; what is the level of access to smartphones amongst the proposed beneficiaries? Is a companion app therefore desirable/ viable?
• One expert felt this would be very desirable especially for users in rural communities.

How does this idea consider user needs?
• How can you share what you have learned from users, especially specific needs from users and how you have used this to design your solution? For example, how have you taken rural and urban settings into consideration for how this project will be arranged? Have you considered the needs of those who are new users of technology?

Some questions to think about:
• Who else you plan to collaborate with?
• What plan do you have to create awareness to the community at larger?
• How are we planning to sustain the intervention?
• What’s your familiarity with and desire to implement ATM accessibility and accessibility in general (WCAG2.0)?

Thanks for sharing the important work you are doing!

In case you missed it, check out this Storytelling Toolkit for inspiration for crafting strong and compelling stories: http://ideo.to/DXld5g Storytelling is an incredibly useful tool to articulate an idea and make it come to life for those reading it. Don’t forget - June 4 at 11:59PM PST is your last day to make changes to your idea on the OpenIDEO platform.

Have questions? Email us at amplify@ideo.org

Looking forward to reading more!

Photo of Sagun Dhakhwa
Team

Dear OpenIDEO,
Thank you very much for your feedbacks. They really helped us with revisiting our concept. Here are some replies to your concerns:

What experts shared for is this idea desirable, feasible and viable?
== We agree, there is a need to lobby with government in a national level. This can be done with the help of Nepal blind association and other organizations working for the rights of disabled people. We already have contacts with their head office and local offices and they are very positive to collaborate with our team.

Concerns related to whether kiosk and companion app will be needed separately:
==The kiosks will still be useful in any part of the country, as services like Citizenship certificates and passports are still given in person in Nepal. Government offices are mostly located accessible from most parts of the district and recent political changes have ensured even more decentralization in the governance. Hence, we can expect the talking-form service will locally be even more useful. Yearly awareness and application collection with mobile Talking-form services can be run across various villages around the country so that people who are not yet registered as blind or who had no one to help them to the government offices can also benefit.
There are around 50% of the population living in the cities and there is a good penetration of Internet, mobile phones and smart phones in the cities. In many cased blind and visually impaired are forced to migrate to cities because lack of facilities for them in the country side. Companion App will therefore be very useful for users in the urban areas.

How does this idea consider user needs?
==We learned that most of the blind users, we met, use assistive technologies in their computers or smart phones in the cities. However we haven't met any one from rural areas. With our earlier experience in SAMBAD project while working with non-literate users, we are considering audio contents and forms because it is a natural way to interact for anyone. The contents should be presented one at a time and should not confuse users. It should be possible for users to navigate through the system using audio menus and a remote controller. For new users of technologies, we will provide audio guides which will help the them to get a training or do a mock up form fill up before filling in the real form.


What plan do you have to create awareness to the community at larger?
== We are planning to organize Talking Form Boot camps in all the 75 districts where we will invite blind and visually impaired users to test our product and educate them about the benefits like independence, privacy and legal security. We will also run mobile Taking-form awareness program and form filling programs yearly in different villages around the country.
How are we planning to sustain the intervention? == We will add a fee for each form entry. Right now in Nepal there is a fee for a form which is given in charity to the Red Cross Society. Similar fee can be charged to the blind users/ or government for each application.

What’s your familiarity with and desire to implement ATM accessibility and accessibility in general (WCAG2.0)?
== We have gone through the WCAG2.0 guidelines and we will follow the guidelines as far as possible. There are however problems because of unavailability of technologies or at times issues of usability of the system if available technology is used at the current state. For instance, there is no screen reader available in Nepali in Windows. If we use a Nepali TTS for Linux, the voices are very mechanical and unnatural. We have worked with technologies that can be used for assistive technologies earlier but this will still be the very first project we will be implementing only for the blind and visually impaired.

In general, we are very excited and thankful to OpenIDEO for providing us with this opportunity. Your feedbacks have changed the way, we initially thought about the solution. We still have a concern on use of Nepali Automatic Speech Recognition. We would be grateful if we will receive some assistance from an expert in ASR from OpenIDEO, if we get selected as the finalist.

Regards
@user

Photo of Rubina Malla
Team

Good to know about this.... it will be milestone on behalf of serving blind people....Best of luck..

Photo of Sagun Dhakhwa
Team

Dear Rubina,

Do you think we can test it in Lalitpur with real blind users.

Photo of Rubina Malla
Team

Yes....u can.

Photo of Pankaj Singh
Team

Great idea. All the best. Hope it get success

Photo of krishna Gopal Shrestha
Team

Very nice. There is no doubt that this will help the targeted people. If we successfully complete the project we can show the world that we are also in the race of technology. Japan is also trying like this project for the paraolympic games which is going to be held in Japan. But they are limited into taking photographs only. So ours is brilliant. Wishing for successful completion.

Photo of Shishant Shrestha
Team

Totally spellbinding and astonishing concept!!
Got the concept splendour which definitely gets people marvelloyd with it's resplendent thoughts deliberating towards the comfort of marginalized (esp. blind) groups. I hope for the best and hope to see the concept implemented as sonner as it can be.
#Truely_Conceptual!!☺

Photo of Aashish
Team

Very innovative and initaitive move towards the disabled people. It should be promoted and executed without any delay.
Good luck

Photo of Shangita Awale
Team

Interesting and useful initiative for the specially abled people and also for illiterates. This concept is new for Nepal and if we could really develop this system then im sure it will be a great help for the people in need. Hope this idea will be given priority and Nepalese in need will get benefited . All the best to you and your team!!!

Photo of Sagar Gopali
Team

The idea is incredible,innovating and motivating.if this is possible then all disable people will get a new hope to live their life in a better way.i m happy to know about this technology.Best of luck for the team.

Photo of Sagun Dhakhwa
Team

Thanks Sagar. We will need help from people like you for making this project successful in Nepal.

Photo of mani sintan
Team

Sounds very interesting and very awesome concept.. Hope for the best for you all

Photo of Sagun Dhakhwa
Team

Thanks Mani. I hope we will make it.

Photo of Sagun Dhakhwa
Team

Dear Mani,
Would you like to test it in Makawanpur if we come up with a prototype?
Do you have contacts with Blind association of Hetauda?

Photo of Mukesh Lama
Team

Hello Sagun,
Super and such a great and good idea and I am 100% sure u will success that helps lot of disabled people facing problem in every sector of Nepal everyday.If you guys sucess to bring that kind of technology in Nepal will be a good news for disabled people to make them easy in every step.At last I wanna say Good luck and really appreciate.

Photo of Sagun Dhakhwa
Team

Dear Mukesh,

Thanks for your encouragements and interest in our project. I am looking forward to it.

Photo of Lawa Raj Neupane
Team

Great idea and brilliant innovation sir, helpful to all blind people. It will be brilliant platform to all those who are not much literate. Since, literacy rate of Nepal is not good, implementation of this project make it easier to deliver services. Best of luck.

Photo of Sagun Dhakhwa
Team

Thanks. We are very eager to start the project.

Photo of Fadila Lagadien
Team

Sagun, this is a great idea not only for blind people but those who are illiterate, have dyslexia, are physically disabled so your beneficiaries' list will be quite large. Perhaps it could be used in other countries too (we - our countries - are BRICS partners) such as South Africa once its completed as it is so universal.

Photo of Sagun Dhakhwa
Team

Dear Fadila Lagadien,
Thanks for your comments. True, it can also be used for illiterate, dyslexics and physically disabled. Actually, its for everyone who can speak and hear. We can do it for South Africa too. Which languages do you use in South Africa? It would be interesting to see how the system like this will be taken in your region and mine.

Photo of Fadila Lagadien
Team

Sagun, we have 11 official languages but the 'business' language in English.

Photo of Sagun Dhakhwa
Team

Dear Fadila,
If English is the main business language, it will be even better. We can also do it for other official languages.

Photo of Fadila Lagadien
Team

Good - I will assist with the other languages once we have the English done.

Photo of Sagun Dhakhwa
Team

Thanks Fadila. We will be in touch!

Photo of Fadila Lagadien
Team

Sure.....

Photo of SurenDra Nirola
Team

Convenient and practical digital transformation. Initiative such a boon. Best of luck team of "talking form"

Photo of Sagun Dhakhwa
Team

Thanks for the good wishes, SurenDra Nirola!

Photo of Amit Mishra
Team

Innovative approach sir ,this is going to benefit lots of people, having wide range of application areas too .

Photo of Sagun Dhakhwa
Team

Yes. Thanks for taking your time to encourage us Amit Mishra.

Photo of kamal k chhetri
Team

Your ideas and concepts would be very fruitful for the visually impaired people and i also view it as a step towards making them independent boosting their own confidence.

Photo of Sagun Dhakhwa
Team

Thanks Kamal K Chettri for your encouragements.

Photo of Prakesh Shrestha
Team

Brilliant idea, just what the doctor recommended for Nepal... good work Sagun

Photo of Sagun Dhakhwa
Team

Thankyou very much Prakesh!

Photo of Arleena Shrestha
Team

Your idea certainly grabs attention as its worth promising for a better life of marginalized people in Nepal. Apart from visually impaired people, it bestows ease to handicapped and illiterate people. And, why not the normal people get access to it as a boon of future technology. Be alert, your beneficiary list is going to lengthen. Relying on others will soon disappear from Nepal, once it gets implemented.
Congratulations and Good Luck :)

Photo of Sagun Dhakhwa
Team

Dear Alreena,

This is the best encouragement and positive note someone has posted for my idea. I really am looking forward to its success and I am super excited to try out my idea.

Photo of Biplab Guragain
Team

Wow finally something is being done to help the visually impaired ppl of Nepal..is this going to be free to use for them..I hope so..best of luck

Photo of Sagun Dhakhwa
Team

Dear Biplab,

We are trying to make it free to use for everyone. Of course there is a cost for maintainance and management for which, we are trying to find resources. We would be grateful if we find some help once the product has been ready.

Photo of Dipak Shrestha
Team

Wow great job... Finally someone do really appreciate works who needs this alot... Well done

Photo of Sagun Dhakhwa
Team

Thanks Dipak Shrestha.

Photo of Subrana Sapkota
Team

This concept is perfect for the disable and normal people as well . Hope this get implemented soon ..My best wishes is with you and your team .

Photo of Sagun Dhakhwa
Team

Dear Subarna,
Thankyou very much for your wishes.

Photo of Prerna Pradhan
Team

Great initiative..finally someone is doing sonething to help the less fortunate..this will be a great achievement..good luck

Photo of IMemyself Tina
Team

wow! I shared this idea with my nephew who is blind since his birth, he thinks it is an excellent idea and many blind people can truly experience life in a liberated state of being at least to some extent with this...thank you and good luck!

Photo of Aleesa Shakya
Team

We really need something like this technology in Nepal. Good Luck!
-Aleesha from Shreejana College of Fine Arts

Photo of Ujwal B
Team

Its a brilliant idea as it will make it easier for visually impaired people to access necessary services without having to queue up or have to rely on others even for the most basic services.

I also think its best used at local government levels where people have a bit more interaction on a regular basis. Once it reaches a level of maturity, then the system could be extended to ministries and departments where the system could be of immense use.

With electronic forms, it would also be possible to audit the quality of information and measure the KPIs of the services hence the officials will also have the incentive to get work done quickly.

Photo of Sagun Dhakhwa
Team

Thank you Ujwal B, for your support and insights.
Yes it has a lot of potential if we can produce a proof of concept.

I am not sure what do you mean by your last paragraph about quality of information and measure of KPIs. Can you explain?

Photo of Ujwal B
Team

Completing the forms using accessible kisok is great but it will also provide an effective way to measure and benchmark how quickly the form was processed. This will bring the added benefit of transparency and quick turnaround, which are two areas in which the government bodies are currently lagging behind.
Officials would have a key performance indexes ( KPI's) to meet, which should be able to measure their performances and the managers would solid basis on which they can improve reportable performances.
For example, lets say an officer assigned to process the form lodged via a kiosk for the visually impaired picks up the task and completes completes the task within 20 minutes, but the KPI for the job is 15 minutes, then he/she is is not meeting the target. But if the official is completing the job, within the prescribed timeframe, then the management should be happy about it and then go about improving the process and peformance in another area.

The quality of information is another thing a kiosk would improve compared to a paper form. A lot of data entry errors like typos, invalid information can be picked up as soon as they are entered or the agency can contact the customer if any information is not correct. The benefit of electronic forms - whether they are for general customers or for visually impaired is that it is possible to apply validations and checks before and even after the forms have been lodged but thats impossible to do with paper forms.

Photo of Dipen Raj
Team

Great idea, hope you guys get success on this. It will definitely help thousand of people on daily basis.

Photo of Sagun Dhakhwa
Team

Thanks for the wishes and encouragements.

Photo of Tisa
Team

Great! best of luck! I have a relative who is visually impaired, i think if something like this technology comes to our country it would be great to help thousands of people like him.

Photo of Sagun Dhakhwa
Team

Dear Tisa,
Thanks for your wishes. If you are in Nepal, we can test our forms with your visually impaired relative when its ready. I hope we can bring something useful for the world with this challenge.

Photo of Tina
Team

Very very impressive... this will definitely help not only visually impaired people but also to illiterate. Best of luck hope we will get to see such technology soon in Nepal. Well, I would like to know more about Automatic Speech Recognition(ASR) in Nepali Language if you can answer. Thanks.

Photo of Sagun Dhakhwa
Team

ASR in Nepali will be a challenge. Though there are mature open source systems like SPHINX. We will still need to do a lot of testing and training. There has not been serious effort on ASR for commercial products so far. There are few research data on it though. I am currently working on this in my lab.

Photo of Surendra
Team

Congratulation! Very very good idea! I would like to know more about it-
How do blind and visually impaired people verify the data in the forms they have submitted?
Can a normal person use this software? Will it contain an user interface?
Thank you.

Photo of Sagun Dhakhwa
Team

Dear Surendra,

Thanks for your wishes and encouragement.
Right after the user has spoken the words or word for a field in the form, the system will recognize it and read out the recorded data. This is how users will verify the entered data.
Secondly, at the end of a group of fields or a complete form , depending upon the length of form, the system will read out the entered data in the fields. This will be the final chance to verify the filled form. If user is not satisfied, he/she will be given a chance to reenter the data. If user is too much frustrated with the system ASR, we will also have a feature where user can spell out the words.

Yes thsi system will contain a GUI and a normal person can also use it. In fact this could be a future technology where user no longer need to type or click.

Photo of Susan Shrestha
Team

its a very good idea ... best of luck ... hope it gets to our country soon and gets implemented

Photo of Sagun Dhakhwa
Team

Thanks for your wishes.

Photo of Vijay
Team

The idea is good and promising. I have few questions
1. Data verification: How does the system verify the filled data from not only the visually impaired people but also by other people also?
2. User interface: How does the visually impaired people know where are the buttons in the screen ? Does the system has interface of touchscreen Braille Writer so that visually impaired people start the system themselves using their own prior knowledge?

Photo of Sagun Dhakhwa
Team

Dear Vijay,

Thanks for your questions:
For Data verification, we will use several approaches like detecting word boundaries in utterances of recorded speech. The boundaries will be used to detect if user entered only the expected word for a field. People tend to utter other sounds when speaking. This will be one big challenge. Second would be converting the sound into words using ASR. Once we have a word, we can check it against a dictionary. But this is not going to be simple we might need to use other smart ways to make this efficient.

For UI, we will assume GUI doesn't exist for visually impaired or blind. We will give them an audio UI which can be navigated using a simple input device with few buttons (with braile like haptic ques). The UI will be designed in such a way that only one question or content will appear at a time. This will of course consume longer time than GUIs but we will see it in trials how fast the audio inputs and interactions will be.

I am thinking of giving Nepali only facility for now because though Nepal has more than 120 different languages, Nepali is a lingua franca.

Photo of Gong Xuan
Team

I will be very happy for my visually impaired friends in Nepal if this get implemented in Nepal sooner...Best of luck. This is simply brilliant.

Photo of Sagun Dhakhwa
Team

Dear Gong Xuan,
Thanks for your encouragement and support. Yes it would be really nice if we can do something for visually impaired. Not just for them in Nepal but elsewhere as well. Right now fingers crossed.

Photo of Prawesh
Team

Seems like a great idea..will this only be implemented in Nepal or due you have intention to implement in other countries too?

Photo of Sagun Dhakhwa
Team

Dear Prawesh,

Thanks for your support and encouragement. Once the proof of concept is ready and it can be proved that it is useful, I am looking forward to implement in other parts of the world too.

Photo of Sagun Dhakhwa
Team

Talking Form is a form which
--allows users to naturally interact with digital contents verbally
--interacts with the user in natural voice
-- allows users to fill in the forms using their speech/voice
-- will be helpful for blind, visually impaired, person with mobility problem, non-literate people
-- will also be useful for normal people and can be a new and more natural way to interact with digital system using their voices.