New EMVI app combines AI and mobile technology to empower visually impaired individuals

Combining AI with mobile technology can open up new worlds for visually impaired people. It's not just about making tasks easier; it's about creating new ways for them to connect with the world around them. Let's dive into how these technologies are becoming invaluable allies, offering an extra 'sense' and a boost in independence for those who need it most.

The introduction of smartphones, now more than 15 years ago, disrupted our daily lives for the better. Mobile apps have transformed how we engage with the world around us. Today, communication is effortless, banking is conveniently accessible, house hunting can be done on the go, sports updates are at our fingertips, and the list goes on. As smartphones became increasingly common, artificial intelligence (AI) emerged as a promising technology, improving capabilities of mobile apps.

How ChatGPT popularized AI

In the months following the release of GPT-3.5 by OpenAI in March 2022, it became obvious that Large Language Models (LLMs) such as the GPT family of artificial intelligence would cause an even greater disruption in our daily lives than smartphones. To help unfold the full potential of these types of AI we should ask ourselves the question how we can utilize them to have a positive impact on our daily lives. Default use cases such as chatbots, search engines, creative writing, and education are already widely used, and have already impacted our way of thinking and working.

However, the possibilities are endless.

AI and mobile solutions for visual impairment

For visually impaired individuals, navigating the world doesn't come with the visual cues most of us take for granted. Simple daily tasks and experiences, like moving through a room without obstacles or experiencing art, come with inherent challenges.

While image-to-text recognition tools have opened some doors by identifying objects in the environment, they fall short of conveying the full picture. These tools might alert a user to a person's presence, but they do miss the nuances, such as emotions, body language, and the actions a person is taking. It's this gap between basic identification and a profound understanding that can be tackled with a combination of artificial intelligence and mobile applications.

The advanced capabilities of LLMs like GPT-4 surpass the basic functionalities of traditional image-to-text models. Imagine not only identifying objects in your surroundings but engaging in an in depth conversation with the AI that explains the nuances of your environment.

Seeing and understanding through AI

GPT-4 can interpret complex questions and scenes, which allows to have a conversation that feels almost human. Questions like ‘Where is my glass on the table?’ or ‘Is there a person in the room?’ are only the beginning. GPT-4 provides much more sophisticated insights especially when questions are asked that require analytical thinking: 'Is this room safe from tripping hazards?' or 'What emotions does this painting convey?'.

The interaction with the AI-model is not just one off, but they are truly conversational and handle the context as you would be talking to a person. The user can be guided through a room in a safe manner, analyze the emotions of a person in the picture, identify the artist of a painting or even explain the story behind a meme received via WhatsApp. These conversations not only act as an extra ‘sense’ in the way that the user is ‘seeing’ the world through the AI, but allows the user to connect and understand their surroundings in a way that was previously impossible.

EMVI & ACA Group: Pioneering visual assistance in mobile apps

When LLMs are integrated into mobile apps, they become assistants which are always at hand instead of mere tools. Native mobile device features like speech input/output and voice commands, further enhance user experience. This combination empowers users to independently interact with the visual world, improving their safety and understanding of their surroundings. The LLM is no longer a tool, it becomes an extension of the user's senses.

EMVI's above-mentioned vision for technology's potential impact on visually impaired individuals inspired ACA Group to collaborate in the development of a valuable mobile app. EMVI aimed for more than just a visual assistant; they envisioned a companion that fosters a deep connection between users and their surroundings.

The EMVI app: Designed for accessibility

The EMVI app prioritizes the needs of visually impaired individuals in different ways:

Enhancing accessibility: It aims for a smooth user experience which is not focused on how visually capable individuals would handle an application, but on how you can handle the application without visual cues.
Intuitive interaction: Native mobile features enable interaction through voice commands and a single button, with communication facilitated through auditory feedback like text-to-speech and media playback.
Visual Assistance on demand: Users can instruct the app to take a picture, initiating a context-aware conversation with their visual assistant. The assistant provides detailed descriptions of surroundings, identifies potential dangers, locates objects, and even interprets emotional ambiance.
Focus on inclusivity: The app is not only tailored to understand the physical world around them. In today's digital age, inclusivity extends to digital communication. The EMVI app integrates with various applications, allowing users to comprehend and engage with visual content, from social media to messaging apps.

Transforming lives through combined technology

Together with EMVI, ACA group combined mobile technology with LLMs to provide an engaging visual assistant to visually impaired individuals. What we have achieved in this collaboration is only the beginning and portrays just a portion of the capabilities and impact that these technologies can have on the lives of individuals. By combining the strengths of the technologies, and tackling their weaknesses we can tailor the user experience of each individual to their needs.

Do you want to transform lives by combining technologies?
Or looking for an experienced app development partner?
Contact our experts