An Overview

‍

EMMA: Emotionally-Minded Mecha Arm | is a project created by myself and a partner for a Gizmo project at the Dyson School of Design Engineering. The goal of the project was to create a device that can interact with a user in a unique and engaging way. EMMA is a mechanical tentacle, designed to be a pet-like device that can respond to a user's hand gestures and facial expressions to give different emotional and visual responses such as movement, following and lights. Users can discover features of Emma by interacting with it like a pet. Overall, EMMA is a creative and innovative project that combines advanced technology with a unique user experience.

‍

‍

‍

‍

The Challenge

‍

• ‍Inovate a product which serves no purpose but has interaction with the user.

• Build & Improve experience in manufacturing and mechanical engineering with, CADing, mechanism design, Rapid Prototyping, 3D Printing, Laser Cutting, Welding, Soldering and electronic design.

• Integrate all components from our design, with specified tolerances

‍

‍

The Solution

‍

• Establish the most seamless interaction a user can have with the product which is mimicking the human interaction.

• Design the whole of Emma in CAD accounting for any and all possible design considerations

• Fabricate using welding and additive manufacturing, creating a shell and chassis for the product including 4 retractable legs.

• ‍Integrate the imperfect fabrications with the perfect 3D prints allowing for tolerances. Soldering and adding lights.

• Develop user interface on the webapp, all communications in the 8 step process. LED's | Motors | Display | WebApp | Camera

• Polish the design adding top and bottom panels with magnets, ensuring LEDs and motors are in sync, Establishing the best wireless communication between the onboard camera and the webapp, the webapp to EMMA.

‍

‍

‍

‍

Complex Outcome

The control of the motors using the AccelStepper and MultiStepper libraries. These libraries provide a high degree of control over the motors, but they can be challenging to use and require careful coding to ensure that the movements of the arm are smooth and accurate.

‍

Why we chose it?

• Asynchronous functions allow the program to continue executing while waiting for serial input to be received. This means that the program does not need to block or pause, which can prevent the motors from stopping.
• Asynchronous functions provide a way to handle serial input as it is received, rather than waiting for all of the input to be received before processing it. This allows the program to respond to serial input more quickly, which can prevent the motors from stopping.
• Asynchronous functions provide a way to handle multiple pieces of serial input concurrently, rather than processing them one at a time. This can improve the performance and responsiveness of the program, and can prevent the motors from stopping.

Why its Hard?

• Asynchronous functions are executed independently of the main program flow, which can make it difficult to predict and control their behavior. This can lead to unexpected results and can make debugging difficult.
• Asynchronous functions are often used to perform tasks that take a long time to complete, such as network requests or complex computations. This can lead to delays and slowdowns in the overall performance of the program.
• Asynchronous functions can cause race conditions, where multiple asynchronous operations are executed simultaneously and compete for shared resources, such as memory or CPU time. This can lead to unpredictable behavior and can be difficult to diagnose and fix.
• Asynchronous functions can be difficult to test and debug, as they can be hard to reproduce and may not always behave the same way in different environments.

‍

Hand and face tracking algorithms

The hand and face tracking using handtrack.js and face detection algorithms. These algorithms are complex and require significant computational power to run in real-time. Additionally, the algorithms may need to be fine-tuned and adjusted to work well with the specific hardware and setup of the Emma Tentacle.

Why we chose it?

• AI and machine learning algorithms are able to analyze and interpret visual data quickly and accurately, which is essential for tracking and identifying the user's hand gestures and facial expressions.
• AI and machine learning algorithms are able to learn and adapt to the user's behavior over time, which can improve the accuracy and reliability of the tracking and identification process.
• AI and machine learning algorithms are able to handle complex, real-world scenarios, such as variations in lighting and background, which can make the tracking and identification process more robust and reliable.
• AI and machine learning algorithms are able to process large amounts of data in real-time, which is essential for providing a smooth and responsive user experience.

Why its Hard?

• AI and machine learning algorithms are complex and require a deep understanding of these technologies to implement and optimize. This can require significant expertise and experience in AI and machine learning.
• AI and machine learning algorithms require large amounts of data to train and optimize, which can be difficult to obtain and manage. This can require significant effort and resources to collect and prepare the data.
• AI and machine learning algorithms require significant computational power to run in real-time, which can be challenging to provide, especially in embedded systems like the Emma Tentacle robot. This can require careful design and optimization of the hardware and software.
• AI and machine learning algorithms are subject to bias and other errors, which can impact the accuracy and reliability of the tracking and identification process. This can require careful analysis and testing to identify and mitigate these issues.

‍

‍

Integration WebApp, ESP, Ardunios

The integration of the various components of the system, including the web app, the ESP, and the Arduino boards. This requires careful planning and coordination to ensure that the different components are able to communicate and interact with each other effectively.

Why we chose it?

• The use of multiple Arduino boards allows for the separation of different tasks and functions, such as controlling the motors and the lights. This can improve the performance and reliability of the system, as different tasks can be handled by different boards, and failure or malfunction of one board will not affect the others.
• The use of an ESP32 allows for the integration of Wi-Fi and networking capabilities, which are essential for the remote control and monitoring of the Emma Tentacle. The ESP32 also has a more powerful processor and more memory than most Arduino boards, which can improve the performance and reliability of the system.
• The use of multiple boards and the ESP32 allows for the distribution of the computational workload, which can improve the performance and reliability of the system. Different boards and the ESP32 can handle different tasks and processes simultaneously, which can reduce the likelihood of bottlenecks and other performance issues.
• The use of multiple boards and the ESP32 allows for the use of specialized libraries and other tools that can improve the performance and reliability of the system. For example, the AccelStepper and MultiStepper libraries can be used to control the motors more accurately and smoothly, and the FastLED library can be used to control the lights more effectively.

‍

Server WebApp

‍

Why we chose it?

• A web app allows for remote control and monitoring of the Emma Tentacle from any device with a web browser, such as a laptop, tablet, or smartphone. This allows users to control and monitor the robot from anywhere, without needing to be physically near the robot.
• A web app allows for easy and intuitive control of the robot using a graphical user interface (GUI). The GUI can provide visual feedback and other information to the user, which can make it easier to understand and control the robot.
• A web app allows for easy integration of advanced features and capabilities, such as machine learning and AI-based tracking and identification, without needing to modify the hardware or firmware of the robot. This allows users to easily add and customize new features and capabilities as needed.
• A web app allows for easy collaboration and sharing of the robot's capabilities and features with other users. Users can share the web app and its features with others, who can then access and use the robot remotely.

However, creating a web app to do the processing in Emma can be challenging for several reasons.

• Web apps require a web server and a network connection to operate, which can be complex and difficult to set up and maintain. This can require significant expertise and experience in web development and networking.
• Web apps can be vulnerable to security and privacy issues, such as hacking and data breaches. This can require careful design and implementation of security measures to protect the web app and the data it processes.
• Web apps can be difficult to optimize for performance and reliability, especially when running on low-power devices such as the ESP32. This can require careful design and optimization of the web app and its underlying infrastructure to ensure smooth and reliable operation.
• Overall, creating a web app to do the processing on the Emma Tentacle can provide many benefits, but it can also be challenging due to the complexity and requirements of web development and networking, as well as the need for security and performance optimization.

‍

‍

‍