Electric Signals in Machine Learning Using AppInventor and Arduino

Recent developments in Artificial Intelligence (AI) have introduced machine learning and its applications into everyday life. As technology becomes increasingly integrated into the educational system, researchers are focused on developing tools that allow students to interact with machine learning in a way that sparks their curiosity and teaches them essential concepts. Our instructional proposal, titled “Electric Signals in Machine Learning Using App Inventor,” focuses on applying learning, transfer, and classification models of audio spectrograms to teach students in the first year of high Secondary school (A’ Lyceum) fundamental concepts of machine learning. This is accomplished using MIT App Inventor and Arduino’s visual programming environments. Students will use the website “Personal Audio Classifier” to train an audio model and App Inventor to connect computer science and machine learning. In addition, with the aid of the Arduino microcontroller, students will engage in visualising Morse code signals and investigating Physical Computing, allowing them to create digital solutions that connect to the real world.

Introduction

Students are exposed to AI every day. At young ages, many children have access to AI-capable devices, such as online search, personal digital assistants, automatic translations, and computer games. However, studies [1] indicate that children do not comprehend how their intelligent games and other applications utilise AI. Education and awareness are essential to dispelling student misconceptions about digital devices. Evangelista et al. [2] state that there is an increasing need to familiarise students of the 21^st century with fundamental AI knowledge.

According to research [3], it is essential to increase students’ curiosity and teach them new technology concepts because AI has infiltrated our lives. Our proposal for instruction concentrates on training learning, audio recording, and spectral analysis models to instruct students in primary machine-learning concepts. Scientists have created tools using the free MIT App Inventor visual programming environment, which enables the construction of mobile applications [4]. We present the Personal Audio Classifier (PAC), a web application that allows users to train and test custom sound classification models using 1–2 second user-recorded audio files. We suggest developing a programme in App Inventor with the PAC extension. Using the Arduino development platform, we create a device that visually represents the training model’s output.

Artificial Intelligence and Machine Learning

AI is a subfield of computer science that involves programming development in computing environments to complete tasks that would otherwise require human intelligence. Today, AI algorithms handle learning, perception, problem-solving, language comprehension, and logical reasoning with a great deal of success, and science is a promising field of computation [5]. Samuel [6] believed in 1959 that we should not teach computers but rather allow them to learn on their own. He coined the term “machine learning” to characterise his theory, which is now generally accepted as the ability of computers to learn independently. A system with AI must be able to store knowledge and utilise it in order to gain new knowledge through experience. AI and machine learning appear in our everyday lives [5], such as on shopping websites (cookies), in self-driving cars, etc., helping and improving people’s lives through technology.

Machine learning is a subfield of AI based on the development of algorithms for training data models. Machine learning algorithms create models that make decisions or make predictions based on training data. Starting from the data itself, as shown in Fig. 1, and not some pre-existing theoretical model, using computational methods that are impossible to apply without a computer, the trained model is generated [7].

Fig. 1. Machine learning training process.

This method of utilising mathematical data models allows a computer to self-learn. It uses algorithms to identify patterns in the data and then uses those patterns to create a predictive model. In this context, machine learning is defined as the ability of a computing system to generate models or patterns from a dataset [8].

According to research [9], machine learning is used when a complex task or problem cannot be solved through conventional methods or when a large amount of data and variables cannot be processed and calculated by human resources using conventional methods. This has motivated many individuals around the globe to study AI and machine learning [10]. As researchers and engineers develop and implement these technologies to make decisions in everyone’s lives, citizens must comprehend the AI-powered world, be able to debate AI-related policies and participate in shaping the next generation of these technologies [9].

In addition, it is becoming increasingly vital that citizens are aware of the ethical and safety concerns that its use raises. According to the study [11], the demand for professionals with AI knowledge will increase significantly in the decades to come. Moreover, 67% of companies are already utilising machine learning, and 97% plan to use it within the next year. Following how humans learn, machine learning develops three types of learning: supervised learning, unsupervised learning, and reinforcement learning.

Supervised Learning is the process where the neural network learns to map input data to known-expected output data, with the ultimate goal of generalising inputs with unknown output [12]. The correct output value for the examined data is indicated by the system. Supervised learning algorithms use the data to train the model by analysing it and generating a model that can be used to characterise new input data.

Unsupervised Learning is the process where the algorithm constructs a model for a set of unsupervised inputs without knowing the desired outputs. It groups unclear information according to similarities, differences, and patterns that may exist without prior training in the data [12].

Reinforcement, in contrast, learning is the process by which an algorithm acquires an action strategy through direct feedback from its environment. Consequently, it refers to the development of a self-sustaining system that “fails” to progress by learning from its errors and interacting with incoming data [12].

Following training, the certification phase of newly acquired knowledge occurs. Initially, the algorithm self-certified through recall procedures using test data. Through evaluations based on the user’s knowledge of how the problem should be solved, new knowledge is produced (as shown in Fig. 2). The new information is then applied to tackle actual problems [12].

Fig. 2. Phases of machine learning.

Machine Learning in Education

The benefits of teaching machine learning to students at all grade levels include introducing students to AI concepts, helping them develop foundational skills for life and work, adopting mental models, and inspiring the next generation of AI and machine learning researchers.

Marques et al. [13] found that teaching machine learning concepts from low secondary to high secondary school can increase students’ comprehension and maintain their interest in the field. Students simultaneously comprehend AI concepts in their daily lives. Furthermore, using machine learning in the classroom can identify students’ weaknesses and recommend methods to improve their performance [14].

Introducing a simplified form of machine learning into education without the complexity of programming has a major learning outcome in understanding how important it is to develop reliable machine learning models for real-world applications and problems. A second crucial aim is for students to investigate and reflect by exchanging views on the possible effects on life and freedom of individuals from the improper use of AI [15]. Evangelista et al. [2] argue that machine learning should be taught to students at a young age because it is such a popular topic. Thus, they have a clearer understanding of what it means for a computer to self-teach and what tools are necessary for this to occur.

Although AI is a much more complex and challenging topic, Sakulkueakulsuk et al. [16], based on the results of their study, found that students had more fun, were more engaged, and cooperated more when implementing the activity in the lab than in the regular classroom. This shows that students can understand very complex concepts if the learning environment is well designed and organised. The majority of curricula for all levels of education in most countries suggest that machine learning and AI should be implemented in schools [15].

Stem in Education

Interdisciplinarity, in the most general sense, refers to the participation of multiple scientific disciplines in the resolution of a problem. STEM is interdisciplinary in nature and occurs when two or more disciplines of science, technology, engineering, and mathematics collaborate to solve a problem. STEM education accentuates the connections between the sciences [17]–[20]. In STEM education, interdisciplinarity occurs when students make connections between the various disciplines inherent in a problem they are asked to solve. Based on the findings of their study, Sakulkueakulsuk et al. [16] determined that AI and machine learning activities can be utilised as effective instruments for interdisciplinary education in secondary schools. In addition, they highlight in their findings that machine learning and AI assist students in adopting futuristic and inter-disciplinary perspectives, as well as being cognizant of the incorporation of new technological solutions into the social context.

Whether referring to natural phenomena or socioeconomic and political issues, the study of a real problem provides a solid foundation for STEM. STEM epistemology addresses real-world, complex problems requiring interdisciplinary approaches [21]. When students are engaged in authentic STEM contexts and asked to solve problems they encounter on a daily basis, they are more motivated because the knowledge they are taught is relevant to their lives. In addition, solving authentic problems through STEM provides emotional outcomes, such as increased student engagement, perseverance, as well as crucial educational outcomes, such as critical and logical thinking and problem-solving skills.

Gao et al. [17] argue that one of the primary educational goals of STEM is to help students acquire content knowledge about one or more STEM disciplines. According to this strategy, priority is given to one or more specific cognitive STEM areas, while the other disciplines serve as a means of acquisition knowledge for these cognitive areas. Additionally, they note that another common learning goal of STEM education in the cognitive domain is to help students develop skills that transcend a single discipline. In other words, students should be able to use knowledge from various disciplines. The degree of knowledge for each discipline varies depending on the nature of the problem or situation, despite the fact that the significance of each discipline is treated equally (helping to understand the situation). Concerning the affective domain, students should focus their interest on, engagement, attitude, and motivation they acquire from STEM content and practises. Lastly, STEM education is anticipated to direct students to STEM professions and, by extension, to meet labour market demands.

Mit App Inventor, Personal Audio Classifier, and Arduino

App Inventor is an event-driven, block-based programming language. It enables individuals with little or no programming experience to create applications. The MIT App Inventor project is developed with a simple drag-and-drop interface (designer–blocks). The Designer interface allows users to drag and place and customise Android screen elements, such as buttons. The “blocks” interface provides code blocks that can be arranged to create programmes [9]. App Inventor extensions enable users to develop applications with robust machine learning capabilities.

Audio Classification is the process of analysing audio recordings by “listening” to them. First, the audio stream serves as input. The time-lapsed audio more closely resembles a video than an image. To train the model, the audio file stream is partitioned into tiny or relatively small analysis windows. The sound is converted into a spectrogram using a particular variety known as a Mel spectrogram, which has been demonstrated to be effective because it provides a representation that is more similar to human hearing. A spectrogram is a highly accurate and comprehensive representation of acoustic information. It is an image-graph in which the x-axis represents time, the y-axis represents frequency, and the colour of each point represents its amplitude. Consequently, a spectrogram displays amplitude variations for each frequency component of a signal [22].

As shown in Fig. 3, the audio representation depicted in the spectrogram resembles an image. Convolutional Neural Networks (CNNs) are utilised because of their resemblance to biological vision systems, are the best for image classification, and have been proven to work very well in spectrograms too. The term “Neural Network” refers to an interconnected network of simple computing elements (neurons) inspired by the Central Nervous System (CNS) of living organisms, particularly humans. It attempts to imitate the behaviour of human brain neurons using various mathematical models inspired by their biological counterparts [23]. Its function combines the way the human brain thinks with abstract mathematical thought. A trained artificial neural network’s primary purpose is to be able to conduct certain processes independently, such as recognising sounds or images [24].

Fig. 3. Audio files are converted into spectrograms and then fed into a CNN plus linear classifier model, which produces predictions about the class the sound belongs to.

Many online tools for image classification are more accessible to students and machine learning novices, whereas audio classification tools are scarcer. Personal Audio Classifier (PAC) [3] is a public machine learning tool that enables users to construct, analyse, and export custom audio classification models on the web, in browsers, without requiring specific knowledge of the underlying technology and programming. It was designed to provide a simple and accessible instrument to enable/encourage individuals with limited machine learning expertise to comprehend and use audio classification by training their classification models.

It is an extension of MIT App Inventor that enables users to create custom mobile applications using audio classification and export models from PAC. App Inventor transforms audio data into image representations via the computer’s microphone, allowing students to test their audio classification models and implement customised models. PAC is hosted at https://c1.appinventor.mit.edu/ and enables users to train and classify various types of sounds [9], [25].

In order to make electronics circuits education more accessible to students, Massimo Banzi and David Cuatrielles created the Arduino platform in 2005 in Italy [26]. It is comprised of a motherboard with an integrated microcontroller, as well as digital and analogue inputs and outputs. The Arduino platform has evolved into an extremely flexible programming method, based on low-cost, standardised hardware, suitable for educational applications. It uses the Wiring programming language which is based on C++. Today, visual programming languages such as Ardublock have been devised to encourage student participation.

An Arduino board can be connected to standard peripherals, such as temperature sensors, Bluetooth, LEDs, displays, and flame sensors. The large community of members who provide information, examples, and advice, resulting in a substantial online knowledge base, is one of its advantages. It features an open-source programming environment, numerous peripheral driver libraries, and online assistance. Additionally, one of its significant advantages is its simplicity of use, even for novices, while retaining flexibility for advanced users [26]. It functions equally well on Mac, Windows, and Linux operating systems and facilitates not only programming but also the interfacing of the electronic components required by an application by providing the necessary units (ready-made) for hardware connections, either in card form or using wires on a Breadboard [26].

The Teaching Proposal

The incorporation of machine learning into the curriculum, and particularly the interdisciplinary approach through Computational Pedagogy [20], is anticipated to pave the way for a future in which engineering learning will be understood by all. According to Sanusi et al. [27] and Tang et al. [5], students can comprehend fundamental concepts through straightforward activities.

Machine learning can be incorporated into the curriculum of the first year of high school in the laboratory course “Computer Applications” under the section “Programming Environments–Creating Applications.” According to the Greek curriculum [28], our educational intervention is based on social constructionism [29] and contemporary theories of “information processing.” To enhance the inquiry approach, self-activity, and cooperative learning, it is recommended to incorporate active learning strategies and real-world examples into the curriculum. Combining model training, programming, and Physical Computing constitutes the intervention’s innovation.

Within the context of this module, examples of machine learning applications may be utilised, where students will train their models and view the results on their mobile devices. They can also “upload” a free application for sound recognition to the Google Play Store. Through this method, students will be able to create simple applications and realise that AI is present in numerous applications in their daily lives.

Using the App Inventor visual programming environment and the Personal Audio Classifier, the activity titled “Electrical signals in Machine Learning with App Inventor and Arduino” seeks to integrate the cognitive field of computer science with machine learning. Students will engage in the visualisation of special Morse signals and Physical Computing, which connects the real and artificial worlds of computers, using the Arduino microcontroller.

Students have basic knowledge of coding from Primary and lower Secondary School, primarily through Logo-like environments (MicroWorlds Pro, Scratch, and Pencil Code), and have been familiar with the fundamentals of Arduino wiring connections from the skills workshops and the Informatics course [30]. In addition, students in the first and second grades of lower Secondary school learn to recognise and convert audio file formats. Furthermore, in the first year of upper Secondary school, students learn the fundamental programming techniques and commands of App Inventor and create mobile applications. Regarding the teaching methodology, the fundamental ideas of Vygotsky and Papert are adopted, according to which the student builds his knowledge by assuming the active role of a researcher who conducts experiments and exchanges opinions with his group in order to acquire a meaningful understanding of the problem under investigation. As an assistant, the teacher monitors the class and intervenes as necessary. Concerning the cognitive objectives, by the end of the instructional intervention, the students will be able to: familiarise themselves with machine learning applications, create trained audio recognition models with machine learning, and understand when there is a satisfactory level of trust and reliability in the model through experimentation. Also, to be able to select and write the appropriate commands for various components, defining their hierarchy, especially the Personal Audio Classifier extension. Furthermore, to engage in coding for Arduino operation, establish a connection between the mobile device application and Bluetooth, familiarize students with the microcontroller, and wire LEDs and resistors. Students must be able to encode the long delay (dash) and brief delay (dot) and then decode, test the result, and do the debugging. Students explore modular programming and structured programming while writing code in Arduino and event-driven visual and object-oriented programming in App Inventor with the use of multiple selections, lists, and procedure calls. Students are expected to engage in programming complexity and artefact creation (Arduino), as well as develop critical thinking, creativity, communication, collaboration, and fine motor skills during construction. Regarding attitudes and behaviours, through participation in the activity, students should be able to solve problems collaboratively (with the division of labour, discussing opinions and experiences with their group, and improving solutions through mutual feedback) and be oriented towards STEM careers.

Class Organization–Teaching Proposal Phases

The instructional intervention occurs in the school’s computer laboratory and takes place with twenty-one students from the first grade of the upper secondary school. Personal Audio Classifier was used to train models on the following special Morse codes: sos, end, error, transmit, and wait. The mobile device reproduced the sound effect for the corresponding Morse signals while simultaneously visualising it with a LED that turns on and off according to the Morse coding functions for visualisation. The teacher intervenes and guides the students if it is necessary. During the inquiry process, the training of the model, the construction phase of the application in the App Inventor environment, and the construction of the Arduino, students utilised one computer and one Arduino per two to three students and carried out the activities collaboratively.

In the initial phase of conceptualization, students structure their solutions based on their research queries or hypotheses. The video “How Does Morse Code Work?” served as the discussion’s beginning point [31]. Then, fundamental knowledge of Morse code is gathered through brainstorming, research, and discussion.

In the second phase, students were introduced to model training and machine learning through exploration and experimentation using the first worksheet. After the model training with the Personal Audio Classifier, the “.mdl” file (export option) was generated, which was required for the subsequent activity. As this was the first time the students had interacted with the Personal Audio Classifier, the teacher provided assistance where necessary. Using DAW (Digital Audio Workstation), for example, Audacity, students then searched for and converted audio files (Morse tones) to appropriate formats (mp3). The next step was connecting the trained model by importing the “PersonalAudioClassifier.aix” extension to App Inventor. Finally, the application interface was designed in the Designer environment, and the “model.mdl” file was imported.

In the third phase, students build their knowledge through activities. It was emphasised that they must give basis to the result returned by “model.mdl” which was a list of classifications of the model. The list consists of sub-lists, each sub-list contained two elements: the label of the classifier that matched the most instances and a confidence level, such as [[sos, 0.88527], [end, 0.34812]], respectively. Students created the application in App Inventor by writing the necessary coding.

In the fourth phase, students collaborated, communicated, designed, and implemented the artifact using the Arduino platform. This activity was devised after students attended an online Arduino fundamentals seminar (https://seminars.etwinning.gr/). In Physical Computing, students exercised fine motor skills by constructing circuits on a breadboard and developed critical thinking by programming the Arduino platform. Specifically, the data obtained from the App Inventor application was “transferred” via Bluetooth to the Arduino development platform, which converted it into a visual signal (Led) of a special Morse code. The word was simultaneously displayed on the computer screen via a serial port.

In the concluding stage, students presented the final results of the activities. Then, there was a discussion regarding the findings, the data, the conclusions, the aspects that impressed them the most, and potential improvements and suggestions for their work.

After completing the instructional intervention, each student was given a rubric evaluation of the intervention to fill out.

Conclusion

Many researchers, such as Evangelista et al. [2], believe that introducing a simplified form of machine learning in education without the complexity of programming helps students understand the significance of developing reliable machine learning models for real-world problems and applications. They believe that machine learning is a topic that should be taught from a young age because it is so widespread. Our proposed intervention aimed to familiarise students with AI through the use of machine learning applications and audio recognition model training, ensuring an adequate level of confidence and reliability. For model training, we suggested using the Personal Audio Classifier (PAC), a public and free machine learning application that enables the online creation, analysis, and extraction of custom audio classification models. Using the microphone on their computers, students generated sound data, which was converted into spectrograms and then trained with PAC while developing the programme in App Inventor to generate visual and audio Morse signals.

Moreover, the students actively engaged in the complexity of programming and the creation of an artifact (Arduino), developed critical thinking, creativity, communication, and collaboration skills, and improved their fine motor skills throughout the time of construction activities. Analysing the results of the evaluation rubric submitted by the students revealed that the participants had a greater comprehension of what machine learning entails and derived multiple benefits from its incorporation into the educational process.

The results of the rubric shown that the students’ familiarisation with AI concepts helped them understand the functioning of many modern devices and applications, and they believe they will acquire fundamental skills essential for their future lives and careers. Most students supported the idea that applying machine learning sparked their interest, and they are considering pursuing careers in the field of AI as the next generation of researchers. There were no significant differences between genders in how they worked, their experiences, or their beliefs about the possibility of pursuing AI-related careers. All students found the lesson interesting and would like to repeat it in the future.

We advocate for the incorporation of machine learning into the curriculum because we anticipate that it will help students grasp the fundamental concepts of machine learning, pave the way for a future in which machine learning is accessible to all, and pique students’ interest in this area of computer science. To enhance exploratory approaches, self-motivation, and collaborative learning, we suggest incorporating active educational techniques and real-world examples into the curriculum.

App Inventor supports additional extensions, such as the Personal Image Classifier (PIC) for image recognition training. Our team is expected to present future publications using one of the aforementioned web-based machine learning tools, which provide a quick, simple, and readily available method for training a model.