Have you ever had a musical dream? Imagine playing the piano as gracefully as a pianist. Piano, the "King of Musical Instruments", interprets perfect sound effects and romantic feelings with its smooth notes, bringing pure enjoyment to people. But for various reasons, maybe you have never learned piano or failed to own a piano.
Now, with HUSKYLENS, we can make a color piano for ourselves, to realize the musical dream. Let’s use the color keys to play graceful notes.
This project uses the color recognition function of HUSKYLENS to recognize different color keys and play different notes, so that your "playing" will be beautiful and pleasant, and also with absolutely wonderful stage effects.
1. Learn the operating principles and application fields of color recognition.
2. Learn to use the color recognition function of HUSKYLENS.
3. Learn to make a color piano using HUSKYLENS.
In today's society, automated production has become a social development trend. Machine vision, as the eyes of "robots", is particularly important.
As one of the important technical directions, color recognition has undergone multiple generations of technology upgrades. And this project is to use the color recognition function of the HUSKYLENS sensor to distinguish and recognize colors, and play the piano according to different colors.
What is color recognition? First of all, we must understand what color is.
Color is a visual effect on light produced by our eyes, brain, and our life experience. The light we see with the naked eye is caused by a very narrow wavelength range produced by electromagnetic waves. The electromagnetic waves of different wavelengths appear in different colors. Color recognition is based on the color attributes under different brightness to recognize and distinguish.
Color recognition is based on Lab color space, with dimension L representing brightness, a and b representing opposite dimensions of color, which is based on the CIE XYZ color space coordinates of nonlinear compression. We can regard these three parameters of Lab as the XYZ of the three-dimensional coordinate system. Compare the Lab parameters of the recognized and learned colors. When the two colors match within a certain error range, they are judged to be the same color.
In our usual use of color recognition, the hue and saturation in the color attributes of the same module are fixed, but the brightness will change due to the influence of the ambient brightness, so you must try to ensure that when using the HUSKYLENS color recognition function, the environment brightness during learning and recognition is consistent with that during actual work.
1) Color recognition is now widely used in industries such as printing, coatings, and textiles for color monitoring and calibration.
2) Help people with color weakness or visual impairment to recognize and enhance their understanding of color.
If you want a color piano to play smoothly, first you should let the HUSKYLENS sensor learn the color of the keys, and let it know the notes corresponding to each color. The color recognition function in the HUSKYLENS sensor uses the sensor's built-in algorithm to identify the ID of different colors and feed them back to the main control board by learning and recording different colors.
The HUSKYLENS sensor is set by default to learn, recognize and track only one color, but there is not only one key on the piano, so we need to set it to recognize multiple colors.
1. Dial the function button to the right or left until the word "Color Recognition" is displayed at the top of the screen.
2. Long press the function button to enter the parameter setting of the color recognition function.
3. Dial the function button until "Learn Multiple" is displayed, then short press the function button, and dial to the right to turn on the "Learn Multiple" switch, that is, the progress bar turns blue and the square icon on the progress bar moves to the right. Then short press the function button to confirm this parameter.
4. Dial the function button to the left until "Save & Return" shows. And the screen prompts "Do you want to save the parameters?" Select "Yes" in default, now short-press the function button to save the parameters and return automatically.
In this way, the multiple learning function is set up.
Point the icon "+" in the center of the HuskyLens screen to the target color block, and a white frame will appear on the screen, which selects the target color block automatically. Adjust the angle and distance of the HuskyLens to the color block so that the white frame can include the entire target color block as far as possible.
When detecting colors, press the“learning button” to learn the first color, and release it to finish learning. Meanwhile, a message “Click again to continue! Click other buttons to finish” will be displayed. Please short press the "learning button" before the countdown ends if you want to learn other colors. If not, short press the "function button" before the countdown ends, or do not press any button to let the countdown ends. The color ID displayed is consistent with the order of learning, that is, ID will be displayed as “ID1”, “ID2”, “ID3”, and so on, and different colors correspond to different colors of frames.
When encountering the same or similar color blocks, a color frame with an ID will be automatically displayed on the screen, and the size of the blue frame is the same as the size of the color blocks. Multiple colors can be recognized and tracked at the same time and different colors correspond to different colors of frames.
When multiple blocks of the same color appear, the separated color blocks cannot be recognized at the same time, and only one block can be recognized at one time.
Note: Color recognition is greatly affected by ambient light. Sometimes HuskyLens may misidentify similar colors. Please try to keep the ambient light unchanged. In the process of using color paper card, you can put a piece of white paper under the paper card to reduce the influence of ambient light on color recognition.
We will complete the task with two steps. First, we will learn to use the color recognition function of HuskyLens and the output of the recognized color ID. Then we can play their corresponding sound according to the output color ID, so that we can complete our color piano.
Task 1: Multiple Color Learning
At the very beginning, we need to allow the HuskyLens camera to recognize multiple colors, distinguish the differences in these colors, and be able to give feedback so that we can add notes later.
Task 2: Define Notes for Each Color
After being able to accurately identify each color, we can define a sound for each color, so that they can be played according to a certain rule, so that a color piano can be realized.
Here we need to let the HuskyLens sensor learn the color of each key and be able to output the color ID so that we can play the corresponding sound with the corresponding color. Such as: when it recognizes red (ID1), play the note Do.
Note: You need to enable the learning function first.
When the ID of the corresponding color is recognized in the HUSKYLENS sensor, the corresponding number will be displayed on the micro:bit.
How to write a program? In Mind+, it comes with instructions for playing notes, which are divided into low, mid, and high notes, as well as various beats. We just need to add the corresponding note to the corresponding ID.
Add the note modules to the completed task 1 program.
Run the program and find a music score, HUSKYLENS will play the corresponding notes according to the recognized colors.
Question 1: In the process, you must have discovered that when HUSKYLENS detects multiple colors, the notes produced by the color piano may not the notes we want. Do you have a good way to solve this problem?
Question 2: In the process, the color recognition will consider the similar color as the same color. How should we solve this problem?
Tip: Adjust the threshold of the recognition frame. In the submenu parameter setting interface of the color recognition function, there is a "Recognition Frame Threshold" parameter. The lower the value of this parameter, the lower the accuracy, but the more similar color blocks are recognized. As shown in the figure below, when the threshold is 20, only one yellow ball can be recognized; when the threshold is 0, all three yellow balls can be recognized. The threshold of the recognition frame can be adjusted according to the actual effect, so that the recognition accuracy is within an acceptable range.
In this lesson, we learned about the working principle of color recognition, learned how to make a color piano, and learned the color recognition function.
Color recognition is a very important function in artificial intelligence visual recognition, and it has been widely used in the industry. Think about what other interesting functions can be achieved by color recognition?
After completing the color piano, we will find a problem that the number of keys is relatively small. If we want to increase the number of keys, as the color increases, there will be many keys with similar colors, which may cause misidentification. If there are more keys, the camera's recognition range will not cover them all. So, is there any way we can broaden the range of our color piano?
(tips: We can use the A and B button on the micro:bit to realize the upgrade and downgrade functions.)