tag:blogger.com,1999:blog-30776219440571987882024-03-11T21:51:49.939-07:00Dror GluskaSoftware Development Consultant<br>
A blog about code anecdotes, a summary of past necessities and explorationUnknownnoreply@blogger.comBlogger145125tag:blogger.com,1999:blog-3077621944057198788.post-69023700267699110512023-01-12T02:09:00.001-08:002023-01-12T07:10:31.878-08:00Visualizing Embedded System Behavior with Perfetto<p>Segger SystemView and Percepio Tracealyzer have been the de-facto standard for visualizing embedded system behavior for the past few years, While Segger has been able to only visualize a single core, Percepio pricing is prohibitive to the hobbyist, last but not least is the toem Impulse which requires eclipse and some people have an aversion to that as well.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYbnEFfenFhSI-iQ9af7WUrTFWNOE_cDP1KjuaSnhjBKDUQKhUppfnVZlSa3JZrHwF3RgX2GowdgPNzSZaY0vVuw-2vfAAglLkYV_S89cLOSTATwkO6bYuvBWpP6s_zzbA3PfbABPDYuejcSIp4Ma9aKNUgDZMzV_XIaeCLkbrxDn61TarpOHUp-Ta/s1920/system%20analyzers.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1239" data-original-width="1920" height="414" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYbnEFfenFhSI-iQ9af7WUrTFWNOE_cDP1KjuaSnhjBKDUQKhUppfnVZlSa3JZrHwF3RgX2GowdgPNzSZaY0vVuw-2vfAAglLkYV_S89cLOSTATwkO6bYuvBWpP6s_zzbA3PfbABPDYuejcSIp4Ma9aKNUgDZMzV_XIaeCLkbrxDn61TarpOHUp-Ta/w640-h414/system%20analyzers.png" width="640" /></a></div><div class="separator" style="clear: both; text-align: center;"><a href="https://www.segger.com/news/systemview-maximum-insight-with-free-real-time-analysis-tool/">Segger SystemView</a> | <a href="https://percepio.com/tracealyzer/">Percepio Tracealyzer</a> | <a href="https://www.toem.de/index.php/resources/all-documents/85-systemview-reader">toem Impulse</a></div><div class="separator" style="clear: both; text-align: center;"><br /></div><h3 style="text-align: left;">Perfetto</h3><div><a href="https://ui.perfetto.dev/">Perfetto</a> is a System profiling, app tracing and trace analysis tool, it is open source and available as an online app, further more, its designed to handle millions of events, have SQL query, visual metrics and can handle multiple cores with no problem. Sounds like a perfect tool for the job no?</div><div><br /></div><div>However, it was designed to visualize chrome and linux kernel (ftrace) traces and unfurtunately no support for the standardized SystemView file format. But how complicated is that format?</div><div><br /></div><div>Apparently Espressif wrote a <a href="https://github.com/espressif/esp-idf/tree/master/tools/esp_app_trace/espytrace">SystemView decoder</a> and Perfetto wrote the protobuf files for ftrace, so its just a matter of mapping between them.</div><div><br /></div><h3 style="text-align: left;">Example</h3><div>This example is taken from sample 3 in the <a href="https://github.com/LiluSoft/Sysview-Perfetto-Converter">converter I wrote</a>.</div><div><br /></div><div><br /></div><div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgv-C2Vl2oAqalzd8oiJx0vDO9kOTnHhqhdDMov6nB8CKBsl8lk8bLIf4EVyoFwi2AeNQoHZZUSKLMKM6Qg4B5-Pu4AFznrSN1as_2f4cylICh-0bO_wHvNy4dNSmlLctpxnRhCdkdszRFdhdEdwNxEklaTaeERhzarTPyqO2fiaHMMOYSh71MwcLfd/s1920/systemview3.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="927" data-original-width="1920" height="310" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgv-C2Vl2oAqalzd8oiJx0vDO9kOTnHhqhdDMov6nB8CKBsl8lk8bLIf4EVyoFwi2AeNQoHZZUSKLMKM6Qg4B5-Pu4AFznrSN1as_2f4cylICh-0bO_wHvNy4dNSmlLctpxnRhCdkdszRFdhdEdwNxEklaTaeERhzarTPyqO2fiaHMMOYSh71MwcLfd/w640-h310/systemview3.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">SystemView with Example 3</td></tr></tbody></table><br /><div class="separator" style="clear: both; text-align: center;"><br /></div><div><br /></div><br /><br /></div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2254fSkY_fEqtd2uO7d1d6F9y-rg6jHxIH6oJKsvD0eAIWf7kElQWDxBblUPhcyyRK9wUV2v-ilzbYIYo9prTNWuzeGhtwikqo4HwtsGHRhxNGF6qfct65BMieIsWvyntuQAjzIwfOq8Cvgt8KAjmelW8RKlux4rtyCw7XXaZD027m_MjoTFLv4ZZ/s1920/perfetto3.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="937" data-original-width="1920" height="312" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2254fSkY_fEqtd2uO7d1d6F9y-rg6jHxIH6oJKsvD0eAIWf7kElQWDxBblUPhcyyRK9wUV2v-ilzbYIYo9prTNWuzeGhtwikqo4HwtsGHRhxNGF6qfct65BMieIsWvyntuQAjzIwfOq8Cvgt8KAjmelW8RKlux4rtyCw7XXaZD027m_MjoTFLv4ZZ/w640-h312/perfetto3.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Perfetto with Example 3</td></tr></tbody></table><br /><div><div class="separator" style="clear: both; text-align: center;"><br /></div></div><div>As always you can find the <a href="https://github.com/LiluSoft/Sysview-Perfetto-Converter">fruits of my labor at my GitHub account</a>.</div><div><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p></div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3077621944057198788.post-90372816474579679792022-12-28T02:56:00.001-08:002023-01-12T07:11:51.401-08:00ESP32 Performance Profiling<p>The ESP32 is very capable but even the most capable of devices can get overwhelmed when using it extensively, so how can we find out what is taking so long or which function should we optimize to make things even better?</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjVFm0aL-Atj_QaKp98f40dVWsV-DUmDI9DaA2tQwDPfTwYeg0qJPEclaQQIMGUwpbjAEVZFZWBGG5e8cmCu8pxAHKD4LyKI3oo9uGTFw7jtPWi_ymy0VdC-v_Tnqi8cyN5kxpgRUDEvF3qAtV5yK8wWUGzYJdcT6-EewSf9MIMtS9JgzBkKtmgvaCL" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="941" data-original-width="1280" height="294" src="https://blogger.googleusercontent.com/img/a/AVvXsEjVFm0aL-Atj_QaKp98f40dVWsV-DUmDI9DaA2tQwDPfTwYeg0qJPEclaQQIMGUwpbjAEVZFZWBGG5e8cmCu8pxAHKD4LyKI3oo9uGTFw7jtPWi_ymy0VdC-v_Tnqi8cyN5kxpgRUDEvF3qAtV5yK8wWUGzYJdcT6-EewSf9MIMtS9JgzBkKtmgvaCL=w400-h294" width="400" /></a></div><br /><br /><p></p><h3 style="text-align: left;">FreeRTOS Real Time Stats</h3><p style="direction: ltr;">FreeRTOS has a function <span style="font-family: courier;">vTaskGetRunTimeStats</span> which can get statistics for tasks runtime, however the API takes some performance away. So it needs to be enabled in the configuration <span style="font-family: courier;">CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS</span>.</p><p style="direction: ltr;">The following is an <a href="https://github.com/espressif/esp-idf/tree/master/examples/system/freertos/real_time_stats">example from esp-idf examples</a>.</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">Getting real time stats over 100 ticks
| Task | Run Time | Percentage
| stats | 938 | 0%
| IDLE | 403920 | 20%
| IDLE | 242954 | 12%
| spin3 | 225340 | 11%
| spin5 | 225360 | 11%
| spin6 | 225344 | 11%
| spin1 | 225392 | 11%
| spin4 | 225392 | 11%
| spin2 | 225360 | 11%
| esp_timer | 0 | 0%
| ipc1 | 0 | 0%
| ipc0 | 0 | 0%
Real time stats obtained
</pre></div>
<p style="direction: ltr;">While this can help you zone in the task that takes the most time it won't help you find the slowest function or stack trace.</p><p style="direction: ltr;">So when analyzing a potential performance issue, I'd use it as the first step to finding the tasks that take unusual amount of the CPU time.</p><h3 style="text-align: left;">Profilers</h3><p style="direction: ltr;">There are 2 general types of profilers, sampling profilers and tracing profilers. Sampling profiles capture the state of the program every x. Tracing Profilers inject hooks which are executed before and after each function.</p><p style="direction: ltr;">While its possible to run both on ESP32, I've encountered problems trying to use a tracing profiler on ESP32 on PlatformIO since it uses the same build_flags -pg for both the bootloader and the application and it causes issues with missing _mcount function.</p><p style="direction: ltr;">That leaves the sampling profiler option still available. But how to determine which function is currently running?</p><p style="direction: ltr;">We can do it in two ways:</p><p style="direction: ltr;">1. Sample each FreeRTOS task, since the stack pointer can be accessed we can check each stack pointer for the current PC (Program Counter) and determine which function is currently running. </p><p style="direction: ltr;">2. Sample the currently executing function, this can be done with interrupts since the interrupts share the currently executing task stack all we need to do is skip the counter's functions and the rest of the stack belongs to the currently running task. Since we have two cores we need to do it for both cores.</p><p style="direction: ltr;">I've chosen to go with option no. 2 since it tells me more about what is currently running.</p><h3 style="text-align: left;">ESP32 Semihosting Profiler</h3><p style="direction: ltr;">It works by sampling the entire call stack and keeping statistics on the number of times a function was seen in the call stack, it then sends that information to the host computer though semihosting file system.</p><p style="direction: ltr;">Once the sampling is done, the raw samples are processed to get the function name and locate the source line and the results are displayed and callgrind file is generated.</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">prvIdleTask tasks.c:3973 -> esp_vApplicationIdleHook freertos_hooks.c:63 : 783 307926512 76424201
vPortTaskWrapper port.c:131 -> prvIdleTask tasks.c:3973 : 783 307926512 76424201
esp_vApplicationIdleHook freertos_hooks.c:63 -> cpu_ll_waiti cpu_ll.h:183 : 781 307926512 76424201
vPortTaskWrapper port.c:131 -> spin_task4 real_time_stats_example_main.c:163 : 206 70213898 30992208
vPortTaskWrapper port.c:131 -> spin_task1 real_time_stats_example_main.c:151 : 204 70204906 31215652
vPortTaskWrapper port.c:131 -> spin_task5 real_time_stats_example_main.c:167 : 202 86176568 34547921
vPortTaskWrapper port.c:131 -> spin_task2 real_time_stats_example_main.c:155 : 201 97310444 38941256
vPortTaskWrapper port.c:131 -> spin_task6 real_time_stats_example_main.c:171 : 201 89345478 35511218
vPortTaskWrapper port.c:131 -> spin_task3 real_time_stats_example_main.c:159 : 201 68584391 30480011
spin_task4 real_time_stats_example_main.c:163 -> spin_task real_time_stats_example_main.c:143 : 134 62236086 27673084
...
</pre></div>
<p style="direction: ltr;"></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgbLRfjil47AjgeL8KIiy2jv5c_VtJN_vQvpok11qtsq-hkMQy7LhsCL2HfxyaSC6vfr5k1oG9FRG_iuBE586_x5-0x2r8ESZyggerfdhzeVb-N7-gAuU64jIeQ02Re_v_Uu0xfVHKp0EIg0HBzMiScJZyfK2XZscSnjS2ebTBzxX9kOwR4FzzqWvIK" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="637" data-original-width="1506" height="270" src="https://blogger.googleusercontent.com/img/a/AVvXsEgbLRfjil47AjgeL8KIiy2jv5c_VtJN_vQvpok11qtsq-hkMQy7LhsCL2HfxyaSC6vfr5k1oG9FRG_iuBE586_x5-0x2r8ESZyggerfdhzeVb-N7-gAuU64jIeQ02Re_v_Uu0xfVHKp0EIg0HBzMiScJZyfK2XZscSnjS2ebTBzxX9kOwR4FzzqWvIK=w640-h270" width="640" /></a></div><br /><p></p><p style="direction: ltr;">As always you can find the <a href="https://github.com/LiluSoft/esp32-semihosting-profiler">fruits of my labor at my GitHub account</a>.</p><pre style="background-color: white; font-stretch: inherit; font-variant-east-asian: inherit; font-variant-numeric: inherit; line-height: inherit; overflow: visible;"><span style="font-family: Times New Roman;"><br /></span></pre>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3077621944057198788.post-48419241912402157082022-11-30T01:45:00.006-08:002023-01-12T07:12:45.124-08:00Generic Gamepad for Toy Cars<p>Some kids love motorized toys, cars, trucks and basically anything that makes a noise or have a motor can be a child's toy.</p><p>I've been searching for a quick, simple, cheap, generic option to replace the remote controls with something I can easily source without building a specialized PCB or costing too much and I think I've found that option.</p><div style="direction: ltr;">This is the <a href="https://s.click.aliexpress.com/e/_DdsH8EJ">battlebot</a> I've been using it for, the original controller stopped working after 3 minutes.</div><div style="direction: ltr;"><br /></div><div style="direction: ltr;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgMouCjgLHPdI-h-AnTEkO5GQ_nna7MSHDl9NHPy9VRQeRb9V61mJOGEzvmocx5g3JhtGsIv0iiEOzVvPLydpxdFbQJs_33GMS2L_PIofU8dzHLNNt6_UuZ9gTPRXJfRexp9Zxo1Zm_7fUqzfrj52ex2L8xeANNSkdRtMRfRWhN4hTun4vQPXyVVeI0" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="276" data-original-width="351" height="240" src="https://blogger.googleusercontent.com/img/a/AVvXsEgMouCjgLHPdI-h-AnTEkO5GQ_nna7MSHDl9NHPy9VRQeRb9V61mJOGEzvmocx5g3JhtGsIv0iiEOzVvPLydpxdFbQJs_33GMS2L_PIofU8dzHLNNt6_UuZ9gTPRXJfRexp9Zxo1Zm_7fUqzfrj52ex2L8xeANNSkdRtMRfRWhN4hTun4vQPXyVVeI0" width="305" /></a></div></div><h4 style="text-align: left;">Wemos D1 R32</h4><p>The <a href="https://s.click.aliexpress.com/e/_DBy1aRp">Wemos D1 R32</a> is ESP32 in Arduino Uno form factor. The pinout is standard while still allowing access to all Arduino Uno standard pins and GPIO2 for onboard led. The <a href="https://user-images.githubusercontent.com/43091864/100515375-0659e000-31bf-11eb-8d8b-72f07e1d36a2.png">schematics</a> are available.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEipZJBKuVZJRt1MZhdIs9MTNQBuT_pUoeZFSXuGGQzRkR879K7mGSVnCxbHgGUySWd7xcPZ_-JlOO1IXD5WzVtJ8uivX_5bvyr0DRYV8j9LPB1XpHuIXOR3ltpYgyuOvaK-JjVzP5_PjQ9boM45R7lNX54NhI4Y_lxcIwqQkuhA4fSiDg4e3J6WaaG7" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="868" data-original-width="1153" height="482" src="https://blogger.googleusercontent.com/img/a/AVvXsEipZJBKuVZJRt1MZhdIs9MTNQBuT_pUoeZFSXuGGQzRkR879K7mGSVnCxbHgGUySWd7xcPZ_-JlOO1IXD5WzVtJ8uivX_5bvyr0DRYV8j9LPB1XpHuIXOR3ltpYgyuOvaK-JjVzP5_PjQ9boM45R7lNX54NhI4Y_lxcIwqQkuhA4fSiDg4e3J6WaaG7=w640-h482" width="640" /></a></div><br /><br /><p></p><p><br /></p><h4 style="text-align: left;">L293D Motor Control Shield</h4><p>The <a href="https://s.click.aliexpress.com/e/_DeuD2B1">motor shield</a> was originally created by <a href="https://learn.adafruit.com/adafruit-motor-shield">Adafruit</a> but has since become ubiquitous through other online shops.</p><p></p><div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEiGEsw59edfe40UHEIj7uor1dVQYi4WiVlcl0j9gs9VWaZDHShzMj3p5dFan7gi02jSbWptRRGzpk0ZGVg-V8GxdDhLBY7hcd6_k9VxyuNUgjdo1LtoxwIVhFjG-4JMk8_0FKpl3O-HxOPjXW4oKYhrXenxpCqg8Nok31NUSb4yMXnLMuW0i_oHRuqH" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="779" data-original-width="973" height="512" src="https://blogger.googleusercontent.com/img/a/AVvXsEiGEsw59edfe40UHEIj7uor1dVQYi4WiVlcl0j9gs9VWaZDHShzMj3p5dFan7gi02jSbWptRRGzpk0ZGVg-V8GxdDhLBY7hcd6_k9VxyuNUgjdo1LtoxwIVhFjG-4JMk8_0FKpl3O-HxOPjXW4oKYhrXenxpCqg8Nok31NUSb4yMXnLMuW0i_oHRuqH=w640-h512" width="640" /></a></div><br /><br /></div>But it was designed to work with Arduino Uno so I had to go through the schematic to understand how the ESP32 should access it.<p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEig4oqZj0mPj3vJMkEJ5ozoblTNjUiCGYh-gmy7cEGmAIB6OdO2cjh3naYvXedS58UbtSoix2xXtEO1AL340NCRGBvsx2Hkcoxq4DqvxWFEC3eUNT4IV2K5jO8X-ja3zfsG_f6fFn9fvbEAag-QzcgelDGUyPNEYr5lm8YT-TBbUJVRXJ6IwC01dDiK" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1130" data-original-width="1899" height="380" src="https://blogger.googleusercontent.com/img/a/AVvXsEig4oqZj0mPj3vJMkEJ5ozoblTNjUiCGYh-gmy7cEGmAIB6OdO2cjh3naYvXedS58UbtSoix2xXtEO1AL340NCRGBvsx2Hkcoxq4DqvxWFEC3eUNT4IV2K5jO8X-ja3zfsG_f6fFn9fvbEAag-QzcgelDGUyPNEYr5lm8YT-TBbUJVRXJ6IwC01dDiK=w640-h380" width="640" /></a></div><br /><div style="background-color: #1e1e1e; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #6a9955;"> DIR_SER - GPIO12</span></div><div><span style="color: #6a9955;"> PWM1A - GPIO13 - servo2</span></div><div><span style="color: #6a9955;"> PWM1B - GPIO5 - servo1</span></div><div><span style="color: #6a9955;"> PWM2A - GPIO23 - dc1</span></div><div><span style="color: #6a9955;"> DIR_LATCH - GPIO19</span></div><div><span style="color: #6a9955;"> DIR_EN - GPIO14</span></div><div><span style="color: #6a9955;"> PWM0A - GPIO27 - dc4</span></div><div><span style="color: #6a9955;"> PWM0B - GPIO16 - dc3</span></div><div><span style="color: #6a9955;"> DIR_CLK - GPIO17</span></div><div><span style="color: #6a9955;"> PWM2B - GPIO25 - dc2</span></div></div><br /><p></p><h4 style="text-align: left;">Bluepad32</h4><p>Bluepad32 was created by Ricardo Quesada, it was designed to to allow using newer gamepad controllers with retro gaming consoles.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgdptqwEVpGm70geamYv6VaTGyatOO-_wk04noZbvhPX-CIU_ybfOhLrA1O3b7GB_GsrBx5t9w24OSF_kP1Nm6rlOS354yHrx7e_LloKoz6vi9hYAReBA_gEu7FOusd8I-kC5RleeVAhxZHPaXwRhL8IJx2lJ6nUnTK3CbHrErsipz0PE-NcyeYdW7G" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="173" data-original-width="512" height="135" src="https://blogger.googleusercontent.com/img/a/AVvXsEgdptqwEVpGm70geamYv6VaTGyatOO-_wk04noZbvhPX-CIU_ybfOhLrA1O3b7GB_GsrBx5t9w24OSF_kP1Nm6rlOS354yHrx7e_LloKoz6vi9hYAReBA_gEu7FOusd8I-kC5RleeVAhxZHPaXwRhL8IJx2lJ6nUnTK3CbHrErsipz0PE-NcyeYdW7G=w400-h135" width="400" /></a></div><div class="separator" style="clear: both; text-align: center;"><br /></div>While it has a <a href="https://gitlab.com/ricardoquesada/bluepad32/-/blob/main/docs/supported_gamepads.md">long list of supported controllers</a>, I've found that the <a href="https://s.click.aliexpress.com/e/_DES9TIL">DualShock 4</a> works best for me.<p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgrEnz7_7xj2nOYhRBlCQInClMtf5oeODLTKtGBw5pEkgTeQrRDKMIPWMYMeESdhOIm1TU9uvcyfp9uKqYd7wDgRoV1xFxKVr34Cc-T1_8v5_4Euu4jBNqh0_jknZknzghw6zCFT1OlQ-DoEiwLDrQ0NVyDedxMqTRE7Qoc-FDmGRPOjgl-BZ8tIw88" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1500" data-original-width="1500" height="240" src="https://blogger.googleusercontent.com/img/a/AVvXsEgrEnz7_7xj2nOYhRBlCQInClMtf5oeODLTKtGBw5pEkgTeQrRDKMIPWMYMeESdhOIm1TU9uvcyfp9uKqYd7wDgRoV1xFxKVr34Cc-T1_8v5_4Euu4jBNqh0_jknZknzghw6zCFT1OlQ-DoEiwLDrQ0NVyDedxMqTRE7Qoc-FDmGRPOjgl-BZ8tIw88" width="240" /></a></div><div style="direction: ltr;">To pair the DualShock 4 to ESP32 you'll need to turn it on while pressing the "SHARE" button and PS Button.</div><div style="direction: ltr;"><br /></div><h4 style="text-align: left;">Firmware</h4><div style="direction: ltr;">After mapping the pins between the ESP32 and the Motor Shield, I needed to write a new Bluepad32 platform, I've called mine uni_platform_motor. The new platform uses Adafruit's AFMotor to control the motors, but you can use anything else you'd like to control.</div><div style="direction: ltr;"><br /></div><div style="direction: ltr;">The uni platform has the following important events:</div><div style="direction: ltr;">- <span style="font-family: courier;">on_init_complete</span> - which fires when the platform is initialized</div><div style="direction: ltr;">- <span style="font-family: courier;">on_device_connected</span> - where you can do motor arming </div><div style="direction: ltr;">- <span style="font-family: courier;">on_device_disconnected</span> - where you can do motor disarming, stopping the engines etc'</div><div style="direction: ltr;">- <span style="font-family: courier;">on_gamepad_data</span> - where you can process incoming joysticks and buttons processing and convert it into motors commands.</div><div style="direction: ltr;"><br /></div><h4 style="text-align: left;">Porting AFMotor</h4><div style="direction: ltr;">The AFMotor was not designed for ESP32 and I did a very crude job of porting it since it was just to see if they can work together and it was good enough.</div><div style="direction: ltr;">Please note that the Motor Driver uses a few pins that might not be mapped to GPIO, (for example: pin 14), to use these pins its not enough to use the <span style="font-family: courier;">gpio_set_direction</span>, but rather you should use the more generic <span style="font-family: courier;">gpio_config</span>.</div><div style="direction: ltr;"><br /></div><h4 style="text-align: left;">Motor Direction Algorithm</h4><div style="direction: ltr;"><a href="https://coderdojoathenry.org/2019/02/24/hackers-how-to-control-a-robots-wheel-motors-based-on-joystick-movements/">Michael Madden explained it best</a>.</div><div style="direction: ltr;"><br /></div><h4 style="text-align: left;">Summary</h4><div style="direction: ltr;">The proposed solution enables relatively cheap components to be bound to a single remote and so I don't have to disassemble the controller or build my own. </div><div style="direction: ltr;">Also, thinking about the future, it can be used for scratch or Arduino development with the same hardware so another one for the pros list.</div><div style="direction: ltr;"><br /></div><div style="direction: ltr;">Lastly, The DualShock4 can be used with other game consoles, PCs, TV Stocks etc' therefore future proofing the whole expense.</div><div style="direction: ltr;"><br /></div><div style="direction: ltr;">I would like to express my gratitude again to Ricardo Quesada for making the Bluepad32.</div><div style="direction: ltr;"><br /></div><div style="direction: ltr;">As always you can find the <a href="https://github.com/drorgl/esp32_battlebot">fruits of my labor at my GitHub account</a>.</div><div style="direction: ltr;"><br /></div><div style="direction: ltr;"><br /></div><div style="direction: ltr;"><br /></div><div style="direction: ltr;"><br /></div><div style="direction: ltr;"><br /></div><div style="direction: ltr;"><br /></div><div style="direction: ltr;"><br /></div><div style="direction: ltr;"><br /></div><div style="direction: ltr;"><br /></div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3077621944057198788.post-72396808186927865882022-11-23T04:15:00.003-08:002023-01-12T07:13:05.751-08:00Embedding Lua in your Projects<p><a href="https://www.lua.org/">Lua</a> is lighweight programing language designed for embedding, it was created in 1993 by Roberto Ierusalimschy, Luiz Henrique de Figueiredo, and Waldemar Celes.</p><p>Lua even has a very fast LuaJIT which sadly I cannot use since I'm aiming for embedded MCU.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://www.lua.org/images/luaa.gif" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="256" data-original-width="256" height="256" src="https://www.lua.org/images/luaa.gif" width="256" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><br /></div>Fortunately, Lua has great documentation and huge community, while the language needs some getting used it, it can do everything I need and more and embedding it was by far the fastest from previous <a href="https://blog.drorgluska.com/2022/11/embedding-micropython-in-your-projects.html">MicroPython</a> and <a href="https://blog.drorgluska.com/2022/11/embedding-quickjs-in-your-projects.html">QuickJS</a>.<div><br /></div><div>Installation of the Lua library was very simple, download the release, extract it in the lib folder and add <a href="https://github.com/drorgl/lua_embedding/blob/b332d9829549fd5dccac4a3db0d7e5a02f6b6764/lib/lua-5.4.4/library.json">library.json</a>.</div><div><br /></div><div>To gain the performance I needed, I've modified <a href="https://github.com/drorgl/lua_embedding/blob/main/lib/lua-5.4.4/luaconf.h#L112-L113">luaconf.h</a></div><div><br /></div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><table><tbody><tr><td><pre style="line-height: 125%; margin: 0px;">1
2
3</pre></td><td><pre style="line-height: 125%; margin: 0px;"><span style="color: #888888;">/* Default configuration ('long long' and 'double', for 64-bit Lua) */</span>
<span style="color: #557799;">#define LUA_INT_DEFAULT LUA_INT_INT</span>
<span style="color: #557799;">#define LUA_FLOAT_DEFAULT LUA_FLOAT_FLOAT</span>
</pre></td></tr></tbody></table></div>
<div><br /></div><div>To initialize Lua a new state is created and the default libraries added to it</div><div><br /></div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><table><tbody><tr><td><pre style="line-height: 125%; margin: 0px;">1
2
3
4
5
6
7
8
9</pre></td><td><pre style="line-height: 125%; margin: 0px;">lua_State <span style="color: #333333;">*</span>L <span style="color: #333333;">=</span> luaL_newstate(); <span style="color: #888888;">/* create state */</span>
<span style="color: #008800; font-weight: bold;">if</span> (L <span style="color: #333333;">==</span> <span style="color: #007020;">NULL</span>)
{
l_message(argv[<span style="color: #0000dd; font-weight: bold;">0</span>], <span style="background-color: #fff0f0;">"cannot create state: not enough memory"</span>);
<span style="color: #008800; font-weight: bold;">return</span> EXIT_FAILURE;
}
printf(<span style="background-color: #fff0f0;">"setting libs</span><span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\r\n</span><span style="background-color: #fff0f0;">"</span>);
luaL_openlibs(L);
</pre></td></tr></tbody></table></div>
<div><br /></div><div>Next is injecting the demo function into the state</div><div><br /></div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><table><tbody><tr><td><pre style="line-height: 125%; margin: 0px;">1
2
3</pre></td><td><pre style="line-height: 125%; margin: 0px;"><span style="color: #008800; font-weight: bold;">function</span> <span style="color: #0066bb; font-weight: bold;">transform</span>(a,b)
<span style="color: #008800; font-weight: bold;">return</span> (a<span style="color: #333333;">^</span><span style="color: #0000dd; font-weight: bold;">2</span><span style="color: #333333;">/</span><span style="color: #007020;">math.sin</span>(<span style="color: #0000dd; font-weight: bold;">2</span><span style="color: #333333;">*</span><span style="color: #007020;">math.pi</span><span style="color: #333333;">/</span>b))<span style="color: #333333;">-</span>(a<span style="color: #333333;">/</span><span style="color: #0000dd; font-weight: bold;">2</span>)
<span style="color: #008800; font-weight: bold;">end</span>
</pre></td></tr></tbody></table></div>
<div><br /></div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><table><tbody><tr><td><pre style="line-height: 125%; margin: 0px;"> 1
2
3
4
5
6
7
8
9
10
11
12</pre></td><td><pre style="line-height: 125%; margin: 0px;"><span style="color: #333399; font-weight: bold;">char</span> <span style="color: #333333;">*</span>code <span style="color: #333333;">=</span> <span style="background-color: #fff0f0;">"function transform(a,b)</span><span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\r\n</span><span style="background-color: #fff0f0;"> \</span>
<span style="background-color: #fff0f0;">return (a^2/math.sin(2*math.pi/b))-(a/2)</span><span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\r\n</span><span style="background-color: #fff0f0;"> \</span>
<span style="background-color: #fff0f0;">end</span><span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\r\n</span><span style="background-color: #fff0f0;">"</span>;
<span style="color: #008800; font-weight: bold;">if</span> (luaL_loadstring(L, code) <span style="color: #333333;">==</span> LUA_OK)
{
<span style="color: #008800; font-weight: bold;">if</span> (lua_pcall(L, <span style="color: #0000dd; font-weight: bold;">0</span>, <span style="color: #0000dd; font-weight: bold;">0</span>, <span style="color: #0000dd; font-weight: bold;">0</span>) <span style="color: #333333;">==</span> LUA_OK)
{
<span style="color: #888888;">// If it was executed successfully we remove the code from the stack</span>
lua_pop(L, lua_gettop(L));
}
}
</pre></td></tr></tbody></table></div>
<div><br /></div><div>And to run the transform function, the function name is pushed along with its arguments, the function is called and the result is pulled back from the stack</div><div><br /></div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><table><tbody><tr><td><pre style="line-height: 125%; margin: 0px;">1
2
3
4
5
6</pre></td><td><pre style="line-height: 125%; margin: 0px;">lua_getglobal(L, <span style="background-color: #fff0f0;">"transform"</span>);
lua_pushnumber(L, i);
lua_pushnumber(L, i);
lua_call(L, <span style="color: #0000dd; font-weight: bold;">2</span>, <span style="color: #0000dd; font-weight: bold;">1</span>);
<span style="color: #333399; font-weight: bold;">double</span> res <span style="color: #333333;">=</span> lua_tonumber(L, <span style="color: #333333;">-</span><span style="color: #0000dd; font-weight: bold;">1</span>);
</pre></td></tr></tbody></table></div>
<div><br /></div><h4 style="text-align: left;">Summary</h4><div>Lua was the easiest to integrate, it is the fastest scripting engine and if speed is your primary concern it should be one of the top candidates.</div><div><br /></div><div>In my particular use case:</div><div><ul style="text-align: left;"><li>x64 native vs Lua slow down is about 4x times</li><li>ESP32 native vs Lua slow down is about 20x times</li></ul><div><br /></div></div><div><br /></div><div><div>As always you can find the <a href="https://github.com/drorgl/lua_embedding">fruits of my labor at my GitHub account</a>.</div><div><br /></div></div><div><br /></div><div><div><br /><p></p></div></div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3077621944057198788.post-20093043343197060042022-11-23T02:28:00.003-08:002022-11-23T03:27:58.329-08:00Embedding QuickJS in your Projects<p><a href="https://github.com/bellard/quickjs">QuickJS</a> is small and embeddable Javascript engine made by the famous <a href="https://en.wikipedia.org/wiki/Fabrice_Bellard">Fabrice Bellard</a>, it supports ES2020 specifications and made available under MIT license.</p><p>You might have read about my porting of his <a href="https://blog.drorgluska.com/2022/07/risc-v-linux-on-esp32.html">TinyEMU to ESP32</a> which could boot linux.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjzb7KVdSlhe4cl6D_-bUvhl0FxC_P8LhJVUAS1dGCqz01RL9kpFJHu2Eoeb6KEoIRt1hkaK_cVS3N82ozT5D_ySSRb4k-keTA25LyrwUwR7oQMorS-zPDnBrlmVMHObh7DqnVJUCSZt7-_3kqlYBsBrWDU98IQo_CdpNpr6bmdF7pq-4S9YTjUU24X" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="2048" data-original-width="2048" height="400" src="https://blogger.googleusercontent.com/img/a/AVvXsEjzb7KVdSlhe4cl6D_-bUvhl0FxC_P8LhJVUAS1dGCqz01RL9kpFJHu2Eoeb6KEoIRt1hkaK_cVS3N82ozT5D_ySSRb4k-keTA25LyrwUwR7oQMorS-zPDnBrlmVMHObh7DqnVJUCSZt7-_3kqlYBsBrWDU98IQo_CdpNpr6bmdF7pq-4S9YTjUU24X=w400-h400" width="400" /></a></div><div><br /></div><div><br /></div><div>During my hunt for scripting engine that can be integrated into a new embedded product I've encountered <a href="https://blog.drorgluska.com/2022/11/embedding-micropython-in-your-projects.html">MicroPython</a> and QuickJS, while the <a href="https://bellard.org/quickjs/quickjs.html">documentation</a> is a bit lacking, the code was much easier to read and compile, there are no complicated build scripts and it was basically dropping the last version in the lib folder and selecting which files should be compiled with <a href="https://github.com/drorgl/quickjs_embedding/blob/9a3a672232d1a6ede2b080b3450e5ff092680b9d/lib/quickjs/library.json">library.json</a><p></p><p>Once the library was compiled, integrating it into my demo was very easy and supporting multiple execution contexts is very easy since its not sharing context and variables.</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><table><tbody><tr><td><pre style="line-height: 125%; margin: 0px;"> 1
2
3
4
5
6
7
8
9
10
11
12
13
14</pre></td><td><pre style="line-height: 125%; margin: 0px;">JSRuntime <span style="color: #333333;">*</span>rt <span style="color: #333333;">=</span> JS_NewRuntime();
<span style="color: #008800; font-weight: bold;">if</span> (<span style="color: #333333;">!</span>rt)
{
fprintf(stderr, <span style="background-color: #fff0f0;">"qjs: cannot allocate JS runtime</span><span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\n</span><span style="background-color: #fff0f0;">"</span>);
}
JS_SetMemoryLimit(rt, <span style="color: #0000dd; font-weight: bold;">80</span> <span style="color: #333333;">*</span> <span style="color: #0000dd; font-weight: bold;">1024</span>);
JS_SetMaxStackSize(rt, <span style="color: #0000dd; font-weight: bold;">10</span> <span style="color: #333333;">*</span> <span style="color: #0000dd; font-weight: bold;">1024</span>);
JSContext <span style="color: #333333;">*</span>ctx <span style="color: #333333;">=</span> JS_NewContext(rt);
<span style="color: #008800; font-weight: bold;">if</span> (<span style="color: #333333;">!</span>ctx)
{
fprintf(stderr, <span style="background-color: #fff0f0;">"qjs: cannot allocate JS context</span><span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\n</span><span style="background-color: #fff0f0;">"</span>);
}
</pre></td></tr></tbody></table></div>
<p>The JSRuntime object represents the JavaScript engine, its responsible for memory allocation and C function calls. </p><p>The JSContext object respresents the execution context where JavaScript functions and variables live.</p><p>Our demo workload</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><table><tbody><tr><td><pre style="line-height: 125%; margin: 0px;">1
2
3</pre></td><td><pre style="line-height: 125%; margin: 0px;"><span style="color: #008800; font-weight: bold;">function</span> transform(a, b) {
<span style="color: #008800; font-weight: bold;">return</span> (a <span style="color: #333333;">^</span> <span style="color: #0000dd; font-weight: bold;">2</span> <span style="color: #333333;">/</span> <span style="color: #007020;">Math</span>.sin(<span style="color: #0000dd; font-weight: bold;">2</span> <span style="color: #333333;">*</span> <span style="color: #007020;">Math</span>.PI <span style="color: #333333;">/</span> b)) <span style="color: #333333;">-</span> a <span style="color: #333333;">/</span> <span style="color: #0000dd; font-weight: bold;">2</span>;
}
</pre></td></tr></tbody></table></div>
<p>We'll <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/eval">eval(uate)</a> the function to get it into the context</p><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><table><tbody><tr><td><pre style="line-height: 125%; margin: 0px;">1
2
3
4
5
6</pre></td><td><pre style="line-height: 125%; margin: 0px;"><span style="color: #008800; font-weight: bold;">const</span> <span style="color: #333399; font-weight: bold;">char</span> <span style="color: #333333;">*</span>expr <span style="color: #333333;">=</span> <span style="background-color: #fff0f0;">"function transform(a,b){return (a^2/Math.sin(2*Math.PI/b))-a/2;}"</span>;
JSValue r <span style="color: #333333;">=</span> JS_Eval(ctx, expr, strlen(expr), <span style="background-color: #fff0f0;">""</span>, <span style="color: #0000dd; font-weight: bold;">0</span>);
<span style="color: #008800; font-weight: bold;">if</span> (JS_IsException(r))
{
printf(<span style="background-color: #fff0f0;">"Error evaluating script</span><span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\r\n</span><span style="background-color: #fff0f0;">"</span>);
}
</pre></td></tr></tbody></table></div><p>And once we have the function in the context, we can get it by name and execute it with the a and b arguments</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><table><tbody><tr><td><pre style="line-height: 125%; margin: 0px;"> 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19</pre></td><td><pre style="line-height: 125%; margin: 0px;">JSValue args[<span style="color: #0000dd; font-weight: bold;">2</span>];
args[<span style="color: #0000dd; font-weight: bold;">0</span>] <span style="color: #333333;">=</span> JS_NewFloat64(ctx, (<span style="color: #333399; font-weight: bold;">double</span>)i);
args[<span style="color: #0000dd; font-weight: bold;">1</span>] <span style="color: #333333;">=</span> JS_NewFloat64(ctx, (<span style="color: #333399; font-weight: bold;">double</span>)i);
JSValue res <span style="color: #333333;">=</span> JS_Call(ctx, func, global, <span style="color: #0000dd; font-weight: bold;">2</span>, args);
<span style="color: #008800; font-weight: bold;">if</span> (JS_IsException(res))
{
printf(<span style="background-color: #fff0f0;">"Error Executing transform</span><span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\r\n</span><span style="background-color: #fff0f0;">"</span>);
}
<span style="color: #008800; font-weight: bold;">if</span> (<span style="color: #333333;">!</span>JS_IsNumber(res))
{
printf(<span style="background-color: #fff0f0;">"is not number!</span><span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\r\n</span><span style="background-color: #fff0f0;">"</span>);
}
<span style="color: #333399; font-weight: bold;">double</span> result;
<span style="color: #008800; font-weight: bold;">if</span> (JS_ToFloat64(ctx, <span style="color: #333333;">&</span>result, res))
{
printf(<span style="background-color: #fff0f0;">"error parsing number</span><span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\r\n</span><span style="background-color: #fff0f0;">"</span>);
}
</pre></td></tr></tbody></table></div>
<h4 style="text-align: left;">Summary</h4><p>QuickJS looks very interesting as a JavaScript runtime engine, its relatively fast and the code seems self explanatory. It has been embedded in <a href="https://docs.rs/quick-js/latest/quick_js/">rust</a>, as an isolated VM in <a href="https://www.npmjs.com/package/quickjs-emscripten">Node JS</a> and more.</p><p>While QuickJS is different from MicroPython, its faster to execute the same workload, it was more readable for me and faster to setup and compile, one of the major drawbacks is that if you want to use it as an alternative to MicroPython, you'll need to implement your own hardware drivers for SPI, I2C etc'.</p><p>You may find Carlos Alberto's <a href="https://calbertts.medium.com/writing-native-modules-in-c-for-quickjs-engine-49043587f2e2">Writing native modules in C for QuickJS engine</a> useful.</p><p>Lastly, I've attempted to modify QuickJS to use floats instead of double since ESP32 FPU is single precision only, it will probably make it non-standard and fail many JavaScript standard tests but I've included it here anyway.</p><p>In my particular use case:</p><p></p><ul style="text-align: left;"><li>x64 native vs QuickJS slowdown is about x9 times</li><li>ESP32 native vs QuickJS slowdown is about x63 times</li><li>ESP32 native vs QuickJS using <a href="https://github.com/drorgl/quickjs_embedding/tree/float-experiment">float</a> slowdown is about x25 times</li></ul><p></p><p>You may view the official benchmarks <a href="https://bellard.org/quickjs/bench.html">here</a>.</p><p>As always you can find the <a href="https://github.com/drorgl/quickjs_embedding">fruits of my labor at my GitHub account</a>.</p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p></div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3077621944057198788.post-12956344497023178712022-11-17T04:59:00.008-08:002022-11-17T21:58:03.264-08:00Embedding Micropython in your Projects<p><a href="https://micropython.org/">MicroPython</a> is a python implementation for microcontrollers and other low resource requirement implementation which can make it perfect for embedding without significantly increasing your delivered executable or firmware.</p><p>The project was created by Damien George at 2013 and has since grown and improved and even earned its place at <a href="https://micropython.org/resources/publish/Spacebel_MicroPythonOBCP_Dasia2018_Paper.pdf">OBCP</a>s and is planned to reach space on board <a href="https://en.wikipedia.org/wiki/Euclid_(spacecraft)">Euclid</a> at 2023 as well as integrating into <a href="https://mediatum.ub.tum.de/doc/1519584/document.pdf">RODOS</a>.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEi1pl4VWnMtoDR2Vf61tbQU9rpOPEF3m89aiO27nRnny0nkfP8UFRbTWTAkkExHCUb-INvnzh6AzEQTdkvLdB9ZDEUXfZLYVDtqK27EjCTNBaWULzuglCG3o5Qzz6K7hWYqtdFERVMNK15mjd3huRI7vjhW06TYti9m8pQCcYFuWkB_5JNSHQ1HIDCz" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="600" data-original-width="589" height="640" src="https://blogger.googleusercontent.com/img/a/AVvXsEi1pl4VWnMtoDR2Vf61tbQU9rpOPEF3m89aiO27nRnny0nkfP8UFRbTWTAkkExHCUb-INvnzh6AzEQTdkvLdB9ZDEUXfZLYVDtqK27EjCTNBaWULzuglCG3o5Qzz6K7hWYqtdFERVMNK15mjd3huRI7vjhW06TYti9m8pQCcYFuWkB_5JNSHQ1HIDCz=w629-h640" width="629" /></a></div><br /><br /><p></p><p>If it can work for spacecrafts, why shouldn't it work for you?</p><p>My original goal was to use MicroPython as an expression evaluation and enrichment engine for embedded data acquisition, however, I've found that the complexity of actually integrating it into a PlatformIO project was too big to miss on an opportunity to make it an easier task.</p><h3 style="text-align: left;">MicroPython Project Structure</h3><p>After downloading the <a href="https://github.com/micropython/micropython/releases">release</a> (in my case 1.19), the archive has the following interesting <a href="https://docs.micropython.org/en/latest/develop/gettingstarted.html#folder-structure">folders</a>:</p><p>py - contains MicroPython VM</p><p>ports - contains platform specific implementation and support functionality</p><p>extmod - extra modules</p><p>I was a bit naïve, so I've decided to build all files in py and windows port but there were so many errors I've realized its probably the wrong approach.</p><p>I've started digging in the makefiles since I wasn't sure what to look for, the <a href="https://docs.micropython.org/en/latest/develop/gettingstarted.html#compile-and-build-the-code">instructions</a> were pretty standard. but once you start looking in the makefiles, there's <a href="https://github.com/micropython/micropython/blob/451ded8d7b746bf73a1f4a4db500716cb7587257/py/mkrules.cmake#L65-L169">plenty going on there</a>.</p><p>But there's no easy way to use that in PlatformIO as far as I know, so I went ahead and checked what was executing when and why, there's <a href="https://docs.micropython.org/en/latest/develop/qstr.html">some documentation about qstr</a> but there are also modules and version info, some of it to optimize the build and prevent a rebuild of the whole qstr header.</p><p>I wanted to avoid precompiling the headers and keeping them in git, since it will complicate version updates, build flags changes and built-in module changes, there have been similar approached with <a href="https://www.nxp.com/docs/en/application-note/AN13242.pdf">NXP port</a> but I wanted it to be more robust and developer friendly.</p><p>I've decided to keep it short and just write a script that executes all the scripts on the appropriate files and folders and after some fiddling with the list of sources that needed to be compiled, the project compiled on both Windows and ESP32.</p><h3 style="text-align: left;">Building MicroPython on PlatformIO</h3><p>PlatformIO uses scons to build, scons keeps a series of flags and dictionaries of the files to build and uses the compiler so actually generate the executable and firmware. Though in ESP32 case it uses cmake as well to build esp-idf.</p><p>PlatformIO also added hooks before and after building the project, allowing extensibility through scripting, in this case I've chosen to execute a script in the library so its always executing before the build.</p><p>I went ahead and made the build system a bit more flexible by allowing different sources and includes to be used for different platforms. You may want to look in <a href="https://github.com/drorgl/micropython_embedding/blob/9a94b282e3dcd18ef3fba6a1b45f2ad7a1b75405/lib/micropython-1.19/scripts/build_settings.py">build_settings.py</a> to see how.</p><p>In short, the script determines the framework and platform and reads the library.json->build->environments, it appends all the sources and flags from common and then looks for a specific platform and appends the sources and flags from there as well.</p><p>I then read list of file selectors (SRC_FILTER) and <a href="https://github.com/drorgl/micropython_embedding/blob/9a94b282e3dcd18ef3fba6a1b45f2ad7a1b75405/lib/micropython-1.19/scripts/source_file_extractor.py">build a list of source files</a> so it can be used to generate the headers with <a href="https://github.com/drorgl/micropython_embedding/blob/9a94b282e3dcd18ef3fba6a1b45f2ad7a1b75405/lib/micropython-1.19/scripts/generate_strings.py">generate_strings.py</a></p><p></p><ul style="text-align: left;"><li>makeversionhdr - generates the mpversion.h</li><li>makeqstrdefs pp - generates qstr.i.last from a batch of source files and headers</li><li>makeqstrdefs split qstr - extracts a list of qstr from the last qstr.i.last, this may run a few times with a few different qstr.i.last</li><li>makeqstrdefs split module - extracts a list of modules from the last qstr.i.last, this may run a few times with a few different qstr.i.last</li><li>makeqstrdefs cat module - generates a collected file from the splitted modle files</li><li>makeqstrdefs cat qstr - generates a collected file from the splitted qstr files</li><li>makemoduledefs - generates moduledefs.h from the collected module file</li><li>qstrdefs_preprocessed - precompiles strings into preprocessed.h</li><li>qstrdefs_generated - generates generated.h from preprocessed.h</li></ul><div>Once this process is done, the qstr and modules dictionaries are ready to be compiled into the library.</div><div><br /></div><h3 style="text-align: left;">Generic Port</h3><div>MicroPython uses a set of configuration header files and functions so it will be <a href="https://docs.micropython.org/en/v1.19.1/develop/porting.html">portable</a> and work across multiple environments but to run it without any special hardware modules requires very little configuration.</div><div><br /></div><div>I've created a "generic" port:</div><div><ul style="text-align: left;"><li>mimimal gc</li><li>unhandled exceptions are sent to stderr</li><li>all filesystem operations are throwing errors</li><li>all python's print are sent to stdout, stdin does nothing</li></ul><h3 style="text-align: left;">Initialization</h3><div>To initialize micropython we need a stack and heap and then initialize the stack, the stack limit, the garbage collector which also handles allocations and finally MicroPython.</div><div><br /></div><div>Please note there are two stack limits, one for python (pystack) and one for the OS limit, which is used to limit the recursion the MicroPython engine uses - it should be less than the OS stack size, looking at MicroPython's code, its about 2k less than the OS stack.</div><div><br /></div><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><table><tbody><tr><td><pre style="line-height: 125%; margin: 0px;"> 1
2
3
4
5
6
7
8
9
10
11
12</pre></td><td><pre style="line-height: 125%; margin: 0px;">instance<span style="color: #333333;">-></span>stack <span style="color: #333333;">=</span> (<span style="color: #333399; font-weight: bold;">uint8_t</span> <span style="color: #333333;">*</span>)malloc(instance<span style="color: #333333;">-></span>stack_size);
instance<span style="color: #333333;">-></span>heap <span style="color: #333333;">=</span> (<span style="color: #333399; font-weight: bold;">uint8_t</span> <span style="color: #333333;">*</span>)malloc(instance<span style="color: #333333;">-></span>heap_size);
mp_stack_ctrl_init();
mp_stack_set_limit(instance<span style="color: #333333;">-></span>stack_size);
<span style="color: #888888;">// Initialize heap</span>
gc_init(instance<span style="color: #333333;">-></span>heap, instance<span style="color: #333333;">-></span>heap <span style="color: #333333;">+</span> instance<span style="color: #333333;">-></span>heap_size);
mp_pystack_init(instance<span style="color: #333333;">-></span>stack, instance<span style="color: #333333;">-></span>stack <span style="color: #333333;">+</span> instance<span style="color: #333333;">-></span>stack_size);
<span style="color: #888888;">// Initialize interpreter</span>
mp_init();
</pre></td></tr></tbody></table></div>
</div><div><br /></div><div><br /></div><h3 style="text-align: left;">NLR</h3></div><div>NLR stands for non-local return, which is how MicroPython handles exceptions in C, using a stack of jumps.</div><div><br /></div><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><table><tbody><tr><td><pre style="line-height: 125%; margin: 0px;"> 1
2
3
4
5
6
7
8
9
10
11
12</pre></td><td><pre style="line-height: 125%; margin: 0px;"><span style="color: #333399; font-weight: bold;">nlr_buf_t</span> nlr;
<span style="color: #008800; font-weight: bold;">if</span> (nlr_push(<span style="color: #333333;">&</span>nlr) <span style="color: #333333;">==</span> <span style="color: #0000dd; font-weight: bold;">0</span>)
{
<span style="color: #333399; font-weight: bold;">mp_obj_t</span> retval <span style="color: #333333;">=</span> mp_call_function_n_kw(func, argc, <span style="color: #0000dd; font-weight: bold;">0</span>, argv);
nlr_pop();
<span style="color: #008800; font-weight: bold;">return</span> retval;
}
<span style="color: #008800; font-weight: bold;">else</span>
{
mp_obj_print_exception(<span style="color: #333333;">&</span>mp_stderr_print, MP_OBJ_FROM_PTR(nlr.ret_val));
<span style="color: #008800; font-weight: bold;">return</span> (<span style="color: #333399; font-weight: bold;">mp_obj_t</span>)nlr.ret_val;
}
</pre></td></tr></tbody></table></div>
</div><div><br /></div><div>1-2 define a temporary nlr buffer and push the current context.</div><div>4 execute a MicroPython function</div><div>5 pop for successful execution</div><div>8 in case there's an error between lines 3-7 the IP will jump to line 9.</div><div>10 display the error</div><div><br /></div><h3 style="text-align: left;">Executing Scripts and Calling Functions</h3><div>While it seems very similar in python, there is a difference, executing a script might have an output, it is not captured, so there is no way to use that output unless you plan on capturing and parsing Python's print.</div><div><br /></div><div>This is why I chose to split that functionality into two parts, the first part executes a script, creating functions / variables on the local/global scope and the second part executes a function and uses its return value.</div><div><br /></div><div>For executing scripts, MicroPython needs to parse the string and compile it into MicroPython bytecode and execute it. So once the script is in the context, we can use the module for looking up the function name and execute it.</div><div><br /></div><div>We'll pass the following script to the compilation code</div><div><br /></div><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><table><tbody><tr><td><pre style="line-height: 125%; margin: 0px;">1
2
3</pre></td><td><pre style="line-height: 125%; margin: 0px;"><span style="color: #008800; font-weight: bold;">import</span> <span style="color: #0e84b5; font-weight: bold;">math</span>;
<span style="color: #008800; font-weight: bold;">def</span> <span style="color: #0066bb; font-weight: bold;">transform</span>(a,b):
<span style="color: #008800; font-weight: bold;">return</span> (a<span style="color: #333333;">**</span><span style="color: #0000dd; font-weight: bold;">2</span><span style="color: #333333;">/</span>math<span style="color: #333333;">.</span>sin(<span style="color: #0000dd; font-weight: bold;">2</span><span style="color: #333333;">*</span>math<span style="color: #333333;">.</span>pi<span style="color: #333333;">/</span>b))<span style="color: #333333;">-</span>a<span style="color: #333333;">/</span><span style="color: #0000dd; font-weight: bold;">2</span>
</pre></td></tr></tbody></table></div>
</div><div><br /></div><div><br /></div><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><table><tbody><tr><td><pre style="line-height: 125%; margin: 0px;"> 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18</pre></td><td><pre style="line-height: 125%; margin: 0px;"><span style="color: #333399; font-weight: bold;">nlr_buf_t</span> nlr;
<span style="color: #008800; font-weight: bold;">if</span> (nlr_push(<span style="color: #333333;">&</span>nlr) <span style="color: #333333;">==</span> <span style="color: #0000dd; font-weight: bold;">0</span>)
{
qstr src_name <span style="color: #333333;">=</span> <span style="color: #0000dd; font-weight: bold;">1</span> <span style="color: #888888;">/*MP_QSTR_*/</span>;
<span style="color: #333399; font-weight: bold;">mp_lexer_t</span> <span style="color: #333333;">*</span>lex <span style="color: #333333;">=</span> mp_lexer_new_from_str_len(src_name, fragment, strlen(fragment), <span style="color: #007020;">false</span>);
qstr source_name <span style="color: #333333;">=</span> lex<span style="color: #333333;">-></span>source_name;
<span style="color: #333399; font-weight: bold;">mp_parse_tree_t</span> pt <span style="color: #333333;">=</span> mp_parse(lex, MP_PARSE_FILE_INPUT);
<span style="color: #333399; font-weight: bold;">mp_obj_t</span> module_fun <span style="color: #333333;">=</span> mp_compile(<span style="color: #333333;">&</span>pt, source_name, <span style="color: #007020;">false</span>);
mp_call_function_0(module_fun);
nlr_pop();
<span style="color: #008800; font-weight: bold;">return</span> <span style="color: #007020;">NULL</span>;
}
<span style="color: #008800; font-weight: bold;">else</span>
{
mp_obj_print_exception(<span style="color: #333333;">&</span>mp_stderr_print, MP_OBJ_FROM_PTR(nlr.ret_val));
<span style="color: #008800; font-weight: bold;">return</span> (<span style="color: #333399; font-weight: bold;">mp_obj_t</span>)nlr.ret_val;
}
</pre></td></tr></tbody></table></div>
</div><div><br /></div><div>And now we can lookup transform (line 1) function and call it (line 10)</div><div><br /></div><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><table><tbody><tr><td><pre style="line-height: 125%; margin: 0px;"> 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18</pre></td><td><pre style="line-height: 125%; margin: 0px;"><span style="color: #333399; font-weight: bold;">mp_obj_t</span> transform_function <span style="color: #333333;">=</span> mp_obj_dict_get(mp_locals_get(), mp_obj_new_str(<span style="background-color: #fff0f0;">"transform"</span>, strlen(<span style="background-color: #fff0f0;">"transform"</span>)));
<span style="color: #333399; font-weight: bold;">mp_obj_t</span> args[<span style="color: #0000dd; font-weight: bold;">2</span>];
args[<span style="color: #0000dd; font-weight: bold;">0</span>] <span style="color: #333333;">=</span> mp_obj_new_float(<span style="color: #0000dd; font-weight: bold;">1</span>);
args[<span style="color: #0000dd; font-weight: bold;">1</span>] <span style="color: #333333;">=</span> mp_obj_new_float(<span style="color: #0000dd; font-weight: bold;">2</span>);
<span style="color: #333399; font-weight: bold;">nlr_buf_t</span> nlr;
<span style="color: #008800; font-weight: bold;">if</span> (nlr_push(<span style="color: #333333;">&</span>nlr) <span style="color: #333333;">==</span> <span style="color: #0000dd; font-weight: bold;">0</span>)
{
<span style="color: #333399; font-weight: bold;">mp_obj_t</span> retval <span style="color: #333333;">=</span> mp_call_function_n_kw(transform_function, argc, <span style="color: #0000dd; font-weight: bold;">0</span>, argv);
nlr_pop();
<span style="color: #008800; font-weight: bold;">return</span> retval;
}
<span style="color: #008800; font-weight: bold;">else</span>
{
mp_obj_print_exception(<span style="color: #333333;">&</span>mp_stderr_print, MP_OBJ_FROM_PTR(nlr.ret_val));
<span style="color: #008800; font-weight: bold;">return</span> (<span style="color: #333399; font-weight: bold;">mp_obj_t</span>)nlr.ret_val;
}
</pre></td></tr></tbody></table></div>
</div><div><br /></div><div><br /></div><h3 style="text-align: left;">Summary</h3><div>MicroPython has advanced light years since its first inception and it is very capable and enables rapid prototyping and even usable for space applications. That being said, its still a bit slow to function as a generic programming language and more appropriate for coordinating calls to C functions.</div><div><br /></div><div>The other thing I've found lacking is documentation, to write this demo I needed to go over multiple sources and demos, look for forks, pull requests and read a lot of source code, its still not 100% clear to me what all the defines are doing exactly and what to expect in terms of performance impact for each of them.</div><div><br /></div><div>In my particular use case:</div><div><ul style="text-align: left;"><li>x64 native vs MicroPython slowdown is about x16 times.</li><li>ESP32 native vs MicroPython slowdown is about x100 times.</li></ul></div><div><br /></div><div>For the curious minds of where did the CPU spent its time, I'm very happy to have found <a href="https://github.com/VerySleepy/verysleepy">Very Sleepy</a></div><div><br /></div><div><div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEiJT1-juXoK1O5ViZpNbsSFR7x55r553r8KgRUbE9_hG3N13qYKUXw7OnyVu0bwbagHbL2kFSG6OwDc3TUkzoazG9L_LGybUzPQcmtpfNDY1FeBJAOy2Dfw0zt_tbGFiiPZkCH-pMmTgxKZvgRb7BZJTxzMB0UgR4r_sRWJ-p5ApeHvsxvHrvhUfwCq" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="600" data-original-width="1103" src="https://blogger.googleusercontent.com/img/a/AVvXsEiJT1-juXoK1O5ViZpNbsSFR7x55r553r8KgRUbE9_hG3N13qYKUXw7OnyVu0bwbagHbL2kFSG6OwDc3TUkzoazG9L_LGybUzPQcmtpfNDY1FeBJAOy2Dfw0zt_tbGFiiPZkCH-pMmTgxKZvgRb7BZJTxzMB0UgR4r_sRWJ-p5ApeHvsxvHrvhUfwCq=s16000" /></a></div><br /><br /></div></div></div></div><div><br /></div><div>As always you can find the <a href="https://github.com/drorgl/micropython_embedding">fruits of my labor at my GitHub account</a>.</div><div><br /></div><div><br /></div><div>Further reading:</div><div><ul style="text-align: left;"><li><a href="https://www.snaums.de/static/resources/2017-12-mpy.pdf">https://www.snaums.de/static/resources/2017-12-mpy.pdf</a></li><li><a href="https://micropython.org/resources/publish/Spacebel_MicroPythonOBCP_Dasia2018_Paper.pdf">https://micropython.org/resources/publish/Spacebel_MicroPythonOBCP_Dasia2018_Paper.pdf</a></li><li><a href="https://mediatum.ub.tum.de/doc/1519584/document.pdf">https://mediatum.ub.tum.de/doc/1519584/document.pdf</a></li><li><a href="https://indico.esa.int/event/269/contributions/4156/attachments/3161/4272/Presentation_-_Qualification_of_a_Virtual_Machine_an_OBCP_Engine_Based_on_MicroPython.pdf">https://indico.esa.int/event/269/contributions/4156/attachments/3161/4272/Presentation_-_Qualification_of_a_Virtual_Machine_an_OBCP_Engine_Based_on_MicroPython.pdf</a></li></ul></div><p><br /></p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3077621944057198788.post-17407985224699195832022-07-28T13:10:00.001-07:002022-07-28T13:10:36.133-07:00Using the ILI9488 <p>The ILITEK <a href="https://s.click.aliexpress.com/e/_Dm04VK3">ILI9488</a> is one of the larger and cheaper SPI displays available to the maker community,, available in 3.5" and 4". However, there are a few workable issues that prevent this display from being great.</p><p></p><div class="separator" style="clear: both; text-align: center;">
<a href="https://s.click.aliexpress.com/e/_DCB5aGF" target="_blank"><img height="357" src="https://blogger.googleusercontent.com/img/a/AVvXsEi0sKtit8vTtvkhxoxq2uvsvd5hRD03pm1-SnVDmYElLnv14AkKnvtDS6_QjEX_fUnAu91KdpILq24BLhmyo0l0lemDj1wQMlAo0PI0mlDCWFh0OCuHFA0No66gStpwzWYf0CrDNhCpisbH8CvWbopRoGT2zAxUz3uX1I4emyDBnY283UO6nN8D7KKV=w640-h357" width="640" /></a>
</div><div><br /></div><h3 style="text-align: left;">Specifications</h3><div>What's called ILI9488 is actually the LCD controller with an optional touch panel, you can mostly find it with XPT2046 resistive touch controller.</div><div><br /></div><div>ILI9488 (<a href="https://focuslcds.com/content/ILI9488.pdf">datasheet</a>):</div><div>- 3/4 wire SPI, software configurable</div><div>- 480x320 Pixels</div><div>- 3 modes supported: 16bit (65k colors) / 18bit (262k colors) / 24bit (16.7m colors)</div><div><div><br /></div><div>XPT2046 (<a href="http://grobotronics.com/images/datasheets/xpt2046-datasheet.pdf">datasheet</a>):</div><div>- 12bit 125khz resistive touch panel</div><div>- pressure sensitive</div><div>- temperature sensor</div><div>- 4-wire SPI</div><div>- Supports touch interrupt</div><div><br /></div><div>5v to 3.3v regulator, please note that you should short J1 if you're using 3.3v</div><div><br /></div><div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhoOpjFo2dgNwlbrL3lF1K20EjxQZTe7EA3h6iysrDjDc_WLvc72UUAmpWI5MWXO71qElXO23sWgFo7CMZf-jJ7sJMj0_7hFAesfqtWQMwHUNjTCqwC_Usx4umXKtRn2FYLhQ49Kkazf8mFDaGzJIoqrxmBNGN4OVAe9a5Tf86WDdJQdYyAS5zUdl2B" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="254" data-original-width="323" src="https://blogger.googleusercontent.com/img/a/AVvXsEhoOpjFo2dgNwlbrL3lF1K20EjxQZTe7EA3h6iysrDjDc_WLvc72UUAmpWI5MWXO71qElXO23sWgFo7CMZf-jJ7sJMj0_7hFAesfqtWQMwHUNjTCqwC_Usx4umXKtRn2FYLhQ49Kkazf8mFDaGzJIoqrxmBNGN4OVAe9a5Tf86WDdJQdYyAS5zUdl2B=s16000" /></a></div><br /><br /></div><div><br /></div><h3 style="text-align: left;">General Issues</h3><div><br /></div><div style="direction: ltr;">The ILI9488 can be bought in two versions, <a href="http://www.lcdwiki.com/res/MSP4021/4.0inch_SPI_Schematic.pdf">one with a diode</a> and<a href="http://www.lcdwiki.com/3.5inch_SPI_Module_ILI9488_SKU:MSP3520"> one without</a>, I've yet to determine the functionality of the diode, but it seems that <a href="https://github.com/Bodmer/TFT_eSPI/discussions/898">others think</a> the diode can prevent the display from releasing the MISO line, unfortunately I didn't keep the diode so I can't validate this claim.</div><div style="direction: ltr;"><br /></div><div style="direction: ltr;">The <a href="http://www.lcdwiki.com/res/MSP4021/4.0inch_SPI_Schematic.pdf">schematics</a> are available if you want to explore it further.</div><div><br /></div><div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgQGOZTPMBvyRn2xEkNaZK4JQd2-W7fr2MLqzLQIpnPF3Meu4MDZF6K_KHSaHcDL_kFRUeNTp2RvqZBcwoOft6ngBbFBz-EpAqBUUJaIhvHnbsguBD4o7F32gS9FD0JEH2rwy2XAhuMb2U98PkH2-YOQlQeijfQdjzJ4jP8ewpr8XyhEDh_DwkirPwI" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="257" data-original-width="425" src="https://blogger.googleusercontent.com/img/a/AVvXsEgQGOZTPMBvyRn2xEkNaZK4JQd2-W7fr2MLqzLQIpnPF3Meu4MDZF6K_KHSaHcDL_kFRUeNTp2RvqZBcwoOft6ngBbFBz-EpAqBUUJaIhvHnbsguBD4o7F32gS9FD0JEH2rwy2XAhuMb2U98PkH2-YOQlQeijfQdjzJ4jP8ewpr8XyhEDh_DwkirPwI=s16000" /></a></div><br /><br /></div><h3 style="text-align: left;">LVGL Issues</h3><div>When I first started working with my ILI9488 the colors were a bit off but I attributed it to cheap and low quality display which was probably defective. but then I've started wondering if its possible to fix since the controller can be configured with other voltages it pushes to the panel. </div><div><br /></div><div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgPUDI8CCl2mfC1t6eY2GbnhzSP0-UIMX_sFieg4K0g4Y5iE2NvQ_Iwlvoo9xrTLtTW9TXhlshHGFB5-jPKFqqCdP7w-B7VGyh16uW5aZbz8E_yvzjRTj6FNNJEkcWHCGyJRUAWOQr8WXOGGNrbGP7TMI3DlZkZH15HL1dOQNbvhiPrudymhwPYI7sw" style="margin-left: auto; margin-right: auto;"><img alt="" data-original-height="631" data-original-width="1000" height="404" src="https://blogger.googleusercontent.com/img/a/AVvXsEgPUDI8CCl2mfC1t6eY2GbnhzSP0-UIMX_sFieg4K0g4Y5iE2NvQ_Iwlvoo9xrTLtTW9TXhlshHGFB5-jPKFqqCdP7w-B7VGyh16uW5aZbz8E_yvzjRTj6FNNJEkcWHCGyJRUAWOQr8WXOGGNrbGP7TMI3DlZkZH15HL1dOQNbvhiPrudymhwPYI7sw=w640-h404" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Original Configuration (16 bit)</td></tr></tbody></table><br />Once I've discovered the <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/2dd9a0ec9ff5fd97ea276d749ca665b7f2daf069/lib/lvgl_esp32_drivers/lvgl_tft/ili9488.c#L58-L77">setting</a>, I've changed the brightness and changed how <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/2dd9a0ec9ff5fd97ea276d749ca665b7f2daf069/lib/lvgl_esp32_drivers/lvgl_tft/ili9488.c#L143-L158">RGB565 is parsed into the format ILI9488 expects</a>. note its not the most optimized way, but the modification was good enough for my tests.</div><div><br /></div><div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjnwvSt1kCA7No5LlR7I8UhPJ9K9QNu2caZIbsPj_UTMrRz0HVT2CbsXqaANbVkWW0As-gK7I6g6n8YUFWOF5jr5lCKNajmkRS0hGvVUZaH4h5YUNo6zgeC8GGypKjIMK0GOfN-lP2yDy0SbAUMMF6MxiDBKkVcZTDnhqgc6NmGoLfFRHKtXJ7DI1MO" style="margin-left: auto; margin-right: auto;"><img alt="" data-original-height="600" data-original-width="1000" height="384" src="https://blogger.googleusercontent.com/img/a/AVvXsEjnwvSt1kCA7No5LlR7I8UhPJ9K9QNu2caZIbsPj_UTMrRz0HVT2CbsXqaANbVkWW0As-gK7I6g6n8YUFWOF5jr5lCKNajmkRS0hGvVUZaH4h5YUNo6zgeC8GGypKjIMK0GOfN-lP2yDy0SbAUMMF6MxiDBKkVcZTDnhqgc6NmGoLfFRHKtXJ7DI1MO=w640-h384" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">After the modifications</td></tr></tbody></table><br /><br /></div><div>I wanted to see if the display will work at 80Mhz, unfortunately its not, but seeing some of the graphics I guess that it can be fixed with a logic analyzer and some patience.</div><div><br /></div><div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjzs2qbtkUDYyTWxhwnsQFobrlIi9lIA14jMxa3EbrEXbfsjvA-3Ucj0z2xyEnEMSfamHHEq2L4nlfXbayDkyqDtK1ZI0k0addHGKEZdita5xS1W0r0aFlLvo5v0VV975FBYtfOOqh7G3xWSSz3RlkbXI5Ic0_1re1GbSXrHV-uWGvANzvlitAErTPR" style="margin-left: auto; margin-right: auto;"><img alt="" data-original-height="654" data-original-width="1000" height="418" src="https://blogger.googleusercontent.com/img/a/AVvXsEjzs2qbtkUDYyTWxhwnsQFobrlIi9lIA14jMxa3EbrEXbfsjvA-3Ucj0z2xyEnEMSfamHHEq2L4nlfXbayDkyqDtK1ZI0k0addHGKEZdita5xS1W0r0aFlLvo5v0VV975FBYtfOOqh7G3xWSSz3RlkbXI5Ic0_1re1GbSXrHV-uWGvANzvlitAErTPR=w640-h418" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">80Mhz</td></tr></tbody></table><br /><br /></div><div><br /></div><h3 style="text-align: left;">ESP32 Specific Issues</h3><div>While it might not be specifically ESP32 issues, its issues that you might encounter while integrating it with ESP32. The most prominent issue is the way CS works in ESP32, it seems that CS issues are common in the embedded world, the STM32 has a similar issue with NSS not properly controlled by the cube's code.</div><div><br /></div><div>To support multiple transactions with multiple devices on the same SPI bus, the ESP32 switches off the CS signal between transactions which is great, however, the way ILI9488 works is that if you switch off CS after you've sent a read request, it switches from 4-wire SPI to 3-wire SPI. </div><div><br /></div><div>There are a few ways to solve this issue:</div><div>1. Use software CS, set to low before a transaction and set to high after you're done receiving.</div><div>2. Use the new <a href="SPI_TRANS_CS_KEEP_ACTIVE">SPI_TRANS_CS_KEEP_ACTIVE</a> flag for transactions.</div><div><br /></div><div>But wait, what's the problem with working half duplex (3-wire)? </div><div>Well, if you share the same SPI bus with the touch panel it locks the MISO line on LOW and won't allow the touch panel from transmitting touch data.</div><div><br /></div><h4 style="text-align: left;">Solutions?</h4><div>Well, I've given up on getting data from the display and for most uses its good enough, so you can disconnect the MISO line from the display and keep it working for the touch panel.</div><div><br /></div><div>Another possibility is to put a 1k resistor in series to the display MISO. This way the panel won't lock the touch and you can keep the buggy code until you can get it fixed to your satisfaction.</div><div><br /></div><div>Unfortunately working in half-duplex is not currently possible if you're using the LVGL driver since it will attempt to set the bus to 4-wire mode for the touch panel to work.</div><div><br /></div><div><br /></div><div><div><br /></div></div></div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3077621944057198788.post-28865159163139850172022-07-24T12:24:00.002-07:002022-07-28T13:14:30.824-07:00LVGL ESP32 and Desktop Development Walkthrough<p>UI for Embedded is always a hassle, find the right MCU, find the right Display, connect the right wires and that's even before writing the first line of code that actually shows anything on the display, drivers, graphic libraries and input libraries can be a pain to use, not to mention a pain to write.</p><p>Fortunately, we no longer live in a cave, we have PlatformIO, LVGL and drivers for many of the LCDs available for commercial use and maker community.</p><p>I propose a way to start quickly so we can save some of the bring up time for a new setup, configuration is done in the kconfig way, so its really go through the menu, change a setting, compile and test.</p><p>There is a <a href="https://github.com/lvgl/lv_port_esp32">getting started for ESP32</a> on LVGL github, Gabor Kiss-Vamosi wrote a <a href="https://blog.lvgl.io/2019-01-31/esp32">tutorial</a> and we have some <a href="https://docs.lvgl.io/latest/en/html/get-started/espressif.html">documentation</a>, Espressif event got an <a href="https://github.com/espressif/esp-iot-solution">example repository</a>, but I've discovered my way is a bit more flexible and easier to use once its set-up, since you can develop on the desktop (LVGL refers to it as <a href="https://docs.lvgl.io/latest/en/html/get-started/pc-simulator.html">Simulator</a>) and don't have to upload each revision, plus you don't have to do it, just fork my project and you're good to go. In any case, the full instructions are below, so you can mix and match versions and for me it provided a pretty much consistent experience.</p><h3 style="text-align: left;">Hardware</h3><p>There are many kits you can buy, each one with its own quirks but my goal was to test what I had and unfortunately I had none of the kits.</p><p>So I got out a <a href="https://s.click.aliexpress.com/e/_DCYzmQJ">perfboard</a>, a few <a href="https://s.click.aliexpress.com/e/_DlUiP5Z">pin headers</a> and a resistor (<a href="https://blog.drorgluska.com/2022/07/using-ili9488.html">more on that later</a>) and built my own. </p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgUMIuAtSNWLRClg6A46vg3Q0oN3oHyxpXFhpVZix9zcmEQ2oRFeAZniDYfuwU2fYSk_dkNTuy0Z4NIhAZkW3T-Lrsh1kPVHSDvYZ2SW9jWZ0YWZms6C9pxV3ef70urmweo5wW0Y7iHr7fDwYDVYfWO89-Nmtxw9mi_HH44tyPVaNwUFS-HlGTTuqYL" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="458" data-original-width="811" height="362" src="https://blogger.googleusercontent.com/img/a/AVvXsEgUMIuAtSNWLRClg6A46vg3Q0oN3oHyxpXFhpVZix9zcmEQ2oRFeAZniDYfuwU2fYSk_dkNTuy0Z4NIhAZkW3T-Lrsh1kPVHSDvYZ2SW9jWZ0YWZms6C9pxV3ef70urmweo5wW0Y7iHr7fDwYDVYfWO89-Nmtxw9mi_HH44tyPVaNwUFS-HlGTTuqYL=w640-h362" width="640" /></a></div><br /><br /><p></p><p>Please note that I have yet to test any of them, so do your own research before purchasing, the following are affiliate links.</p><h4 style="text-align: left;">ESP32-LCDKit</h4><p>The <a href="https://espressif-docs.readthedocs-hosted.com/projects/espressif-esp-dev-kits/en/latest/esp32/esp32-lcdkit/user_guide.html">ESP32-LCDKit</a> looks like its the most versatile for Graphic development, the <a href="https://espressif-docs.readthedocs-hosted.com/projects/espressif-esp-dev-kits/en/latest/_static/schematics/esp32-lcdkit/SCH_ESP32-LCDKit_V1.1_20190218.pdf">schematic</a> is available and you can use which ever ESP32 you want (as long as its a 38 pins)</p><div class="separator" style="text-align: center;"><a href="https://s.click.aliexpress.com/e/_DeZEhJd" style="margin-left: 1em; margin-right: 1em;" target="_blank"><img height="640" src="//ae01.alicdn.com/kf/H4fb78e12600a4e3a9773e36f54d0d6fdH.jpg" width="640" /></a></div><p></p>
<h4 style="text-align: left;">ESP-WROVER-KIT</h4><p>Next in line is the <a href="https://docs.espressif.com/projects/esp-idf/en/latest/hw-reference/get-started-wrover-kit.html">ESP-WROVER-KIT</a>, like the previous one, its <a href="https://dl.espressif.com/dl/schematics/ESP-WROVER-KIT_V4_1.pdf">schematics</a> are also available, but one major drawback is that the kit does not include a touch screen, only a display so its less suitable for interactive UI.</p>
<div style="text-align: center;"><a href="https://s.click.aliexpress.com/e/_Ddh8MiP" target="_blank"><img height="640" src="//ae01.alicdn.com/kf/Hd27ec85ae8544da783c706a60e57662cp.jpg" width="640" /></a></div><div style="text-align: left;"><br /></div><h4 style="text-align: left;">ESP32-S2-Kaluga-1 Kit</h4><div style="text-align: left;"><br /></div><div style="text-align: left;">Next in line from Espressif is the <a href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32s2/hw-reference/esp32s2/user-guide-esp32-s2-kaluga-1-kit.html">ESP32-S2-Kaluga-1 Kit</a>, this kit is more versatile and according to the documentation it does have a 3.2"<a href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32s2/hw-reference/esp32s2/user-guide-esp-lyrap-lcd32-v1.2.html"> touch screen</a> but it can also have an <a href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32s2/hw-reference/esp32s2/user-guide-esp-lyrat-8311a_v1.3.html">audio extension</a>, a <a href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32s2/hw-reference/esp32s2/user-guide-esp-lyrap-toucha-v1.1.html">touch panel</a> (not display, just a board with touch) and a <a href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32s2/hw-reference/esp32s2/user-guide-esp-lyrap-cam-v1.1.html">camera module</a>.</div><div style="text-align: left;"><br /></div>
<div style="text-align: center;"><a href="https://s.click.aliexpress.com/e/_Dk3Hwin" target="_blank"><img height="640" src="//ae01.alicdn.com/kf/S04e99794c25e4a4badcb050c51db6e8b7.jpg" width="640" /></a></div><div><br /></div><div style="direction: ltr;"><br /></div><h4 style="text-align: left;">WT32-SC01</h4><div style="direction: ltr;"><br /></div><div style="direction: ltr;">Last but not least, The WT32-SC01 seems like it has great potential for having all the hardware on a single board, mounting holes and capacitive touch screen, <a href="https://www.amstron.es/wp-content/uploads/2021/06/HMI-TFT_3.5_Wi-Fi_BT_BLE_MCU_WT32-SC01DataSheetV3.3-2-with-nuts-201228_AMSTRON.pdf">schematics</a> and <a href="https://github.com/wireless-tag-com">code samples</a> are available.</div><div><br /></div><div><div style="text-align: center;">
<a href="https://s.click.aliexpress.com/e/_DkiOE8L" target="_blank"><img height="640" src="//ae01.alicdn.com/kf/S655fdde09f8e411589f25c187044a05bR.jpg" width="640" /></a></div>
<h3 style="text-align: left;">Source Control</h3><p>We can't really start a project without source control, how can we track changes? how can we go back to a stable state?</p><p>Atlassian has a great <a href="https://wac-cdn.atlassian.com/dam/jcr:e7e22f25-bba2-4ef1-a197-53f46b6df4a5/SWTM-2088_Atlassian-Git-Cheatsheet.pdf">cheat sheet</a> and there is <a href="https://code.visualstudio.com/docs/editor/versioncontrol">built in support</a> in Visual Studio Code. </p><p>Lets initialize a new git repo:</p><p><span style="font-family: courier;">git init</span></p>I also recommend committing each stage of your setup, it will help you to track changes and find out which code caused the change.<h3 style="text-align: left;">PlatformIO</h3><p>PlatformIO is a development platform that enables writing code in multiple platforms while maintaining a consistent experience.</p><p>In this case, we'd like to initialize a new project:</p><p><span style="font-family: courier;">pio init</span></p><h3 style="text-align: left;">LVGL</h3><p>Now that we have an empty project, we'll need to add lvgl to it, so go ahead and extract latest release from <a href="https://github.com/lvgl/lvgl/releases">https://github.com/lvgl/lvgl/releases</a> into <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/tree/master/lib/lvgl-8.3.0">lib/lvgl</a>.</p><p>Than take <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/lib/lvgl-8.3.0/library.json">library.json</a> from our example project and copy it into lib/lvgl root, what this library.json file actually does is allow you to select which parts of lvgl gets compiled.</p><h3 style="text-align: left;">Native / Desktop Drivers</h3><p>There are multiple ways of working with LVGL but one of the better ways is getting your UI completely disconnected from your business logic and running the UI on your PC, this way its easier to design, debug and verify, on top of it, you can use LVGL's <a href="https://docs.lvgl.io/master/others/snapshot.html">snapshot API</a> to automatically validate your views so they can stay consistent no matter which changes you do. </p><p>The way LVGL works on the desktop is by using <a href="https://www.libsdl.org/">SDL2</a>.</p><p>So first we'll extract the latest source from <a href="https://github.com/lvgl/lv_drivers">https://github.com/lvgl/lv_drivers</a> into <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/tree/master/lib/lv_drivers">lib/lv_drivers</a></p><p>Then we'll copy <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/lib/lv_drivers/lv_drv_conf_template.h">lv_drv_conf_template.h</a> to <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/include/native/lv_drv_conf.h">include/native/lv_drv_conf.h</a> and enable the file (change <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/include/native/lv_drv_conf.h#L11">#if 0 to 1</a>)</p><p>Then we'll modify <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/lib/lv_drivers/library.json">library.json</a> to include <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/lib/lv_drivers/library.json#L12">SDL dependency</a>, otherwise PlatformIO dependency detection won't work properly due to the way SDL is included through a #DEFINE macro.</p><p><span style="font-family: courier;"> "dependencies":[</span></p><p><span style="font-family: courier;"> {</span></p><p><span style="font-family: courier;"> "name":"SDL2"</span></p><p><span style="font-family: courier;"> }</span></p><p><span style="font-family: courier;"> ],</span></p><p>And modify library.json to <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/lib/lv_drivers/library.json#L20">remove an incompatible source file</a>:</p><p><span style="font-family: courier;"> "build": {</span></p><p><span style="font-family: courier;"> "srcFilter" : [</span></p><p><span style="font-family: courier;"> "-<display/ILI9341.c>"</span></p><p><span style="font-family: courier;"> ]</span></p><p><span style="font-family: courier;"> }</span></p><p>Lastly, we'll update <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/include/native/lv_drv_conf.h#L17">include/native/lv_drv_conf.h</a> in appropriate place and copy our menu configuration keys to SDL configuration keys, otherwise our menu won't control SDL (Desktop) display properly.</p><p><span style="font-family: courier;"> #include "lvgl_native_drivers.h"</span></p><p><span style="font-family: courier;"> #define USE_SDL 1</span></p><p><span style="font-family: courier;"> #define SDL_HOR_RES CONFIG_SDL_HOR_RES</span></p><p><span style="font-family: courier;"> #define SDL_VER_RES CONFIG_SDL_VER_RES</span></p>Since our native environment will need to use SDL, we should also add SDL2 to lib, there are different libraries for different environments, such as Windows and Linux.<div>In Windows, we'll need to download <a href="https://www.libsdl.org/release/SDL2-devel-2.0.22-mingw.zip">SDL for mingw</a>, extract it into our lib folder and copy <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/lib/SDL2-2.0.22/library.json">library.json</a> from this project to your SDL library</div><div><br /><div>In Ubuntu its as simple as</div><div><span style="font-family: courier;">apt-get install libsdl2-2.0 libsdl2-dev</span></div><div><span style="font-family: courier;"><br /></span></div><div><div style="direction: ltr;">Lastly we need to add our native drivers configuration script <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/scripts/run_lvgl_native_drivers_kconfig.py">run_lvgl_native_drivers_kconfig.py</a> and configure it with <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/platformio.ini#L51">custom_lvgl_native_drivers_kconfig_save_settings</a> and <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/platformio.ini#L52">custom_lvgl_native_drivers_kconfig_output_header</a> configuration keys</div><div><h3 style="text-align: left;">ESP32 Hardware Display Drivers</h3><p>Now that we have our desktop setup, we also want our hardware setup so we can flash our device and see how our design looks on the real hardware.</p><p>Please note that the drivers are not always configured ideally, if the colors seems a bit off, you should read the datasheet and make sure everything is configured properly.</p><p>Lets start by extracting the latest source from <a href="https://github.com/lvgl/lvgl_esp32_drivers">https://github.com/lvgl/lvgl_esp32_drivers</a> into <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/tree/master/lib/lvgl_esp32_drivers">lib/lvgl_esp32_drivers</a> and copy the <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/master/lib/lvgl_esp32_drivers/library.json">library.json</a> from this project</p><p>Some drivers are not working properly with PlatformIO's scons configuration and needs to be enabled/disabled on a per-file basis, you should look in <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/master/lib/lvgl_esp32_drivers/library.json">library.json</a> as an example.</p><p>Another thing we want to tell our library.json is which framework it should work with, for example, in our setup we have native and esp32 environments and the esp32 drivers should not be compiled on the native environment since none of Espressif's libraries exist or even needed for desktops.</p><p>Then we'll modify <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/master/lib/lvgl_esp32_drivers/lvgl_helpers.c">lvgl_helper.c</a> to include "<a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/lib/lvgl_esp32_drivers/lvgl_helpers.c#L10">lv_conf.h</a>" right after "sdkconfig.h", the vanilla setup assumes your lvgl is part of esp32 components which can make desktop configuration a problem.</p><p><span style="font-family: courier;"> #include "sdkconfig.h"</span></p><p><span style="font-family: courier;"> #include "lv_conf.h"</span></p><p>Then we need to add lvgl kconfig script (<a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/master/scripts/run_lvgl_kconfig.py">run_lvgl_kconfig.py</a>) and set its configuration <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/platformio.ini#L33">custom_lvgl_kconfig_save_settings</a>, <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/platformio.ini#L34">custom_lvgl_kconfig_output_header</a>, <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/platformio.ini#L35">custom_lvgl_kconfig_include_headers</a> configuration sections to each relevant environment in platformio.ini</p><p>And add lvgl esp32 drivers kconfig script (<a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/master/scripts/run_lvgl_esp32_drivers_kconfig.py">run_lvgl_esp32_drivers_kconfig.py</a>) and <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/platformio.ini#L37">custom_lvgl_esp32_drivers_kconfig_save_settings</a>, <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/platformio.ini#L38">custom_lvgl_esp32_drivers_kconfig_output_header </a>configuration section to each relevant environment in platformio.ini</p><p>These two scripts and their setting enables platformio.ini to use the target scripts for easy configuration. To see which scripts are installed for each environment:</p><div style="background-color: #1e1e1e; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div>> pio run --list-targets</div><div>Environment Group Name Title Description</div><div>------------- -------- -------------------------- --------------------------- -----------------------------------</div><div>native Custom lvgl-config lvgl-config Executes lvgl config</div><div>native Custom lvgl-esp32-drivers-config lvgl-esp32-drivers-config Executes lvgl esp32 drivers config</div><div>native Custom lvgl-native-drivers-config lvgl-native-drivers-config Executes lvgl native drivers config</div><br /><div>esp32 Custom lvgl-config lvgl-config Executes lvgl config</div><div>esp32 Custom lvgl-esp32-drivers-config lvgl-esp32-drivers-config Executes lvgl esp32 drivers config</div><div>esp32 Custom lvgl-native-drivers-config lvgl-native-drivers-config Executes lvgl native drivers config</div><div>esp32 Platform buildfs Build Filesystem Image</div><div>esp32 Platform erase Erase Flash</div><div>esp32 Platform menuconfig Run Menuconfig</div><div>esp32 Platform size Program Size Calculate program size</div><div>esp32 Platform upload Upload</div><div>esp32 Platform uploadfs Upload Filesystem Image</div><div>esp32 Platform uploadfsota Upload Filesystem Image OTA</div></div><p>Now that we've assigned each configuration output to a different folder under include, we should tell platformio to include headers from these folders so each environment will get a <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/platformio.ini#L27">different</a> <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/platformio.ini#L43">set</a> of configuration files.</p><h3 style="text-align: left;">LVGL Configuration</h3><p>LVGL uses configuration files separate from the driver configuration files to configure some aspects of it, such as fonts, widgets, colors and layouts, but we need to tell it where to take the configuration from.</p><p>We do it by adding these flags to <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/platformio.ini#L28-L30">build_flags</a> for relevant environments in platformio.ini:</p><p><span style="font-family: courier;"> -DLV_LVGL_H_INCLUDE_SIMPLE</span></p><p><span style="font-family: courier;"> -DLV_CONF_INCLUDE_SIMPLE</span></p><p><span style="font-family: courier;"> -DLV_CONF_PATH=lv_conf.h</span></p><h3 style="text-align: left;">Environment Hardware Abstraction</h3><p>There is a small library in the demo project called <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/tree/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/lib/lvgl_hal">lvgl_hal</a>, it contains the setup for the drivers, obviously different from native to ESP32, you may need to modify it to your environment / programming.</p><p style="direction: ltr;">So copy <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/tree/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/lib/lvgl_hal">lvgl_hal</a> to your lib folder.</p><h3 style="text-align: left;">Changing Configuration</h3><p>Lets start with ESP32 configuration, LVGL can run lean or resource intensive, larger buffers may help with rendering speed, caching images and data can also help, my usual setup is 240Mhz CPU speed and 80Mhz PSRAM speed. </p><p>To make these configuration, you'll need to modify ESP32 configuration by running:</p><p><span style="font-family: courier;">pio run -e esp32 -t menuconfig</span></p><p>We can configure ESP32 drivers:</p><p><span style="font-family: courier;">pio run -e esp32 -t lvgl-esp32-drivers-config</span></p><p><span style="font-family: courier;"></span></p><div class="separator" style="clear: both; text-align: center;"><span style="font-family: courier;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEiu8KgrRj_GVg-coCVma0Gz9xearIXKorN5-qVEGwQuTixHwiv_SgvnGg2dc9_vej1mm4nomXKmg6fj5LaZlJ6rRNTYHnjAICryjaIpPg5-GAkqrXZdr3aSQopyPNdChr-0p7ViN2fw85-ZW0xNvoyGzlkf2XZKvIOHf2rlxx1J3Q1fthMIZlK5bPeT" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="478" data-original-width="958" height="320" src="https://blogger.googleusercontent.com/img/a/AVvXsEiu8KgrRj_GVg-coCVma0Gz9xearIXKorN5-qVEGwQuTixHwiv_SgvnGg2dc9_vej1mm4nomXKmg6fj5LaZlJ6rRNTYHnjAICryjaIpPg5-GAkqrXZdr3aSQopyPNdChr-0p7ViN2fw85-ZW0xNvoyGzlkf2XZKvIOHf2rlxx1J3Q1fthMIZlK5bPeT=w640-h320" width="640" /></a></span></div><span style="font-family: courier;"><br /><br /></span><p></p><p></p><div class="separator" style="clear: both; text-align: center;"><br /></div>And native drivers, which at the time of this writing is only resolution:<p></p><p><span style="font-family: courier;">pio run -e native -t lvgl-native-drivers-config</span></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEi8nh3y5ZzVStWw2YJmBApauxcgguhAhsTu23aH2j9fFuDT8vW01cKuQEj4ETf3LXPGPzpK0mcR9CoEJD4KV_DQZcYsuEHddUqtddwlbqbGKY4wRGHjUZRTgJ9HFLWryHlpWdo_IYmEuvVrp_xJciAFj4U3-uBhgWlyQYp7trjZWjX6iYUp3qOOg2wk" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="478" data-original-width="959" height="318" src="https://blogger.googleusercontent.com/img/a/AVvXsEi8nh3y5ZzVStWw2YJmBApauxcgguhAhsTu23aH2j9fFuDT8vW01cKuQEj4ETf3LXPGPzpK0mcR9CoEJD4KV_DQZcYsuEHddUqtddwlbqbGKY4wRGHjUZRTgJ9HFLWryHlpWdo_IYmEuvVrp_xJciAFj4U3-uBhgWlyQYp7trjZWjX6iYUp3qOOg2wk=w640-h318" width="640" /></a></div><br /><br /><p></p><p>And lastly we'll want to configure our lvgl, using the same configuration for both desktop and embedded can help you to find bugs quicker.</p><p><span style="font-family: courier;">pio run -e esp32 -t lvgl-config</span></p><p><span style="font-family: courier;">pio run -e native -t lvgl-config</span></p><p><span style="font-family: courier;"></span></p><div class="separator" style="clear: both; text-align: center;"><span style="font-family: courier;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjOfO21PIck9k_ASq5Kz3YqC9hHoYJBKaHuhXeipzmp5AoQofmOTUUYCdFa0zqtdpcKVk25GR7BUqE1iBvwgc0YHG-41AOFuwhOk4o9LLGA5tVvM-7I-4INfPHT97MO516RyLwVgY1dGC9IGH82AnKMf1bEAgKRlYUPMq55ItFfuyY018msihcH1QR9" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="479" data-original-width="958" height="320" src="https://blogger.googleusercontent.com/img/a/AVvXsEjOfO21PIck9k_ASq5Kz3YqC9hHoYJBKaHuhXeipzmp5AoQofmOTUUYCdFa0zqtdpcKVk25GR7BUqE1iBvwgc0YHG-41AOFuwhOk4o9LLGA5tVvM-7I-4INfPHT97MO516RyLwVgY1dGC9IGH82AnKMf1bEAgKRlYUPMq55ItFfuyY018msihcH1QR9=w640-h320" width="640" /></a></span></div><span style="font-family: courier;"><br /><br /><br /></span><p></p><h3 style="text-align: left;">Runner</h3><p>The <a href="https://github.com/drorgl/esp32-lvgl-configuration-demo/blob/aa0b45ae647ddb2d6b8fa086959e59b7b9693447/lib/runner/runner.h">runner</a> library is intended as an abstraction of the main function, on a desktop its int main(argc,argv), on ESP32 its appmain() and on Arduino its setup() and loop(), instead of writing the same ifdefs everywhere, just copy the runner library, include it in your main file and use it:</p><p><span style="font-family: courier;">MAIN(){</span></p><p><span style="font-family: courier;">}</span></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEigi9KrnIxvHbh65V248JtAtQ1R9IAGKkf1cYnD3VJU3_oA-Wrs_uYm0V2NyUAfFI_tEzu_BcKnmxaTg7XYKNpozSj9mVwAYVN39DVePPECxpqUIFFsTBlK7cVwhYO006L_1M8MpxvkcI-yo5BVR68rVZq3j89_NP5wvojNHp3Krqma0sDqncHHRa54" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="347" data-original-width="482" height="460" src="https://blogger.googleusercontent.com/img/a/AVvXsEigi9KrnIxvHbh65V248JtAtQ1R9IAGKkf1cYnD3VJU3_oA-Wrs_uYm0V2NyUAfFI_tEzu_BcKnmxaTg7XYKNpozSj9mVwAYVN39DVePPECxpqUIFFsTBlK7cVwhYO006L_1M8MpxvkcI-yo5BVR68rVZq3j89_NP5wvojNHp3Krqma0sDqncHHRa54=w640-h460" width="640" /></a></div><br />Unfortunately my ILI9488 is not configured properly (<a href="https://blog.drorgluska.com/2022/07/using-ili9488.html">more on that later</a>)<p></p></div></div></div></div><div><div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEiE9ZHjkLNShjXxb-lEhlZ4DUAGVmsAbPu5wi7lssWWT12BxcOQvozEf7bzUFdXgJNb5jDpstZarbA2bJPZUerqHI9qZ4qkQPVkeSqkgp7YIum3Ak_CZw-k67e6nQqDEWGus1etUnUJB3YM4wM1suFPGaOvrrKQ4eQ5HiukuSri3263ho5sHCiYC08Q" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="464" data-original-width="650" height="456" src="https://blogger.googleusercontent.com/img/a/AVvXsEiE9ZHjkLNShjXxb-lEhlZ4DUAGVmsAbPu5wi7lssWWT12BxcOQvozEf7bzUFdXgJNb5jDpstZarbA2bJPZUerqHI9qZ4qkQPVkeSqkgp7YIum3Ak_CZw-k67e6nQqDEWGus1etUnUJB3YM4wM1suFPGaOvrrKQ4eQ5HiukuSri3263ho5sHCiYC08Q=w640-h456" width="640" /></a></div><br /><br /></div>If you're looking for a solution to the ILI9488 configuration issue, you can read about it <a href="https://blog.drorgluska.com/2022/07/using-ili9488.html">here</a>.</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjnwvSt1kCA7No5LlR7I8UhPJ9K9QNu2caZIbsPj_UTMrRz0HVT2CbsXqaANbVkWW0As-gK7I6g6n8YUFWOF5jr5lCKNajmkRS0hGvVUZaH4h5YUNo6zgeC8GGypKjIMK0GOfN-lP2yDy0SbAUMMF6MxiDBKkVcZTDnhqgc6NmGoLfFRHKtXJ7DI1MO=s640" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="384" data-original-width="640" height="384" src="https://blogger.googleusercontent.com/img/a/AVvXsEjnwvSt1kCA7No5LlR7I8UhPJ9K9QNu2caZIbsPj_UTMrRz0HVT2CbsXqaANbVkWW0As-gK7I6g6n8YUFWOF5jr5lCKNajmkRS0hGvVUZaH4h5YUNo6zgeC8GGypKjIMK0GOfN-lP2yDy0SbAUMMF6MxiDBKkVcZTDnhqgc6NmGoLfFRHKtXJ7DI1MO=w640-h384" width="640" /></a></div><br /><div><br /></div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3077621944057198788.post-91327159704195510502022-07-12T08:00:00.001-07:002023-04-13T06:19:41.150-07:00RISC-V Linux on ESP32<p>I've been playing with the idea of running linux on ESP32 since the first days I've met its more robust module, the WROVER-B, on paper it seem possible since its a dual core 240Mhz and has 16MB flash and 8MB RAM, compared to our antique machines that could run linux, it seems like a beast.</p><p>Doing some research on it, I've understood that its MMU is insufficient for running Linux on it. during the past few years I've been looking into it to see if anyone else found the time to implement it and eventually I've decided its going to be a good opportunity to learn a bit more about RISCV and Buildroot. Two subjects I've been putting off for longer than I'd like to admit.</p><p>I've decided to start with something rather to write it all from scratch, which I didn't have time or energy to do for this project, I've looked into <a href="https://wiki.qemu.org/Documentation/Platforms/RISCV">QEMU emulation for RISCV</a> but taking this project apart and getting only a few components out of it to run on an embedded system seemed like too much work. Eventually I've found out about Fabrice Bellard's <a href="https://bellard.org/tinyemu/">TinyEMU</a> (<a href="https://bellard.org/jslinux/vm.html?cpu=riscv64&url=https://bellard.org/jslinux/buildroot-riscv64.cfg&mem=256">demo</a>).</p><p></p><div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjQkEdM4GAPNJGcxxyzokiHSIszX_hnyz7MsTOIRsRiOt4dLqCxcH_UY6QWm5ct6FHT9z33PhkgaEXNONtHVFt-y5eDSvcI4rk6nQa130niNhsVeFF-zahimTvrTHV6YGjPnVh_3G_tyoq1QuxbdgnBtcBvu2Nbd6oKHIpSWMRP9s0vN9_FE9KKS0sp" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="966" data-original-width="1509" height="410" src="https://blogger.googleusercontent.com/img/a/AVvXsEjQkEdM4GAPNJGcxxyzokiHSIszX_hnyz7MsTOIRsRiOt4dLqCxcH_UY6QWm5ct6FHT9z33PhkgaEXNONtHVFt-y5eDSvcI4rk6nQa130niNhsVeFF-zahimTvrTHV6YGjPnVh_3G_tyoq1QuxbdgnBtcBvu2Nbd6oKHIpSWMRP9s0vN9_FE9KKS0sp=w640-h410" width="640" /></a></div><br /><br /></div><h3 style="text-align: left;">RISC-V</h3><p><a href="https://en.wikipedia.org/wiki/RISC-V">RISC-V</a> is the new ISA kid in the block, well, not really a kid and not really new, but it becomes more and more popular, Espressif got out the <a href="https://www.espressif.com/en/news/ESP32_C3">ESP32-C3</a> at 2020.</p><h3 style="text-align: left;">Previous Successes</h3><p>Max Filippov <a href="https://github.com/jcmvbkbc/linux-xtensa/tree/xtensa-5.19-esp32">patched</a> the kernel to support ESP32 back at <a href="https://www.reddit.com/r/esp32/comments/dtlj7n/booting_linux_on_esp32_realtime_video/">2019</a>, I'm pretty sure it runs a lot faster since its not an emulation.</p><p style="direction: ltr;">Li XiongHui wrote the <a href="https://github.com/juiceRv/JuiceVm">juiceVM</a> which implemented RISCV ISA and runs on ESP32, he wrote about it on <a href="https://whycan.com/p_66202.html">whycan</a> and <a href="https://www.reddit.com/r/esp32/comments/om106r/boot_linux_500_on_esp32/">reddit</a> and has <a href="https://www.youtube.com/watch?v=w9UlZIYZtwI">video of it booting on YouTube</a>, the video does state x30 speedup, which means the system booted in about 6 hours. Li never released the source code so the only improvements that can be done is by him and judging from my own life, you never have enough time for these things.</p><h3 style="text-align: left;">TinyEMU</h3><p>"TinyEMU is a system emulator for the RISC-V and x86 architectures. Its purpose is to be small and simple while being complete."</p><p>Looking at its <a href="https://bellard.org/tinyemu/tinyemu-2019-12-21.tar.gz">source code</a>, seemed like the project went from mission impossible (with my current resources) to mission possible. Oh the naiveté.</p><h3 style="text-align: left;">ESP32</h3><p>The ESP32 is a dual core 240Mhz MCU, it was released at September 2016 and its still one of the best value for money MCU you can get, one of its versions has 8MB or RAM and 16MB of FLASH. That amount of RAM on any of its competitors takes more than "I want it" to get it working, at the time it came out especially so. Espressif did an amazing job with esp-idf and one of their best features is listening to their customers and with the help of the maker community they've built an amazing framework.</p><h3 style="text-align: left;">TinyEMU on ESP32</h3><p>Will it even compile?</p><p>Apparently yes, making it compile was very easy, some tweaks here and there and a missing standard library and it was compiled perfectly. </p><p>But what can I do about memory? the ESP32 only has 8MB and half of it is not even accessible as a standard but rather bank switched with its own APIs (<a href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/system/himem.html">himem</a>).</p><p>My thinking at the time was that it doesn't matter so much since the kernel will probably not need so much memory once it starts, for example, if I don't access files, the memory holding the file system and the file system functions will be left alone other than a periodic flush.</p><p>So something like a swap file will probably be good enough for this experiment, right?</p><p>Not so fast.</p><p>First, I had to find out that the standard way of accessing the SD card is not fast enough at around 150k per second. <a href="https://blog.drorgluska.com/2022/06/esp32-sd-card-optimization.html">So I went to a journey to find all the bottlenecks</a>.</p><p>Then I've discovered that 3MB for the virtual pages is too slow due to the <a href="https://www.reddit.com/r/esp32/comments/ezs5sg/how_slow_is_psram_vs_sram_anyone_have/">PSRAM</a>, using 80Mhz only improved a bit, so I went on a journey to find the fastest search trees, I've tried <a href="https://en.wikipedia.org/wiki/Splay_tree">splay tree</a> and eventually rested on <a href="https://en.wikipedia.org/wiki/AVL_tree">AVL tree</a> which was good enough.</p><p>But I could squeeze more out of the ESP32, I've implemented a rudimentary direct-mapped-cache so most memory accesses won't even search for their page (Professor Luis Ceze has a <a href="https://courses.cs.washington.edu/courses/cse378/09wi/lectures/lec15.pdf">great presentation on the subject</a>).</p><p>I was still not happy enough, my SD card is slow and no matter what I did, it slowed things down. I've measured how many page faults I had and decided that dirty pages should only be written once they are abandoned, so my <a href="https://en.wikipedia.org/wiki/Cache_replacement_policies#Least_recently_used_(LRU)">LRU cache</a> pushed dirty pages into himem and only when these himem pages reclaimed they got pushed to the page file.</p><p>At that point I was content enough, my kernel booted in 1:35 minutes. </p><p><span style="color: red;">UPDATE 2023-04-13: Improved Speed by 40%</span></p><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="266" src="https://www.youtube.com/embed/f3a3xeTRj_A" width="320" youtube-src-id="f3a3xeTRj_A"></iframe></div><p><br /></p><h3 style="text-align: left;">Buildroot</h3><p>"<a href="https://buildroot.org/">Buildroot</a> is a simple, efficient and easy-to-use tool to generate embedded Linux systems through cross-compilation."</p><p>I've always wanted to learn how the big guys do it, how embedded cameras, kiosks and perhaps even satellites gets their OS build without all the bloat and package managers.</p><p>So once I got my basic emulator working (without all the optimizations), it was time to start learning buildroot. </p><p>Apparently its simple, you just download the archive, work through a few menus, read some documentation, modify the rootfs with overlays and you're done. issue a make command and you have your kernel and your rootfs.</p><p>In the old days (or so I've heard), you've had to build RISC-V toolchain and patch the kernel to get things going, These days buildroot comes with a precompiled toolchain from <a href="https://bootlin.com/">bootlin</a>, so the whole experience was fun and easy to learn.</p><h3 style="text-align: left;">Thoughts for the future</h3><p>Using the emulator to run embedded RISC-V code, implement <a href="https://github.com/sifive/example-gpio">SiFive GPIO</a> and provide more flexible VM for running code on multiple embedded platforms. Communicating over the console is implemented in <a href="https://bellard.org/tinyemu/diskimage-linux-riscv-2018-09-23.tar.gz">TinyEMU patches</a>, or see <a href="https://stackoverflow.com/questions/57228650/how-to-print-to-console-in-linux-from-risc-v-assembly">here</a> or through <a href="https://developer.ibm.com/articles/l-virtio/">virtio</a>.</p><h3 style="text-align: left;">Closing</h3><p>This was a wonderful journey of learning, I've learned more than I wanted about:</p><p></p><ul style="text-align: left;"><li>file systems (thank you <a href="http://elm-chan.org/fsw/ff/00index_e.html">fatfs</a>)</li><li><a href="https://buildroot.org/">buildroot</a></li><li><a href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/system/himem.html">himem</a></li><li>RISC-V Device Tree and <a href="file:///C:/Users/Dror/Downloads/devicetree-specification-v0.4-rc1.pdf">FDT</a> (Flattened Device Tree)</li><li><a href="https://github.com/riscv-software-src/riscv-pk">Proxy Kernel and BBL</a> (Berkeley Boot Loader)</li><li>SBI (Supervisor Binary Interface)</li><li>hvc0</li><li>virtio</li><li>memory layouts and caches</li><li>kernel memory and size optimizations (<a href="https://elinux.org/Kernel_Size_Tuning_Guide">here</a>, <a href="https://elinux.org/System_Size">here</a> and <a href="https://elinux.org/Kernel_Size_Tuning_Guide_Config_Option_Impact">here</a>).</li></ul><div>You can find the fruits of this labor at:</div><div><ul style="text-align: left;"><li>ESP32 VMM TinyEMU Port <a href="https://github.com/drorgl/esp32-tinyemu">https://github.com/drorgl/esp32-tinyemu</a></li><li>Buildroot for RISC-V 32 Getting Started <a href="https://github.com/drorgl/buildroot-tinyemu">https://github.com/drorgl/buildroot-tinyemu</a></li></ul></div><div><br /></div>Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-3077621944057198788.post-22038566387362463332022-06-29T04:13:00.003-07:002022-07-05T12:39:58.932-07:00ESP32 SD Card Optimization<p> Reading and writing SD Cards with ESP32 should be simple, however, the amount of moving parts in esp-idf makes that a complicated task, first to understand and then to optimize.</p><p>Lets disect the stack:</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhkePXIqplO0ELjPj-hNhx-ZSxor5hV0UFLdzu9HWBta-vpC0p3Eorm6pLbfuHRhs-J8TTnclIxA3UGX4AprUP2BEeNBGF2GIIAuOhpJDO8cSrI947DYnCN0FSidOQ0zDupigtLBAvRSldi63nXJAac0CrJw5um7QTN701Kau-z_VUHcLRkEwIOQbaT" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="479" data-original-width="1173" height="261" src="https://blogger.googleusercontent.com/img/a/AVvXsEhkePXIqplO0ELjPj-hNhx-ZSxor5hV0UFLdzu9HWBta-vpC0p3Eorm6pLbfuHRhs-J8TTnclIxA3UGX4AprUP2BEeNBGF2GIIAuOhpJDO8cSrI947DYnCN0FSidOQ0zDupigtLBAvRSldi63nXJAac0CrJw5um7QTN701Kau-z_VUHcLRkEwIOQbaT=w640-h261" width="640" /></a></div><br /><h3 style="text-align: left;">File Operations</h3><p></p><p>Working with files should not be new to you, however, there are some differences between ESP32 and working on a full pledged OS, these differences can affect how the product will behave.</p><p>On OS the buffered and unbuffered behaves very similar, in the end its the OS's job to buffer the disk operations so the system and applications are more responsive</p><p>However, due to resource limitations, on an MCU there is no such buffering other than the Standard library buffering and disk operations requirements.</p><p>In turn, this means you'll need to use the API in a manner suitable for these limitations.</p><p>Its probably best to avoid the buffered APIs if you don't plan to write unstructured data, such as logs and textual information, the unbuffered calls will pass the read and write requests as-is, so you can have as much control as you need without modifying any of the libraries.</p><h3 style="text-align: left;">Standard Library</h3><p>The standard library is responsible for translating the buffered and unbuffered calls to the next layer, so eventually these functions will <a href="https://github1s.com/bminor/newlib/blob/HEAD/newlib/libc/include/reent.h#L139-L158">call</a> the platform's <a href="https://github1s.com/espressif/esp-idf/blob/HEAD/components/vfs/vfs.c#L1240-L1305">implementation</a>. </p><p>Specifically in ESP32, newlib was compiled with 128 bytes buffer, this creates very inefficient calls to the file system since each fread/fwrite will split each call to 128 bytes which is significantly less than the sector size.</p><p>You may ease some of this inefficiency by increasing the buffer size for the buffered calls:</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; text-align: left; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #008800; font-weight: bold;">if</span>(setvbuf(file, <span style="color: #007020;">NULL</span>, _IOFBF, <span style="color: #0000dd; font-weight: bold;">4096</span>) <span style="color: #333333;">!=</span> <span style="color: #0000dd; font-weight: bold;">0</span>) {
<span style="color: #888888;">//handle error</span>
}
</pre></div>
<h3 style="text-align: left;">File Stream Adapter</h3><p>The file stream adapters, or the <a href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/storage/vfs.html">VFS</a> enables mounting multiple file systems under different root paths, so in essence each root path refers to a different mount.</p><p>When opening a file through the standard library, a new file descriptor is created in the file system driver (currently only FATFS), it then gets translated into VFS file descriptor and returned to the standard library.</p><p>When writing to the file descriptor, the process reverses and the file descriptor is <a href="https://github1s.com/espressif/esp-idf/blob/HEAD/components/vfs/vfs.c#L429-L436">translated back</a> to the filesystem's file descriptor.</p><h3 style="text-align: left;">File System</h3><p>Currently only <a href="http://elm-chan.org/fsw/ff/00index_e.html">FATFS</a> is implemented as file system (<a href="https://en.wikipedia.org/wiki/Design_of_the_FAT_file_system">FAT</a>) in ESP32, however, using other file systems such as <a href="https://github.com/littlefs-project/littlefs">LittleFS</a> requires very little effort.</p><p>The file system is responsible for opening, reading, writing and other file system operations to read and persist from a physical medium.</p><p>For example, when opening a file, the FATFS will go to the root directory and traverse subdirectories until it finds where the file is located while reading the file allocation table, all of that is done with raw access to the underlying storage, i.e. read sector x,y,z.</p><p>FATFS was designed for resource constrained systems, so it will not buffer more than it must and will read and write as little as possible to achieve the task it needs, even if it means inefficient calls to the storage. for example, writing unaligned data will result in read sector, update part of it, write it back.</p><p>This creates <a href="http://elm-chan.org/fsw/ff/doc/appnote.html#fs1">inefficiencies</a> when working with unaligned random accesses to files since it requires FATFS to "waste" time getting the data it needs in blocks and not using part of that data.</p><p>But these inefficiencies can be reduced by making sure all your random access to files is aligned to the sector sizes. </p><p>More issues you may encounter is when your reads and writes cross the cluster boundaries, each cluster can be hosted on a different place in the block device which will require two separate calls to the underlying storage.</p><p>To reduce the amount of waste these calls create, FATFS implements <a href="http://elm-chan.org/fsw/ff/doc/config.html#use_fastseek">fast seek</a> (and in <a href="https://docs.espressif.com/projects/esp-idf/en/stable/esp32/api-reference/kconfig.html#config-fatfs-use-fastseek">esp-idf</a>), which caches the <a href="http://elm-chan.org/fsw/ff/doc/lseek.html">cluster link map table</a>, however this works only on read only files or preallocated write files.</p><p>Also, if you plan to use your design as a product, keep in mind that Windows formats SD cards as <a href="http://elm-chan.org/fsw/ff/doc/config.html#fs_exfat">EXFAT</a> which is turned off by default in esp-idf. this can cause some usability issues to your customers and if you choose you can override that configuration. There are two ways of doing it, each with its own advantages and drawbacks:</p><p>1. copy the component/fatfs to your project's root directory under component/fatfs and rebuild the project.</p><p>2. <a href="https://docs.platformio.org/en/latest/scripting/examples/override_package_files.html">Override package files</a> and enable it in your ffconf.h.</p><h3 style="text-align: left;">File System Adapter</h3><p>The file system adapter's job is to read and write data from a block device, optionally with <a href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/storage/wear-levelling.html">wear leveling</a> driver in between, since SD card implement wear levelling internally, this layer is not used when working with SD cards.</p><p>Most block devices such as SD cards and eMMC implement their storage in blocks, usually 512 or 1024 bytes per block (or sector), which means that the FATFS layer must speak in sectors, so if it needs to update 2048 bytes, it will do it in 512 bytes chunks in most cases.</p><h3 style="text-align: left;">Hardware Drivers</h3><p>The hardware driver's job is to read and write sectors to the underlying storage media, currently <a href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/peripherals/sdspi_host.html">SD SPI</a> and <a href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/storage/sdmmc.html">SDMMC</a> 1 and 4 bits are impended.</p><p>Recommendations:</p><p></p><ul style="text-align: left;"><li>Configure your hardware drivers as fast as possible, <a href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/peripherals/sdspi_host.html#_CPPv423sdspi_host_set_card_clk18sdspi_dev_handle_t8uint32_t">SPI </a>and <a href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/peripherals/sdspi_host.html#_CPPv423sdspi_host_set_card_clk18sdspi_dev_handle_t8uint32_t">SDMMC</a>. </li><li>Prefer Sequential over Random reads and writes, SD cards were designed for sequential access.</li><li>If you must use random access, avoid using the buffering APIs </li><li>If you must use random read / write, prefer aligned read and write in sector size chunks.</li><li>If you must use buffering APIs, increase the buffer to at least the sector size in sector size increments (your average buffer size aligned to 512 or 1024 )</li><li>Use FATFS <a href="http://elm-chan.org/fsw/ff/doc/lseek.html">cluster cache</a> (or fast seek) if applicable, please note that to reduce incompatibilities the cache is disabled for writeable files in <a href="https://github.com/espressif/esp-idf/blob/c2ccc383dae2a47c2c2dc8c7ad78175a3fd11361/components/fatfs/vfs/vfs_fat.c#L350">vfs_fat.c</a>, however, you can override it if you know what you're doing.</li><li>Enable EXFAT if its a customer facing product</li><li>Prefer SDMMC over SPI, though it requires pull ups and you might need to adjust your design due to <a href="https://docs.espressif.com/projects/esptool/en/latest/esp32/advanced-topics/boot-mode-selection.html?highlight=gpio12#select-bootloader-mode">GPIO bootloader modes</a>, however, I was able to use high speed sdmmc by activating the internal pullup on GPIO 12 programmatically and it was enough.</li></ul><p></p><p>This has been a learning experience, hopefully my lessons will ease your learning experience.</p><p>Relevant github repo: <a href="https://github.com/drorgl/esp32-sd-card-benchmarks">https://github.com/drorgl/esp32-sd-card-benchmarks</a></p><p><br /></p>Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-3077621944057198788.post-23952350363509708782022-06-08T13:10:00.002-07:002022-06-08T13:11:10.712-07:00PlatformIO Menu Configuration Integration<p>One of the hurdles when building a custom library or software is how to allow the user to configure it. We can have a header file with comments and tell the user how to configure it but if multiple defines start to have dependencies or the value is more than true or false, there's a chance the user will get this wrong and the a whole class of issues is going to be opened and time is going to be wasted.</p><p></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEg0iYPd2-yOCjHxFWeOotqlDQBHoTFjuOzfcyWMklnhl9fdXtdckrD2VZPKQQ0R5zrwH13y0MOWznFNnPLj4xxSPmRNkWA1ppRjQtMsiB3f5eTwrPNC9AH_dk5Nkpiu1-ufw3Y9ek1jwsFVyeuBzd1h5yYHbxkQugjl4XlHlUyMZr-x_2eqxXZGzZUw" style="margin-left: auto; margin-right: auto;"><img alt="" data-original-height="750" data-original-width="1000" height="480" src="https://blogger.googleusercontent.com/img/a/AVvXsEg0iYPd2-yOCjHxFWeOotqlDQBHoTFjuOzfcyWMklnhl9fdXtdckrD2VZPKQQ0R5zrwH13y0MOWznFNnPLj4xxSPmRNkWA1ppRjQtMsiB3f5eTwrPNC9AH_dk5Nkpiu1-ufw3Y9ek1jwsFVyeuBzd1h5yYHbxkQugjl4XlHlUyMZr-x_2eqxXZGzZUw=w640-h480" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><a href="https://www.pexels.com/photo/abstract-art-circle-clockwork-414579/">source</a></td></tr></tbody></table><br /><br /><p></p><p>The linux kernel solved it with the Kconfig utility, which was reimplemented using the <a href="https://pypi.org/project/kconfiglib/">kconfiglib</a>. fortunately its easier to install and use and the integration requires only menuconfig and genconfig utilities.</p><p>To implement it in our software we need to do the following:</p><h3 style="text-align: left;">KConfig file</h3><p>write your <a href="https://www.kernel.org/doc/html/latest/kbuild/kconfig-language.html">configuration file</a>, here's an example with one boolean and one string configuration values:</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; text-align: left; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #008800; font-weight: bold;">mainmenu</span> <span style="background-color: #fff0f0;">"Sample configurable project using Kconfig"</span>
<span style="color: #008800; font-weight: bold;">config</span> FOO
<span style="color: #007020;">bool</span> <span style="background-color: #fff0f0;">"Foo module"</span>
<span style="color: #008800; font-weight: bold;">help</span>
<span style="color: #dd4422;"> The infamous Foo module</span>
<span style="color: #008800; font-weight: bold;">config</span> BAR
<span style="color: #007020;">string</span> <span style="background-color: #fff0f0;">"Bar Value"</span>
<span style="color: #008800; font-weight: bold;">help</span>
<span style="color: #dd4422;"> The Bar Value</span>
</pre></div>
<h3 style="text-align: left;">platformio.ini</h3><p>Once we have the configuration file we need to tell menuconfig and genconfig where the file is, where to save the configuration settings and where to generate the header file with the configuration values.</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #888888;">; menuconfig runner</span>
<span style="color: #0000cc;">extra_scripts</span> <span style="color: #333333;">=</span> <span style="background-color: #fff0f0;"></span>
<span style="background-color: #fff0f0;"> scripts/run_menuconfig.py</span>
<span style="color: #888888;">; path to Kconfig file</span>
<span style="color: #0000cc;">custom_kconfig_config</span> <span style="color: #333333;">=</span> <span style="background-color: #fff0f0;">scripts/configs/Kconfig</span>
<span style="color: #888888;">; configuration settings file</span>
<span style="color: #0000cc;">custom_kconfig_save_settings</span> <span style="color: #333333;">=</span> <span style="background-color: #fff0f0;">include/custom_config.config</span>
<span style="color: #888888;">; configuration settings file and header file header comment</span>
<span style="color: #0000cc;">custom_kconfig_comment_header</span> <span style="color: #333333;">=</span><span style="background-color: #fff0f0;"></span>
<span style="background-color: #fff0f0;"> File Header</span>
<span style="background-color: #fff0f0;"> hello world</span>
<span style="color: #888888;">; output configuration header file</span>
<span style="color: #0000cc;">custom_kconfig_output_header</span> <span style="color: #333333;">=</span> <span style="background-color: #fff0f0;">include/custom_config.h</span>
</pre></div>
<p>All we need to do now is execute the <a href="https://github.com/drorgl/platformio-kconfig/blob/da6bdf8751fa9496c89cbe5e08d9d0da84ada2b5/scripts/run_menuconfig.py">runner</a></p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">pio run -t kconfig
</pre></div>
<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEiXjkhFFbqNZyU8h2qXnf6PuJc80Ksbs3XldHAivRJMIFT2sg22fBie0uoS0urbf9pHUb8Jx19iQvI_Wm8vj3iX-43jyHx0sGOKLlGZ7G9mVfkTSzBOiOAiBHak-T08HspgOFxmby3LGRo1e-h-Xzi8cQ1VqMW8HyI6vG954PFp9KG1xDJ8bW3JH9Sc" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="430" data-original-width="655" height="420" src="https://blogger.googleusercontent.com/img/a/AVvXsEiXjkhFFbqNZyU8h2qXnf6PuJc80Ksbs3XldHAivRJMIFT2sg22fBie0uoS0urbf9pHUb8Jx19iQvI_Wm8vj3iX-43jyHx0sGOKLlGZ7G9mVfkTSzBOiOAiBHak-T08HspgOFxmby3LGRo1e-h-Xzi8cQ1VqMW8HyI6vG954PFp9KG1xDJ8bW3JH9Sc=w640-h420" width="640" /></a></div><br />when we quit and save, menuconfig will generate the configuration setting file to custom_kconfig_save_settings:<p></p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #888888;">#File Header</span>
<span style="color: #888888;">#hello world</span>
CONFIG_FOO<span style="color: #333333;">=</span>y
CONFIG_BAR<span style="color: #333333;">=</span><span style="background-color: #fff0f0;">"hello world"</span>
</pre></div>
<p>and then execute genconfig and generate the header file to custom_kconfig_output_header:</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #888888;">// File Header</span>
<span style="color: #888888;">// hello world</span>
<span style="color: #557799;">#define CONFIG_FOO 1</span>
<span style="color: #557799;">#define CONFIG_BAR "hello world"</span>
</pre></div>
<p>and lastly, we can use the header file like any other header file and get our configuration from it:</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #557799;">#include <custom_config.h></span>
MAIN()
{
printf(<span style="background-color: #fff0f0;">"Program started!</span><span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\r\n</span><span style="background-color: #fff0f0;">"</span>);
printf(<span style="background-color: #fff0f0;">"Bar Value %s</span><span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\r\n</span><span style="background-color: #fff0f0;">"</span>, CONFIG_BAR);
}
</pre></div>
<p><br /></p>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3077621944057198788.post-21120645912291857502022-06-07T23:25:00.002-07:002022-06-07T23:27:15.752-07:00PlatformIO Dynamic Code Analysis and Coverage<p>Like static code analysis, dynamic code analysis provides another level of coding errors detection and can help diagnose memory access issues such as use-after-free or uninitialized read and even memory leaks and memory write overruns.</p><p></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEi00GkyRVh-jo52eIxlhd-AINvom3-ZTi-ubJNn-l2YExD6oyooBIQ_Q3O2Hr7fGTgDmwQNYALX_froHQg8Az8LmDk2l87_xCDQf-9Tgu84j-IEOF7HZI_dcijWJoNHRgQpnLsa1-FLbCLrkPuy5bzf8CUq5fBUDlZ5_xoE8vHBr8zsSWZP-esMZKBu" style="margin-left: auto; margin-right: auto;"><img alt="" data-original-height="750" data-original-width="1125" height="426" src="https://blogger.googleusercontent.com/img/a/AVvXsEi00GkyRVh-jo52eIxlhd-AINvom3-ZTi-ubJNn-l2YExD6oyooBIQ_Q3O2Hr7fGTgDmwQNYALX_froHQg8Az8LmDk2l87_xCDQf-9Tgu84j-IEOF7HZI_dcijWJoNHRgQpnLsa1-FLbCLrkPuy5bzf8CUq5fBUDlZ5_xoE8vHBr8zsSWZP-esMZKBu=w640-h426" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><a href="https://www.pexels.com/photo/photo-of-gray-faucet-2339722/">source</a></td></tr></tbody></table><br /><br /><p></p><p>On top of that, we can write every line of code that has been used and later correlate that and say which lines were executed and which were not and that is the essence of code coverage. </p><h3 style="text-align: left;">Code Coverage</h3><p>A unit test is just code that executes your code, by executing your code and providing it with information it needs and checking its return values and side effects we can determine if your code does what it was designed to do.</p><p>Running unit tests is usually done by creating another executable or firmware that runs all your tests one by one and reports their status. </p><p>Eventually, if this executable writes which lines were executed, a reporting tool can later read that data and cross reference it with the source code to produce statistics and reports.</p><p>To make it work, we need to disable optimizations and compile debug symbols, otherwise we'll get bad results if any. </p><p>So first, we'll add the required flags to our environment in platformio.ini:</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #0000cc;">build_flags</span> <span style="color: #333333;">=</span> <span style="background-color: #fff0f0;">-ggdb -lgcov -O0 --coverage</span>
</pre></div>
<p>Now when we run tests, the coverage data will be written into the gcda and gcno files we can view it with a <a href="https://marketplace.visualstudio.com/items?itemName=JacquesLucke.gcov-viewer">VSCode Extension</a> directly:</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjowUMNfbmroAkmCSJs0twyU7UC80LOwtTeP-e5eP2aBbTqbqFxBl-dKFrwvPpqX1bXZi8IPOKbcannBs3Nm90mdLOpr6pZl_VG3aKqD0T8JNWHy72CObyzd6h3Kk-jR9_mA4N4zWR-9Rt6OW1YtB1RrK45p-JJYA1AEEAvZNDGJmoThM06a55FkgOW" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="359" data-original-width="710" height="325" src="https://blogger.googleusercontent.com/img/a/AVvXsEjowUMNfbmroAkmCSJs0twyU7UC80LOwtTeP-e5eP2aBbTqbqFxBl-dKFrwvPpqX1bXZi8IPOKbcannBs3Nm90mdLOpr6pZl_VG3aKqD0T8JNWHy72CObyzd6h3Kk-jR9_mA4N4zWR-9Rt6OW1YtB1RrK45p-JJYA1AEEAvZNDGJmoThM06a55FkgOW=w640-h325" width="640" /></a></div><br /><br /><p></p><p>But it might not be enough, we can also view it in command line or even in CI so well need to build a report from it, we'll use <a href="https://gcovr.com/">gcovr</a> do build a tracefile from it and since we have more than one test module, we'll want to merge all the tracefiles to a single report.</p><p>Lets start by adding our <a href="https://github.com/drorgl/esp32-test-development/blob/a862bf3f9e27131e2b7d0e0009d3c5af4096c6b3/dynamic-code-analysis/scripts/run_gcovr.py">runner</a> to platformio.ini:</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #0000cc;">extra_scripts</span> <span style="color: #333333;">=</span> <span style="background-color: #fff0f0;"></span>
<span style="background-color: #fff0f0;"> scripts/run_gcovr.py</span>
</pre></div>
<p>The runner hooks into the test executable generation and executes gcovr and generate a tracefile for each test executable.</p><p>When the tests are done, we can get the coverage info by running:</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">pio run -t gcovr
</pre></div>
<p>We can then browse the .reports/coverage.html</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEiFi1ixuuN6-YKmaHY7YP0lFq6sqeM4tXrHWASHOK0XRiqE4n8MnNJq2qve_AdXA7lvxtb-4vhnfXFO9wEzSIj41IqjuyDC4C3rn3nk42PpGu07nkCWl1sT9clbUClTjH-tq2B2y6ypsQjsORlKkOJl3WefwwNwL9LZM0KMpwTbX2D1zxSQRxkdd4NW" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="452" data-original-width="1181" height="245" src="https://blogger.googleusercontent.com/img/a/AVvXsEiFi1ixuuN6-YKmaHY7YP0lFq6sqeM4tXrHWASHOK0XRiqE4n8MnNJq2qve_AdXA7lvxtb-4vhnfXFO9wEzSIj41IqjuyDC4C3rn3nk42PpGu07nkCWl1sT9clbUClTjH-tq2B2y6ypsQjsORlKkOJl3WefwwNwL9LZM0KMpwTbX2D1zxSQRxkdd4NW=w640-h245" width="640" /></a></div><br />If you have other requirements from gcovr, there are many <a href="https://gcovr.com/en/stable/manpage.html">options</a> for the <a href="https://gcovr.com/en/stable/output/index.html">output</a> and you can use a <a href="https://gcovr.com/en/stable/guide/configuration.html">configuration file</a> to specify what you need.<p></p><p><br /></p><h3 style="text-align: left;">Dr. Memory</h3><p>By hooking into the PlatformIO testing mechanism and intercepting the compiled tests we can execute the tests under <a href="https://drmemory.org/">Dr. Memory</a>'s supervision and get a <a href="https://drmemory.org/page_types.html">report</a> of where exactly code was leaking or accessing memory it wasn't supposed to, it does require the developer to not leave dangling memory after tests so each malloc will need to be matched with free and every new will have to be delete(ed).</p><p>This step does not require any intervention from the user other than add <a href="https://github.com/drorgl/esp32-test-development/blob/a862bf3f9e27131e2b7d0e0009d3c5af4096c6b3/dynamic-code-analysis/scripts/run_drmemory.py">run_drmemory.py</a> to extra_scripts:</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #0000cc;">extra_scripts</span> <span style="color: #333333;">=</span> <span style="background-color: #fff0f0;"></span>
<span style="background-color: #fff0f0;"> scripts/run_drmemory.py</span>
</pre></div>
<p>Just note that Dr. Memory can do <a href="http://gina.xmmg.com/dpweb/docs/drmem1.11.0.txt">more than that</a>, so its good to explore it further.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhzwDBQ-Gb89MMbLigO071ZFlbrPZ02nlLW9pKt8aCyWh_ZjHVNKoM7S6Ppd1sVgi35696Pzs0203rReHEBmiMGy2muwlY4qFgJ2q0Z0wkIebwBDOh4D22A9Jzlk1NTe2wb-IvLRosnL-_wuneX8qEhqGf2_mCWRbTO71q4EuvpCjcIacbtjAKghhr_" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="455" data-original-width="1185" src="https://blogger.googleusercontent.com/img/a/AVvXsEhzwDBQ-Gb89MMbLigO071ZFlbrPZ02nlLW9pKt8aCyWh_ZjHVNKoM7S6Ppd1sVgi35696Pzs0203rReHEBmiMGy2muwlY4qFgJ2q0Z0wkIebwBDOh4D22A9Jzlk1NTe2wb-IvLRosnL-_wuneX8qEhqGf2_mCWRbTO71q4EuvpCjcIacbtjAKghhr_=s16000" /></a></div><br /><br /><div><br /></div><div>To sum things up, dynamic code analysis tools are essential to writing quality code with the added benefit of finding which parts of your code have leaks or not covered by enough tests.<p></p><p><a href="https://blog.drorgluska.com/2022/06/platformio-static-code-analysis.html">Continue reading about static code analysis...</a></p></div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3077621944057198788.post-41454410614676268332022-06-02T06:57:00.005-07:002022-07-03T05:48:33.217-07:00PlatformIO Static Code Analysis<p></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEj-8oNtlCuzs0_eugiOPLjlDy3iWmuQxwmnZPbr4he62dqRQWUxrVZepp6OmlwY89bvz4GpFjxbP5W7gtOfxjuI_6wEApfiF0HOBr4WPcAacY08-k9K1VBZf23f5iSaFzOOmaBG3a5ia6HDoszBBWvSXWkPO4vVWQrgYTTZvH3vWIufrm5wHzmfXXzo" style="margin-left: auto; margin-right: auto;"><img alt="" data-original-height="750" data-original-width="1125" height="266" src="https://blogger.googleusercontent.com/img/a/AVvXsEj-8oNtlCuzs0_eugiOPLjlDy3iWmuQxwmnZPbr4he62dqRQWUxrVZepp6OmlwY89bvz4GpFjxbP5W7gtOfxjuI_6wEApfiF0HOBr4WPcAacY08-k9K1VBZf23f5iSaFzOOmaBG3a5ia6HDoszBBWvSXWkPO4vVWQrgYTTZvH3vWIufrm5wHzmfXXzo=w400-h266" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><a href="https://www.pexels.com/photo/grayscale-photo-of-computer-laptop-near-white-notebook-and-ceramic-mug-on-table-169573/">Source</a></td></tr></tbody></table><br /><br /><p></p><p>Writing code and testing it with unit and integration tests can provide high quality executable, however, it does not indicate if its a maintainable code, moreover code can have side effects when used outside its designed use, so in essence, tests pass, QA approved but then it gets in the field and crashes. </p><p>One of the ways to make sure code is maintainable, understandable and quick to get into is code reviews, however, using style checks and static code analysis can help automate some of it before a human can review it.</p><p>Applications are like Swiss cheese, one tool can verify certain aspects of it but many tools can discover more possible issues.</p><h3 style="text-align: left;">Code Style </h3><p>Code style defines how a code looks like to the developer, it will most likely not affect how the code is run but it can affect how quickly a developer can understand what its doing.</p><h4 style="text-align: left;">clang-format</h4><div><a href="https://clang.llvm.org/docs/ClangFormat.html">clang-format</a> is a C++ code formatter and supports <a href="https://clang.llvm.org/docs/ClangFormatStyleOptions.html">many formatting options</a> that can help with readability and consistent source code format.</div><div><br /></div><div>to install clang-format:</div><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">pip install clang-format
</pre></div>
</div><div><br /></div><div>Once installed, we can generate a default clang-format configuration file and edit it if needed, clang-format defines a few standard styles: LLVM, GNU, Google, Chromium, Microsoft, Mozilla, WebKit</div><div><br /></div><div>the following line will generate the configuration based on LLVM:</div><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">clang-format -style<span style="color: #333333;">=</span>LLVM -dump-config > .clang-format
</pre></div>
</div><div><br /></div><div>For PlatformIO specific integration you can use my <a href="https://github.com/drorgl/esp32-test-development/blob/master/static-code-analysis/scripts/run_clangformat.py">runner</a> and execute it as follows:</div><div><br /></div><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">pio run -t format
</pre></div>
</div><div>
</div><div><br /></div><h4 style="text-align: left;">cpplint</h4><p><a href="https://github.com/cpplint/cpplint">cpplint</a> is a tool that checks <a href="https://google.github.io/styleguide/cppguide.html">Google's C++ Style Guide</a>, it can report the issues to command line or you can use the <a href="https://marketplace.visualstudio.com/items?itemName=mine.cpplint">VSCode Extension</a> and see the issues while writing code.</p><p>While the style guide is very strict, you can control how cpplint behaves by adjusting the linelength and which rule to apply or ignore in cpplint.cfg</p><p>For PlatformIO specific integration you can use my <a href="https://github.com/drorgl/esp32-test-development/blob/master/static-code-analysis/scripts/run_cpplint.py">runner</a> and execute it as follows:</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">pio run -t lint
</pre></div>
<p style="text-align: center;"><span style="font-size: medium;">By combining clang-format and cpplint you can avoid ever styling your code manually</span></p><h3 style="text-align: left;">Static Code Analysis</h3>While coding style can really help for readability of code and preventing confusion, styling by itself does not contribute to the quality of code executed. Lets explore what is available <h4 style="text-align: left;">Compiler Warnings</h4><p>A great source of warnings and issues that originate from mistakes or lack of understanding of C/C++ languages is compiler warnings, the default warning level attempts to balance between certain mistakes and an attempt not to overwhelm the developer.</p><p>These flags can help you to switch between simple mistakes and pedantic development style, if you find yourself in a pickle or would like to avoid the pickle all together it might be beneficial to use the more restrictive warning levels but like everything in software, the tool does not make the software and you'll need to understand why you need to fix what the compiler tells you to fix.</p><p>Chris Coleman wrote about <a href="https://interrupt.memfault.com/blog/best-and-worst-gcc-clang-compiler-flags">The Best and Worst GCC Compiler Flags For Embedded</a> other than the <a href="https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html">GCC warnings documentation</a> but in summary:</p><p><span style="font-family: courier;">-Wall</span> - enables warnings about questionable practices that are easy to avoid</p><p><span style="font-family: courier;">-Wextra</span> - more warnings </p><p><span style="font-family: courier;">-Wshadow</span> - shadowing is a readability issue that can also lead to bugs since the developer might get confused about which variable is actually in use.</p><p><span style="font-family: courier;">-Wdouble-promotion</span> - some MCUs have FPU that supports floats only, whenever a floating point gets promoted to double for any reason this warning will tell you about it since you might lose performance over it.</p><p><span style="font-family: courier;">-Wformat=2</span> - checks scanf and printf mistakes</p><p><span style="font-family: courier;">-Wformat-truncation</span> - checks snprintf has enough room, heuristics based.</p><p><span style="font-family: courier;">-Wundef</span> - warning if undefined identifier is evaluated in the preprocessor.</p><p><span style="font-family: courier;"><a href="https://gcc.gnu.org/onlinedocs/gcc/C_002b_002b-Dialect-Options.html">-Weffc++</a></span> - Warn about violations of style guidelines from Scott Meyers’ Effective C++ series of books</p><p>In any case, you can always view which warnings are enabled by:</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">gcc -Q --help<span style="color: #333333;">=</span>warnings
</pre></div>
<p>If you'd like to see which ones are enabled with using a certain warning level:</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; text-align: left; width: auto;"><pre style="line-height: 125%; margin: 0px;">gcc -Wall -Wextra -Q --help<span style="color: #333333;">=</span>warnings
</pre></div>
<h3 style="text-align: left;">PlatformIO Check</h3><div>PlatformIO <a href="https://docs.platformio.org/en/latest/core/userguide/cmd_check.html">check</a> provides easy access to two static code analyzers, cppcheck and clang-tidy. to use them we need to add:</div><div><br /></div><div><br /></div><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #0000cc;">check_tool</span> <span style="color: #333333;">=</span> <span style="background-color: #fff0f0;">cppcheck, clangtidy</span>
</pre></div>
</div><div><br /></div><div>and then run:</div><div><br /></div><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">pio check
</pre></div>
</div><div><br /></div><div>This should get you results similar to this when running the checks:</div><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">Checking native > cppcheck (platform: native)
----------------------------------------------------------------------------------------------
src\port_arduino.h:6: [low:style] Function types shall be in prototype form with named parameters [misra-c2012-8.2]
...
================================ [PASSED] Took 3.53 seconds ==================================</pre><pre style="line-height: 125%; margin: 0px;">Checking native > clangtidy (platform: native)
----------------------------------------------------------------------------------------------
src\main.cpp:6: [medium:warning] system include stdio.h not allowed [llvmlibc-restrict-system-libc-headers]
...
================================ [PASSED] Took 1.34 seconds ==================================
Component HIGH MEDIUM LOW
------------------ ------ -------- -----
lib\circularbuffer 0 0 32
...
Total 2 17 60
Environment Tool Status Duration
------------- --------- -------- ------------
native cppcheck PASSED 00:00:03.533
native clangtidy PASSED 00:00:01.341
================================= 2 succeeded in 00:00:04.874 ================================
</pre></div>
</div><h3 style="text-align: left;">cppcheck</h3><p><a href="https://cppcheck.sourceforge.io/">cppcheck</a> is a free static code analyzer, it detects common mistakes and also supports a <a href="https://sourceforge.net/p/cppcheck/wiki/ListOfChecks/">subset</a> of MISRA standard, you can find examples and explanations in the <a href="https://docs.zephyrproject.org/latest/contribute/coding_guidelines/index.html">Zephyr documentation</a>.</p><p>To support MISRA checks, you'll need to add a few things to the default PlatformIO configuration.</p><p>1. in platformio.ini section:</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #996633;">check_flags</span> <span style="color: #333333;">=</span>
cppcheck: --enable<span style="color: #333333;">=</span>all --addon<span style="color: #333333;">=</span>./scripts/misra.json --addon<span style="color: #333333;">=</span>cert --addon<span style="color: #333333;">=</span>threadsafety --addon<span style="color: #333333;">=</span>y2038
</pre></div>
<p>2. download <a href="https://github.com/danmar/cppcheck/blob/main/addons/misra.py">misra.py</a>, <a href="https://github.com/danmar/cppcheck/blob/main/addons/misra_9.py">misra_9.py</a> and <a href="https://github.com/danmar/cppcheck/blob/main/addons/cppcheckdata.py">cppcheckdata.py</a> from cppcheck repository and place it in scripts folder.</p><p>3. add misra.json to the scripts folder, this is the configuration for the MISRA addon, I've disabled rule 17.7 in this example.</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">{
<span style="color: #007700;">"script"</span>: <span style="background-color: #fff0f0;">"scripts/misra.py"</span>,
<span style="color: #007700;">"args"</span>: [<span style="background-color: #fff0f0;">"--rule-texts=scripts/misra.txt"</span>,<span style="background-color: #fff0f0;">"--suppress-rules 17.7"</span>]
}
</pre></div>
<p>4. download <a href="https://github.com/tim-rose/makeshift/blob/master/data/cppcheck/misra.txt">misra.txt</a> and place it in the scripts folder for the addon to pick up and use as messages.</p><p>This should get you results similar to this when running the checks:</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">Checking native > cppcheck (platform: native)
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
src\port_arduino.h:6: [low:style] Function types shall be in prototype form with named parameters [misra-c2012-8.2]
src\port_arduino.h:7: [low:style] Function types shall be in prototype form with named parameters [misra-c2012-8.2]
src\port_arduino.h:10: [low:style] Function types shall be in prototype form with named parameters [misra-c2012-8.2]
lib\examplelib\ProductionCode.h:3: [low:style] Function types shall be in prototype form with named parameters [misra-c2012-8.2]
src\main.cpp:6: [low:style] The Standard Library input/output functions shall not be used [misra-c2012-21.6]
src\main.cpp:22: [low:style] Do not use the rand() function for generating pseudorandom numbers [cert-MSC30-c]
lib\circularbuffer\CircularBuffer.cpp:32: [low:style] There should be no unused parameters in functions [misra-c2012-2.7]
lib\circularbuffer\CircularBuffer.cpp:88: [low:style] A string literal shall not be assigned to an object unless the object's type is pointer to const-qualified char [misra-c2012-7.4]
lib\circularbuffer\CircularBuffer.cpp:102: [low:style] A string literal shall not be assigned to an object unless the object's type is pointer to const-qualified char [misra-c2012-7.4]
lib\circularbuffer\CircularBuffer.cpp:105: [low:style] A string literal shall not be assigned to an object unless the object's type is pointer to const-qualified char [misra-c2012-7.4]
lib\circularbuffer\CircularBuffer.h:50: [low:style] Function types shall be in prototype form with named parameters [misra-c2012-8.2]
lib\circularbuffer\CircularBuffer.h:52: [low:style] Function types shall be in prototype form with named parameters [misra-c2012-8.2]
lib\circularbuffer\CircularBuffer.h:53: [low:style] Function types shall be in prototype form with named parameters [misra-c2012-8.2]
lib\circularbuffer\CircularBuffer.h:54: [low:style] Function types shall be in prototype form with named parameters [misra-c2012-8.2]
</pre></div>
<p><br /></p><h3 style="text-align: left;">clang-tidy</h3><div><a href="https://clang.llvm.org/extra/clang-tidy">clang-tidy</a> is llvm's static code analayzer, one of the more interesting features is that it can fix some errors it finds. </div><div><br /></div><div>to run it with fix you can run it as follows:</div><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">pio check --flags <span style="background-color: #fff0f0;">"clangtidy: --fix"</span>
</pre></div>
</div><div><br /></div><div>This should get you results similar to this when running the checks:</div><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">Checking native > clangtidy (platform: native)
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
src\main.cpp:6: [medium:warning] system include stdio.h not allowed [llvmlibc-restrict-system-libc-headers]
src\main.cpp:6: [medium:warning] inclusion of deprecated C++ header 'stdio.h'; consider using 'cstdio' instead [hicpp-deprecated-headers,modernize-deprecated-headers]
src\main.cpp:9: [medium:warning] system include stdlib.h not allowed [llvmlibc-restrict-system-libc-headers]
src\main.cpp:9: [medium:warning] inclusion of deprecated C++ header 'stdlib.h'; consider using 'cstdlib' instead [hicpp-deprecated-headers,modernize-deprecated-headers]
src\main.cpp:16: [medium:warning] declaration must be declared within the '__llvm_libc' namespace [llvmlibc-implementation-in-namespace]
</pre></div>
</div><div><h3>Flawfinder</h3><div><a href="https://dwheeler.com/flawfinder/">Flawfinder</a> is a simple tool for scanning source code for possible security weaknesses (or "flaws"). </div><div><br /></div><div>For PlatformIO specific integration you can use my <a href="https://github.com/drorgl/esp32-test-development/blob/master/static-code-analysis/scripts/run_flawfinder.py">runner</a> and execute it as follows:</div><div><br /></div><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">pio run -t flawfinder
</pre></div>
</div><p>This should get you results similar to this when running flawfinder:</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">flawfinder -C -c -D -i -S -Q include src lib\arduino-printf lib\circularbuffer lib\defectedLib lib\examplelib lib\runner
src\main.cpp:21:2: [0] (format) printf:If format strings can be influenced by an attacker, they can be exploited (CWE-134). Use a constant for the format specification. Constant format string, so not considered risky.
printf("test broken %d\r\n", FindFunction_WhichIsBroken(78));
src\main.cpp:24:2: [0] (format) printf:If format strings can be influenced by an attacker, they can be exploited (CWE-134). Use a constant for the format specification. Constant format string, so not considered risky.
printf("displaying float %.6f", c);
lib\circularbuffer\CircularBuffer.cpp:90:5: [2] (buffer) char:Statically-sized arrays can be improperly restricted, leading to potential overflows or other issues (CWE-119!/CWE-120). Perform bounds checking, use functions
that limit length, or ensure that the size is larger than the maximum possible length.
char sval[10];
lib\circularbuffer\CircularBuffer.cpp:98:9: [0] (format) snprintf:If format strings can be influenced by an attacker, they can be exploited, and note that sprintf variations do not always \0-terminate (CWE-134). Use a constant for the format specification. Constant format string, so not considered risky.
snprintf(sval, sizeof(sval), "%d", buffer[printIndex]);
lib\circularbuffer\CircularBuffer.cpp:90:5: [2] (buffer) char:Statically-sized arrays can be improperly restricted, leading to potential overflows or other issues (CWE-119!/CWE-120). Perform bounds checking, use functions
that limit length, or ensure that the size is larger than the maximum possible length.
char sval[10];</pre></div></div><h3 style="text-align: left;">doxygen</h3><p>while doxygen is not a static code analyzer per-se, it can help to generate documentation, call graphs and help other developers to understand the code in shorter time.</p><p>To ease with function documentation, you can use the <a href="https://marketplace.visualstudio.com/items?itemName=cschlosser.doxdocgen">Doxygen VSCode Extension</a>.</p><p>Install <a href="https://www.doxygen.nl/manual/install.html">doxygen</a> and <a href="https://graphviz.org/download/">graphviz</a> which is used to generate the graphs.</p><p>generate a basic configuration file:</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">doxygen -g .doxygen
</pre></div>
<p>As a quick start override the following settings:</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #996633;">OUTPUT_DIRECTORY</span> <span style="color: #333333;">=</span> docs
<span style="color: #996633;">INPUT</span> <span style="color: #333333;">=</span> lib src
<span style="color: #996633;">BUILTIN_STL_SUPPORT</span> <span style="color: #333333;">=</span> YES
<span style="color: #996633;">EXTRACT_ALL</span> <span style="color: #333333;">=</span> YES
<span style="color: #996633;">EXTRACT_STATIC</span> <span style="color: #333333;">=</span> YES
<span style="color: #996633;">WARN_NO_PARAMDOC</span> <span style="color: #333333;">=</span> YES
<span style="color: #996633;">RECURSIVE</span> <span style="color: #333333;">=</span> YES
<span style="color: #996633;">STRIP_CODE_COMMENTS</span> <span style="color: #333333;">=</span> NO
<span style="color: #996633;">REFERENCED_BY_RELATION</span> <span style="color: #333333;">=</span> YES
<span style="color: #996633;">REFERENCES_RELATION</span> <span style="color: #333333;">=</span> YES
<span style="color: #996633;">GENERATE_LATEX</span> <span style="color: #333333;">=</span> NO
<span style="color: #996633;">MACRO_EXPANSION</span> <span style="color: #333333;">=</span> YES
<span style="color: #996633;">HAVE_DOT</span> <span style="color: #333333;">=</span> YES
<span style="color: #996633;">UML_LOOK</span> <span style="color: #333333;">=</span> YES
<span style="color: #996633;">CALL_GRAPH</span> <span style="color: #333333;">=</span> YES
<span style="color: #996633;">CALLER_GRAPH</span> <span style="color: #333333;">=</span> YES
<span style="color: #996633;">INTERACTIVE_SVG</span> <span style="color: #333333;">=</span> YES
</pre></div>
<p>Generate the documentation:</p><p>doxygen .doxygen</p>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">doxygen .doxygen
</pre></div>
<p>And finally, you should be able to see the documentation by opening the docs/html/index.html, search for functions and browse the documentation, you'll see that just after a few minutes you already know the basic structure, which function references which and you'll get a feeling of the general flow of the application.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEi3opTNQTnalaDGWJqGwW7VjcH4g1PvInql5ZFrRMQoQ4-MoCMXuIxDigDa30wF6XwMqaXU2DRM58M5Sc203TsJ1pWi3-NqY9HrVg_JnKqS0-OiSXo1DZg75qM9b0KcOPm9QeXaW8kvtwS1Nt5eaQMdK-xEzALhZmTQfotZ9Tu5jRHDR4232DW7jQAN" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="406" data-original-width="667" height="244" src="https://blogger.googleusercontent.com/img/a/AVvXsEi3opTNQTnalaDGWJqGwW7VjcH4g1PvInql5ZFrRMQoQ4-MoCMXuIxDigDa30wF6XwMqaXU2DRM58M5Sc203TsJ1pWi3-NqY9HrVg_JnKqS0-OiSXo1DZg75qM9b0KcOPm9QeXaW8kvtwS1Nt5eaQMdK-xEzALhZmTQfotZ9Tu5jRHDR4232DW7jQAN=w400-h244" width="400" /></a></div><p><br /></p><h3>Code Metrics</h3>Code metrics are used to find hot spots that go against clean code's most prominent rule, “The first rule of functions is that they should be small.”<br />With code metrics we can find long functions, long files, <a href="#">complex functions</a> and the most basic thing we can do to our team members to save them from this.<br /><h4 style="font-size: medium; font-weight: 400;">Lizard</h4><div style="font-size: medium; font-weight: 400;">Lizard is a <a href="https://github.com/terryyin/lizard">code complexity analyzer</a>, while it supports many languages, we only want it to work with C/C++ code.</div><div style="font-size: medium; font-weight: 400;"><br /></div><div style="font-size: medium; font-weight: 400;">First, we'll need to setup lizard's setting in platformio.ini, in this case we're limiting the complexity to 15, maximum length of functions to 100 lines and maximum arguments passed to a function is 1.</div><div style="font-size: medium; font-weight: 400;"><br /></div><div style="font-size: medium; font-weight: 400;">
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #0000cc;">cyclomatic_complexity_analyzer</span> <span style="color: #333333;">=</span> <span style="background-color: #fff0f0;">--CCN 15 --length 100 --arguments 1 --warning-msvs</span>
</pre></div>
</div><div style="font-size: medium; font-weight: 400;"><br /></div><div style="font-size: medium; font-weight: 400;">For PlatformIO specific integration you can use my <a href="https://github.com/drorgl/esp32-test-development/blob/master/static-code-analysis/scripts/run_lizard.py">runner</a> and execute it as follows:</div><div style="font-size: medium; font-weight: 400;"><br /></div><div style="font-size: medium; font-weight: 400;">
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">pio run -t lizard
</pre></div>
</div><div style="font-size: medium; font-weight: 400;"><br /></div><div style="font-size: medium; font-weight: 400;">And see results similar to this:</div><div style="font-size: medium; font-weight: 400;">
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">lib\examplelib\ProductionCode2.c(4): warning: ThisFunctionHasNotBeenTested has 6 NLOC, 1 CCN, 28 token, 2 PARAM, 8 length
lizard -Eduplicate include src lib\arduino-printf lib\circularbuffer lib\defectedLib lib\examplelib lib\runner
*** Error 1
================================================
NLOC CCN token PARAM length location
------------------------------------------------
8 2 48 0 10 setup@16-25@src\main.cpp
3 1 5 0 4 loop@27-30@src\main.cpp
7 2 19 0 10 main@10-19@src\port_arduino.h
5 1 44 1 5 CircularBuffer::CircularBuffer@32-36@lib\circularbuffer\CircularBuffer.cpp
4 1 12 0 4 CircularBuffer::~CircularBuffer@38-41@lib\circularbuffer\CircularBuffer.cpp
4 1 10 0 4 CircularBuffer::IsEmpty@43-46@lib\circularbuffer\CircularBuffer.cpp
4 1 10 0 4 CircularBuffer::IsFull@48-51@lib\circularbuffer\CircularBuffer.cpp
8 3 49 1 8 CircularBuffer::Put@53-60@lib\circularbuffer\CircularBuffer.cpp
11 3 51 0 12 CircularBuffer::Get@62-73@lib\circularbuffer\CircularBuffer.cpp
4 1 10 0 4 CircularBuffer::Capacity@75-78@lib\circularbuffer\CircularBuffer.cpp
5 2 23 1 5 CircularBuffer::Next@80-84@lib\circularbuffer\CircularBuffer.cpp
18 5 116 0 22 CircularBuffer::Print@86-107@lib\circularbuffer\CircularBuffer.cpp
11 2 69 0 11 dynamic_buffer_overrun_018@5-15@lib\defectedLib\bufferLibrary.c
4 1 14 0 4 memory_leak_001@17-20@lib\defectedLib\bufferLibrary.c
9 3 36 1 9 FindFunction_WhichIsBroken@11-19@lib\examplelib\ProductionCode.c
4 1 9 1 4 FunctionWhichReturnsLocalVariable@21-24@lib\examplelib\ProductionCode.c
6 1 28 2 8 ThisFunctionHasNotBeenTested@4-11@lib\examplelib\ProductionCode2.c
3 1 11 0 3 setup@30-32@lib\runner\runner.h
13 file analyzed.
==============================================================
NLOC Avg.NLOC AvgCCN Avg.token function_cnt file
--------------------------------------------------------------
19 5.5 1.5 26.5 2 src\main.cpp
9 7.0 2.0 19.0 1 src\port_arduino.h
0 0.0 0.0 0.0 0 src\sdkconfig.h
1 0.0 0.0 0.0 0 lib\arduino-printf\arduino-printf.h
65 7.0 2.0 36.1 9 lib\circularbuffer\CircularBuffer.cpp
29 0.0 0.0 0.0 0 lib\circularbuffer\CircularBuffer.h
18 7.5 1.5 41.5 2 lib\defectedLib\bufferLibrary.c
2 0.0 0.0 0.0 0 lib\defectedLib\bufferLibrary.h
16 6.5 2.0 22.5 2 lib\examplelib\ProductionCode.c
2 0.0 0.0 0.0 0 lib\examplelib\ProductionCode.h
7 6.0 1.0 28.0 1 lib\examplelib\ProductionCode2.c
1 0.0 0.0 0.0 0 lib\examplelib\ProductionCode2.h
12 3.0 1.0 11.0 1 lib\runner\runner.h
===============================================================================================================
No thresholds exceeded (cyclomatic_complexity > 15 or length > 1000 or nloc > 1000000 or parameter_count > 100)
==========================================================================================
Total nloc Avg.NLOC AvgCCN Avg.token Fun Cnt Warning cnt Fun Rt nloc Rt
------------------------------------------------------------------------------------------
181 6.6 1.8 31.3 18 0 0.00 0.00
Duplicates
===================================
Total duplicate rate: 0.00%
Total unique rate: 100.00%</pre></div></div><h3><br /></h3><div>Now we can see a warning that a function has two parameters (over 1 of the limit we set)</div><div>We can also see statistics for the entire project analysis, this can help us locate functions that are approaching the limits we set and find duplicate code.</div><h3>Coding Standards</h3><p>In addition to MISRA, there are other interesting standards that you should be aware of.</p><p><a href="https://wiki.sei.cmu.edu/confluence/display/c/SEI+CERT+C+Coding+Standard">SEI CERT C Coding Standard</a> (<a href="https://resources.sei.cmu.edu/downloads/secure-coding/assets/sei-cert-c-coding-standard-2016-v01.pdf">PDF</a>), what I really like about this standard is the explanation each rule have and why its in the standard.</p><p>Lastly but not less important is the <a href="https://www.autosar.org/fileadmin/user_upload/standards/adaptive/17-03/AUTOSAR_RS_CPP14Guidelines.pdf">AUTOSAR Guidelines for the use of the C++14 language in critical and safety-related systems (PDF)</a>, like the others, one of the more important sections is the rational for including the guidelines can provide an important insight into the "why" and is always a good reading material.</p><h3 style="text-align: left;">Test Sample</h3><div>Lastly, if you're currently evaluating code standard tools and static code analysis tools, you may find the <a href="https://github.com/regehr/itc-benchmarks">itc-benchmarks</a> beneficial.</div><div><br /></div><div><a href="https://blog.drorgluska.com/2022/06/platformio-dynamic-code-analysis-and.html">Continue reading about dynamic code analysis...</a></div><h3 style="text-align: left;">References:</h3><p><a href="https://elib.dlr.de/133945/1/2020_Gentsch_SAST.pdf">Evaluation of Open Source Static Analysis Security Testing (SAST) Tools for C</a></p><p><a href="https://elinux.org/images/e/ed/ELCE_FOSS_Static_Analysis_Tools_for_Embedded_Systems_and_How_to_Use_Them.pdf">FOSS Static Analysis Tools for Embedded Systems and How to Use Them</a><br /></p><p><a href="https://www.stroustrup.com/JSF-AV-rules.pdf">Joint Strike Fighter Air Vehicle C++ Coding Standards</a><br /></p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3077621944057198788.post-59888437467936738632022-05-22T21:55:00.002-07:002022-06-07T23:37:57.930-07:00Unit Testing with ESP32 and PlatformIO<p>Writing Quality Code and Tests usually goes hand in hand, some find it more productive to write code and then write tests for it and others, following TDD/BDD write tests before code. The benefits of each is out of scope for this article, what is in scope is how to reach the nirvana of software development by using tests.</p><p><br /></p><p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgQ7PY2qh-3VTMu5EhwsUCEFvSlnzi2qXPJGuI8-OlM7GmarDVHlr0q9PLFuKshO30JV1MeN0Rf8FfqPGL2YhoY7vkykdOuj4LJVMr54hNW9DXAakgwiXu2XnCO50iSSKkvreMBBji2rHYZz8A_SGAmaDbfNA6mgcB4JMrdCOFXzhHtCiDwH-QMUYDp" style="margin-left: auto; margin-right: auto;"><img alt="" data-original-height="750" data-original-width="1190" height="404" src="https://blogger.googleusercontent.com/img/a/AVvXsEgQ7PY2qh-3VTMu5EhwsUCEFvSlnzi2qXPJGuI8-OlM7GmarDVHlr0q9PLFuKshO30JV1MeN0Rf8FfqPGL2YhoY7vkykdOuj4LJVMr54hNW9DXAakgwiXu2XnCO50iSSKkvreMBBji2rHYZz8A_SGAmaDbfNA6mgcB4JMrdCOFXzhHtCiDwH-QMUYDp=w640-h404" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><a href="https://www.pexels.com/photo/red-and-yellow-hatchback-axa-crash-tests-163016/">source</a></td></tr></tbody></table><br /><br /></p><p></p><p>Lets start with the basics.</p><p>PlatformIO tests can be run on:</p><p></p><ul><li>Locally - on the development machine, it requires that gcc is installed and in the path. for windows this can be done with mingw gcc.</li><li>On Device - PlatformIO automates this by building the tests, uploading them and resetting the device in order for them to execute and wait for 2 lines, one stating how many tests were executed and another stating if any of the tests failed.</li><li>Remotely - PlatformIO automates this by building the code on the development machine, copying the firmware to the remote agent, uploading to the device connected to the remote agent and waiting for the test results.</li></ul><p></p><p>To reach that nirvana we can write tests that execute on the ESP32, but no matter how fast your machine is, compiling, uploading and executing tests on ESP32 can take anywhere from 1 to 10 minutes and that will reduce the effectiveness of the TDD/BDD cycles, one of the big enemies of workflow is slow workflow where the mind wanders off.</p><p>Another option would be to do quick development cycles on the development machine and every once in a while running the tests on the device and lastly ensuring code quality by allowing the CI on the build machine to make sure the code is built and tested on an actual device.</p><h3>The basic Tenet</h3><p>All testing should be written in the same way, we prepare code and data for the unit under test, we execute the operation and we should verify the outcome is the one we expected, this is called the AAA Pattern, or Arrange-Act-Assert.</p><p>Arrange - prepare objects and data for operation</p><p>Act - execute operation</p><p>Assert - check results/side effects</p><p>If you have the same arrange for many tests, consider the "setup" phase most testing frameworks have, but to maintain a coherent style one should strive to keep the generic parts in the setup phase and the test specific parts in the test itself.</p><p>If you elected to use the "setup" phase, just note that you also have the "tear down" phase to do the cleanup from the setup phase - keeping them out of the test is usually a good practice.</p><p>Another tenet is repeatability, tests which are not repeatable cannot be trusted and are soon ignored. avoid testing random values in unit tests.</p><h3>Unit Tests</h3><div>Unit tests are the most basic kind of testing but are sometimes misunderstood, the purpose of the unit tests are to verify a needed functionality works in the way it was designed to work during the life of the application, there is a broad interpretation of what constitutes a 'unit' but most agree that a unit is a small chunk of code (or function) that can be accessed from outside the module its in.</div><div><br /></div><div>Note: while testing every individual functionality manually can also verify the code "works", its not doing it throughout the life of the application, a manual test can lead to a waste of time and will not test the same functionality every time. </div><div><br /></div><div>PlatformIO provides a way to run tests and split them into groups, each group can be set to either run or not run on each platform (ESP32, Native and more), we can split by library, header or other significant group.</div><div><br /></div><h3 style="text-align: left;">Project Tests Structure</h3><div>For very simple projects you can put a single main file in the test folder and run all tests from it, keep in mind if it gets too big it might not fit in ESP32. </div><div>For more <a href="https://docs.platformio.org/en/stable//advanced/unit-testing/structure.html#test-hierarchy">complex projects</a>, its possible to create multiple executables in the test folder by using subfolders, each subfolder starting with <span style="font-family: courier;">test_</span> will create a separate executable.</div><div><br /></div><div>Like in all other unit test demos, we'll start with a simple calculator, it has addition, subtraction, multiplication and division.</div><div><br /></div><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #333399; font-weight: bold;">int</span> Calculator<span style="color: #333333;">::</span>add(<span style="color: #333399; font-weight: bold;">int</span> a, <span style="color: #333399; font-weight: bold;">int</span> b)
{
<span style="color: #008800; font-weight: bold;">return</span> a <span style="color: #333333;">+</span> b;
}
<span style="color: #333399; font-weight: bold;">int</span> Calculator<span style="color: #333333;">::</span>sub(<span style="color: #333399; font-weight: bold;">int</span> a, <span style="color: #333399; font-weight: bold;">int</span> b)
{
<span style="color: #008800; font-weight: bold;">return</span> a <span style="color: #333333;">-</span> b;
}
<span style="color: #333399; font-weight: bold;">int</span> Calculator<span style="color: #333333;">::</span>mul(<span style="color: #333399; font-weight: bold;">int</span> a, <span style="color: #333399; font-weight: bold;">int</span> b)
{
<span style="color: #008800; font-weight: bold;">return</span> a <span style="color: #333333;">*</span> b;
}
<span style="color: #333399; font-weight: bold;">int</span> Calculator<span style="color: #333333;">::</span>div(<span style="color: #333399; font-weight: bold;">int</span> a, <span style="color: #333399; font-weight: bold;">int</span> b)
{
<span style="color: #008800; font-weight: bold;">return</span> a <span style="color: #333333;">/</span> b;
}
</pre></div>
</div><div><a href="https://github.com/drorgl/esp32-test-development/blob/b454e1123bddedc324e245b45fb783705aa56a5f/esp32-unit-test-unity/lib/calculator/src/calculator.cpp#L20-L39">calculator.cpp</a><br /></div><div><br /></div><h3 style="text-align: left;">Testing Frameworks</h3><div>PlatformIO supports out of the box a few <a href="https://docs.platformio.org/en/stable//advanced/unit-testing/frameworks/index.html#unit-testing-frameworks">testing frameworks</a>, the more popular ones in the embedded world are unity, cpputest and doctest.</div><div><br /></div><div>Now we'll write some tests</div><h4 style="text-align: left;">Unity</h4><div>Unity is popular due to its low overhead and simple use, it was the first framework supported on PlatformIO.</div><div><br /></div><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #557799;">#include <calculator.h></span>
<span style="color: #557799;">#include <unity.h> </span><span style="color: #888888;">//Unity Testing Framework</span>
<span style="color: #557799;">#include <runner.h> </span><span style="color: #888888;">//Simplifies main()</span>
Calculator calc;
<span style="color: #333399; font-weight: bold;">void</span> <span style="color: #0066bb; font-weight: bold;">test_function_calculator_addition</span>(<span style="color: #333399; font-weight: bold;">void</span>) {
TEST_ASSERT_EQUAL(<span style="color: #0000dd; font-weight: bold;">32</span>, calc.add(<span style="color: #0000dd; font-weight: bold;">25</span>, <span style="color: #0000dd; font-weight: bold;">7</span>));
}
<span style="color: #333399; font-weight: bold;">void</span> <span style="color: #0066bb; font-weight: bold;">test_function_calculator_subtraction</span>(<span style="color: #333399; font-weight: bold;">void</span>) {
TEST_ASSERT_EQUAL(<span style="color: #0000dd; font-weight: bold;">20</span>, calc.sub(<span style="color: #0000dd; font-weight: bold;">23</span>, <span style="color: #0000dd; font-weight: bold;">3</span>));
}
<span style="color: #333399; font-weight: bold;">void</span> <span style="color: #0066bb; font-weight: bold;">test_function_calculator_multiplication</span>(<span style="color: #333399; font-weight: bold;">void</span>) {
TEST_ASSERT_EQUAL(<span style="color: #0000dd; font-weight: bold;">50</span>, calc.mul(<span style="color: #0000dd; font-weight: bold;">25</span>, <span style="color: #0000dd; font-weight: bold;">2</span>));
}
<span style="color: #333399; font-weight: bold;">void</span> <span style="color: #0066bb; font-weight: bold;">test_function_calculator_division</span>(<span style="color: #333399; font-weight: bold;">void</span>) {
TEST_ASSERT_EQUAL(<span style="color: #0000dd; font-weight: bold;">32</span>, calc.div(<span style="color: #0000dd; font-weight: bold;">96</span>, <span style="color: #0000dd; font-weight: bold;">3</span>));
}
<span style="color: #333399; font-weight: bold;">void</span> <span style="color: #0066bb; font-weight: bold;">process</span>() {
UNITY_BEGIN();
RUN_TEST(test_function_calculator_addition);
RUN_TEST(test_function_calculator_subtraction);
RUN_TEST(test_function_calculator_multiplication);
RUN_TEST(test_function_calculator_division);
UNITY_END();
}
MAIN(){
process();
}
</pre></div>
</div><div><a href="https://github.com/drorgl/esp32-test-development/blob/b454e1123bddedc324e245b45fb783705aa56a5f/esp32-unit-test-unity/test/test_common/test_calculator.cpp">test_calculator.cpp</a></div><h4 style="text-align: left;">doctest</h4><div><a href="https://github.com/doctest/doctest">doctest</a> is a similar framework to <a href="https://github.com/catchorg/Catch2">Catch</a>, except for the slow compilation time, some developers really like Catch's way of doing things but really hate the slow compilation time, doctest while <a href="https://docs.platformio.org/en/stable//advanced/unit-testing/frameworks/doctest.html#unit-testing-frameworks-doctest">natively supported by PlatformIO</a> still have issues when compiling for ESP32, you can find a <a href="https://github.com/drorgl/esp32-test-development/blob/master/esp32-unit-test-doctest/lib/doctest/include/doctest/doctest.h">modified doctest</a> in the examples that works fine with both ESP32 and natively.</div><div><br /></div><div><br /></div><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #557799;">#define DOCTEST_CONFIG_IMPLEMENT</span>
<span style="color: #557799;">#define DOCTEST_THREAD_LOCAL</span>
<span style="color: #557799;">#include <doctest/doctest.h> </span><span style="color: #888888;">//doctest testing framework</span>
<span style="color: #557799;">#include <runner.h> </span><span style="color: #888888;">//Simplifies main()</span>
MAIN(){
<span style="color: #008800; font-weight: bold;">const</span> <span style="color: #333399; font-weight: bold;">int</span> argc_ <span style="color: #333333;">=</span> <span style="color: #0000dd; font-weight: bold;">3</span>;
<span style="color: #008800; font-weight: bold;">const</span> <span style="color: #333399; font-weight: bold;">char</span> <span style="color: #333333;">*</span>argv_[] <span style="color: #333333;">=</span> {
<span style="background-color: #fff0f0;">"exe"</span>,
<span style="background-color: #fff0f0;">"-d"</span>,
<span style="background-color: #fff0f0;">"-s"</span>};
<span style="color: #008800; font-weight: bold;">return</span> doctest<span style="color: #333333;">::</span>Context(argc_, argv_).run();
}
<span style="color: #557799;">#include <calculator.h></span>
Calculator calc;
TEST_CASE(<span style="background-color: #fff0f0;">"calculator addition"</span>){
CHECK(<span style="color: #0000dd; font-weight: bold;">32</span><span style="color: #333333;">==</span> calc.add(<span style="color: #0000dd; font-weight: bold;">25</span>, <span style="color: #0000dd; font-weight: bold;">7</span>));
}
TEST_CASE(<span style="background-color: #fff0f0;">"calculator subtraction"</span>){
CHECK(<span style="color: #0000dd; font-weight: bold;">20</span> <span style="color: #333333;">==</span> calc.sub(<span style="color: #0000dd; font-weight: bold;">23</span>, <span style="color: #0000dd; font-weight: bold;">3</span>));
}
TEST_CASE(<span style="background-color: #fff0f0;">"calculator multiplication"</span>){
CHECK(<span style="color: #0000dd; font-weight: bold;">50</span> <span style="color: #333333;">==</span> calc.mul(<span style="color: #0000dd; font-weight: bold;">25</span>, <span style="color: #0000dd; font-weight: bold;">2</span>));
}
TEST_CASE(<span style="background-color: #fff0f0;">"calculator division"</span>){
CHECK(<span style="color: #0000dd; font-weight: bold;">32</span> <span style="color: #333333;">==</span> calc.div(<span style="color: #0000dd; font-weight: bold;">96</span>, <span style="color: #0000dd; font-weight: bold;">3</span>));
}
</pre></div>
</div><div><a href="https://github.com/drorgl/esp32-test-development/blob/b454e1123bddedc324e245b45fb783705aa56a5f/esp32-unit-test-doctest/test/test_common/test_calculator.cpp">test_calculator.cpp</a></div><div><br /></div><h4 style="text-align: left;">CppUTest</h4><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #557799;">#include <runner.h> </span><span style="color: #888888;">//Simplifies main()</span>
<span style="color: #557799;">#define CPPUTEST_USE_LONG_LONG 1 </span><span style="color: #888888;">//mandatory for cpputest to work with esp32</span>
<span style="color: #557799;">#include "CppUTest/CommandLineTestRunner.h"</span>
<span style="color: #557799;">#include "CppUTest/TestPlugin.h"</span>
<span style="color: #557799;">#include "CppUTest/TestRegistry.h"</span>
<span style="color: #557799;">#include "CppUTestExt/IEEE754ExceptionsPlugin.h"</span>
<span style="color: #557799;">#include "CppUTestExt/MockSupportPlugin.h"</span>
MAIN(){
<span style="color: #008800; font-weight: bold;">const</span> <span style="color: #333399; font-weight: bold;">char</span> <span style="color: #333333;">*</span> argv_[] <span style="color: #333333;">=</span> {
<span style="background-color: #fff0f0;">""</span>
<span style="background-color: #fff0f0;">""</span>,
<span style="background-color: #fff0f0;">"-v"</span>,
<span style="background-color: #fff0f0;">"-c"</span>,
<span style="background-color: #fff0f0;">"-o"</span>,
<span style="background-color: #fff0f0;">"eclipse"</span>
<span style="color: #888888;">//"teamcity"//"eclipse"//"junit"</span>
};
<span style="color: #008800; font-weight: bold;">return</span> CommandLineTestRunner<span style="color: #333333;">::</span>RunAllTests(<span style="color: #0000dd; font-weight: bold;">5</span>, argv_);
}
<span style="color: #557799;">#include <calculator.h></span>
Calculator calc;
TEST_GROUP(Calculator){ };
TEST(Calculator, Addition){
CHECK(<span style="color: #0000dd; font-weight: bold;">32</span><span style="color: #333333;">==</span> calc.add(<span style="color: #0000dd; font-weight: bold;">25</span>, <span style="color: #0000dd; font-weight: bold;">7</span>));
}
TEST(Calculator, Subtraction){
CHECK(<span style="color: #0000dd; font-weight: bold;">20</span> <span style="color: #333333;">==</span> calc.sub(<span style="color: #0000dd; font-weight: bold;">23</span>, <span style="color: #0000dd; font-weight: bold;">3</span>));
}
TEST(Calculator, Multiplication){
CHECK(<span style="color: #0000dd; font-weight: bold;">50</span> <span style="color: #333333;">==</span> calc.mul(<span style="color: #0000dd; font-weight: bold;">25</span>, <span style="color: #0000dd; font-weight: bold;">2</span>));
}
TEST(Calculator, Division){
CHECK(<span style="color: #0000dd; font-weight: bold;">32</span> <span style="color: #333333;">==</span> calc.div(<span style="color: #0000dd; font-weight: bold;">96</span>, <span style="color: #0000dd; font-weight: bold;">3</span>));
}<span style="font-family: Times New Roman;"><span style="white-space: normal;">
</span></span></pre></div>
</div><div><a href="https://github.com/drorgl/esp32-test-development/blob/b454e1123bddedc324e245b45fb783705aa56a5f/esp32-unit-test-cpputest/test/test_common/test_calculator.cpp">test_calculator.cpp</a></div><div><br /></div><div>If you're migrating to PlatformIO and already use a different framework which is not on the supported list, you might be interested to know how to setup PlatformIO to work with <a href="https://docs.platformio.org/en/latest/advanced/unit-testing/frameworks/custom/index.html">custom framework</a>. </div><div><br /></div><h3 style="text-align: left;">Environments</h3><div>To use PlatformIO effectively, we'll need to tell it which setups its going to work with, such as <a href="https://docs.platformio.org/en/latest/platforms/index.html">Platforms</a>, <a href="https://docs.platformio.org/en/latest/boards/index.html">Boards</a> and <a href="https://docs.platformio.org/en/stable/frameworks/index.html">Frameworks</a>, the combination of these can be grouped into <a href="https://docs.platformio.org/en/stable/projectconf/section_env.html">environments</a> in <a href="https://docs.platformio.org/en/latest/projectconf/index.html">platformio.ini</a>, however, environments can contain <a href="https://docs.platformio.org/en/stable/projectconf/section_env.html">more than that</a>. </div><div><br /></div><div>Lets define two environments, one for ESP32 and one for natively running on the development machine.</div><div><br /></div><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #008800; font-weight: bold;">[env:esp32]</span>
<span style="color: #0000cc;">platform</span> <span style="color: #333333;">=</span> <span style="background-color: #fff0f0;">espressif32</span>
<span style="color: #0000cc;">board</span> <span style="color: #333333;">=</span> <span style="background-color: #fff0f0;">esp32doit-devkit-v1</span>
<span style="color: #0000cc;">framework</span> <span style="color: #333333;">=</span> <span style="background-color: #fff0f0;">espidf</span>
<span style="color: #008800; font-weight: bold;">[env:native]</span>
<span style="color: #0000cc;">platform</span> <span style="color: #333333;">=</span> <span style="background-color: #fff0f0;">native</span>
</pre></div>
</div><h3 style="text-align: left;">Running Tests</h3><div><a href="https://docs.platformio.org/en/stable//core/userguide/cmd_test.html">Running tests in PlatformIO</a> is pretty simple, this command will run all tests in all environments, so if we have a native and ESP32 environment defined, it will executes all tests against them.</div><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">pio <span style="color: #007020;">test</span>
</pre></div>
</div><div><br /></div><div>But lets say we want to run only native environment:</div><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">pio <span style="color: #007020;">test</span> -e native
</pre></div>
</div><div><br /></div><div>Lastly, some labs have the device connected to a <a href="https://docs.platformio.org/en/stable//core/userguide/remote/cmd_test.html">remote agent</a> and shared by multiple developers or even a CI agent</div><div><br /></div><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">pio remote <span style="color: #007020;">test</span>
</pre></div>
</div><h3 style="text-align: left;">Limiting Tests to Specific Environments</h3><div>We can always use ifdef guards to build tests for specific platforms but for the sake of order we're probably going to want to group common, embedded and desktop tests. I propose that we'll have 3 separate folders for our sample tests and tell PlatformIO to ignore the incompatible tests on the incompatible platforms.</div><div><br /></div><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #008800; font-weight: bold;">[env:esp32]</span>
<span style="color: #0000cc;">platform</span> <span style="color: #333333;">=</span> <span style="background-color: #fff0f0;">espressif32</span>
<span style="color: #0000cc;">board</span> <span style="color: #333333;">=</span> <span style="background-color: #fff0f0;">esp32doit-devkit-v1</span>
<span style="color: #0000cc;">framework</span> <span style="color: #333333;">=</span> <span style="background-color: #fff0f0;">espidf</span>
<b><span style="color: #0000cc;">test_ignore</span> <span style="color: #333333;">=</span> <span style="background-color: #fff0f0;">test_desktop</span></b>
<span style="color: #008800; font-weight: bold;">[env:native]</span>
<span style="color: #0000cc;">platform</span> <span style="color: #333333;">=</span> <span style="background-color: #fff0f0;">native</span>
<b><span style="color: #0000cc;">test_ignore</span> <span style="color: #333333;">=</span> <span style="background-color: #fff0f0;">test_embedded</span></b>
</pre></div>
</div><div><br /></div><div>Now that we have our tests working, lets see how the results look like</div><div><br /></div><div>
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">>pio test -e native
Verbosity level can be increased via `-v, -vv, or -vvv` option
Collected 3 tests
Processing test_common in native environment
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Building...
Testing...
test\test_common\test_calculator.cpp:49: test_function_calculator_addition [PASSED]
test\test_common\test_calculator.cpp:50: test_function_calculator_subtraction [PASSED]
test\test_common\test_calculator.cpp:51: test_function_calculator_multiplication [PASSED]
test\test_common\test_calculator.cpp:52: test_function_calculator_division [PASSED]
------------------------------------------------------------------------------------ native:test_common [PASSED] Took 2.75 seconds ------------------------------------------------------------------------------------
Processing test_desktop in native environment
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Building...
Testing...
test\test_desktop\test_calculator.cpp:49: test_function_calculator_addition [PASSED]
test\test_desktop\test_calculator.cpp:50: test_function_calculator_subtraction [PASSED]
test\test_desktop\test_calculator.cpp:51: test_function_calculator_multiplication [PASSED]
test\test_desktop\test_calculator.cpp:44: test_function_calculator_division: Expected 32 Was 33 [FAILED]
------------------------------------------------------------------------------------ native:test_desktop [FAILED] Took 2.42 seconds ------------------------------------------------------------------------------------
======================================================================================================= SUMMARY =======================================================================================================
Environment Test Status Duration
------------- ------------ -------- ------------
native test_common PASSED 00:00:02.753
native test_desktop FAILED 00:00:02.421
_________________________________________________________________________________________________ native:test_desktop _________________________________________________________________________________________________
test\test_desktop\test_calculator.cpp:44:test_function_calculator_division:FAIL: Expected 32 Was 33
</pre></div>
</div><div><br /></div><h3 style="text-align: left;">Emulators</h3><div>PlatformIO has <a href="https://docs.platformio.org/en/stable//advanced/unit-testing/simulators/index.html">integrated a few emulators</a>, unfortunately none of them is for ESP32, however, <a href="https://github.com/espressif/qemu">Espressif has worked on QEMU</a>, which might make it into the supported emulators one day.</div><div><br /></div><div><h3>Multithreading</h3><p>Testing multithreaded code does not technically constitutes a unit test, however, sometimes we want to verify the integration between components bridged by <a href="https://en.wikipedia.org/wiki/Software_design_pattern#Concurrency_patterns">concurrency patterns</a>.</p><p>More over, unit testing multithreaded code can lead to issues, such as irreproducible or sparse failures with no certain way to check why and in turn lead to tests being ignored or deleted.</p><p>In general, when developing multithreaded code the main issues raise when threads share memory and data, depending on architecture and word size, accessing unaligned variables and structs usually compiles to multiple instructions that access and dissect the aligned memory into smaller chunks which will cause issues if multiple threads access that data since the context switch can occur between instructions.</p><p>With that being said, if you must test multithreaded code, you'll need to control the timing or wait for something to happen and not depend on sleeps and delays since it will make your code non-deterministic.</p><p>If you must share data between threads, do it with the appropriate concurrency pattern, such as messages, queues and lists.</p><p>You may find more interesting patterns with <a href="https://www.etlcpp.com/documentation.html">etl</a>.</p><h3>RTOS</h3><p>RTOS use on a microcontroller can enable higher quality code by separating responsibilities to individual tasks and functions, allowing them to interact safely, the downside of that is the added complexity of working with RTOS. The <a href="https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/system/freertos.html">esp-idf has integrated FreeRTOS</a> already for you, saving some of the learning curve but I do recommend reading the FreeRTOS documentation and even going into some of its source code to understand exactly what's going on, if you really need it, FreeRTOS has been ported to <a href="https://www.freertos.org/FreeRTOS-Windows-Simulator-Emulator-for-Visual-Studio-and-Eclipse-MingW.html">Windows</a>/<a href="https://www.freertos.org/FreeRTOS-simulator-for-Linux.html">Linux</a> so you test a part of your code on your development machine.</p><p><br /></p><div style="text-align: left;"><a href="https://blog.drorgluska.com/2022/06/platformio-static-code-analysis.html">Continue to static code analysis... </a></div><p><br /></p></div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3077621944057198788.post-82085313769758906552021-11-15T12:29:00.001-08:002021-11-15T12:41:41.334-08:00USART with DMA on STM32<p> I've been working with many projects that use the USART and not one was like the other alghough hardware resources were pretty similar. </p><p>So I've sat down and decided to make a boilerplate for USART with DMA implementation that uses binary semaphores to notify when data arrives and buffers the output to create as little delay as possible as well as leave as much CPU as possible for the rest of the system.</p><p>For this demo I'll be using the STM32F446 Nucleo-64.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjUOuKlqVm7mWAwiRo5JluqFNEg-Yv4YAWdk7xeKvBdoxKmqKPYGA8Cy1-oi2iY2ppk6Im3Y1nu_1hzN8EVwzGVkG_JXyJ_EKJ9SAUn8YV1Chxn7_vRWl8kUX_Fnz-1JMR5zXCwjS_fE1k//" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1280" data-original-width="1280" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjUOuKlqVm7mWAwiRo5JluqFNEg-Yv4YAWdk7xeKvBdoxKmqKPYGA8Cy1-oi2iY2ppk6Im3Y1nu_1hzN8EVwzGVkG_JXyJ_EKJ9SAUn8YV1Chxn7_vRWl8kUX_Fnz-1JMR5zXCwjS_fE1k/w400-h400/image.png" width="400" /></a></div><br /><br /><p></p><p>By default, it has the USART2 pins connected to the on board ST-Link so its possible to just open a terminal, watch logs and send commands to the MCU with as little effort as possible.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgnN3nnmmDYvfzZiEEr7prugmNPsZ_xhoDHHfAMlGPyvAi853uSBbzcm24A_09pu86gvvvxJGQJftkQt9pdzeObNQzmNEYpnVlEp9LLM-0JNdea_EDjpVYPLmFBF-_fJSQK6FU9Impk8Xw//" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="642" data-original-width="774" height="531" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgnN3nnmmDYvfzZiEEr7prugmNPsZ_xhoDHHfAMlGPyvAi853uSBbzcm24A_09pu86gvvvxJGQJftkQt9pdzeObNQzmNEYpnVlEp9LLM-0JNdea_EDjpVYPLmFBF-_fJSQK6FU9Impk8Xw/w640-h531/image.png" width="640" /></a></div><br />Once we have the basics setup in the IDE and the USART2 Enabled as Asynchronous, We'll go ahead and add DMA Channels:<p></p><p></p><div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcArQoJ6uJ9kFJmTNyUMjg74eD9C_KmAlp0p2YVQRPAJue-Pf8nvKHkwotkHR9t-g4v98ZGX64f00-zRFDZqgkljtF7BqJPjdqF-F-XfXLQs9xTntiAswnUwHiq_BUvRxHl-SO7NFi8Gc//" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="335" data-original-width="739" height="290" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcArQoJ6uJ9kFJmTNyUMjg74eD9C_KmAlp0p2YVQRPAJue-Pf8nvKHkwotkHR9t-g4v98ZGX64f00-zRFDZqgkljtF7BqJPjdqF-F-XfXLQs9xTntiAswnUwHiq_BUvRxHl-SO7NFi8Gc/w640-h290/image.png" width="640" /></a></div><br /><br /></div>One for read, one for write and set them both to Normal mode.<div><br /></div><div>Enable global interrupts:</div><div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEilU4jfwlML7ZnujJS_RJIu4NFQrKm8V5Us25sC91J5gknX4bGT8oQn2VPWxv11V5tJud7pL-Tn8at1wHalZdoIg2hZr_RApvJAz5zVw973QqZz4VCcPeGGYNB8Y2_r4j4vyciHT-2IgU0//" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="134" data-original-width="493" height="174" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEilU4jfwlML7ZnujJS_RJIu4NFQrKm8V5Us25sC91J5gknX4bGT8oQn2VPWxv11V5tJud7pL-Tn8at1wHalZdoIg2hZr_RApvJAz5zVw973QqZz4VCcPeGGYNB8Y2_r4j4vyciHT-2IgU0/w640-h174/image.png" width="640" /></a></div><br /><br /><p></p><p><br />We then go ahead and add FreeRTOS, so we can demo a general application:</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2hXTaUk_ARE81PIX2KCvwHBwa1KW20GwPGMKXGjD655dBBGTZLzBX3kThtvmQe-2nIoSuFN5a6mem2vkw3okZVPNloGaDDyaLlE2CnkUaiHLo6FA4-EDpIdaLJY1Hn2J-5ndm_YKh_l4//" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="841" data-original-width="742" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2hXTaUk_ARE81PIX2KCvwHBwa1KW20GwPGMKXGjD655dBBGTZLzBX3kThtvmQe-2nIoSuFN5a6mem2vkw3okZVPNloGaDDyaLlE2CnkUaiHLo6FA4-EDpIdaLJY1Hn2J-5ndm_YKh_l4/w565-h640/image.png" width="565" /></a></div><br />And go ahead and USE_NEWLIB_REENTRANT so we can use printf:<p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNVNi8cWlyzTPmAi99_pGsrTeRtQfpse5ZkkhFtnLmkAxZjUP6esKSknovvUhgpgZ8JFHd2VrwkVf2W6murI8-1PmblxGQmZON2AtWkpFyNja9bZGm9e2iwvtMzvZXQbAjTBaRaatBnpU//" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="206" data-original-width="755" height="174" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNVNi8cWlyzTPmAi99_pGsrTeRtQfpse5ZkkhFtnLmkAxZjUP6esKSknovvUhgpgZ8JFHd2VrwkVf2W6murI8-1PmblxGQmZON2AtWkpFyNja9bZGm9e2iwvtMzvZXQbAjTBaRaatBnpU/w640-h174/image.png" width="640" /></a></div><br />And lastly we'll go to project manager and mark the Generate peripheral initialization as pair of '.c/.h' files per peripheral for just to keep our application a bit cleaner:<p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi2iO3caERqNapoPYV2Fh46V5lf-NcA0QwEQcwBdecR6tzggk3jXjQWo4_LBFJsef5On3_KCCqDrW_iXPYjxHTZoVUxudFIygtZ3V5G_MqYhoq-9_wSQcwK9TnYs90dil9iA_M5kvy6cT8//" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="572" data-original-width="1273" height="288" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi2iO3caERqNapoPYV2Fh46V5lf-NcA0QwEQcwBdecR6tzggk3jXjQWo4_LBFJsef5On3_KCCqDrW_iXPYjxHTZoVUxudFIygtZ3V5G_MqYhoq-9_wSQcwK9TnYs90dil9iA_M5kvy6cT8/w640-h288/image.png" width="640" /></a></div><div><br /></div><div>A known bug (<a href="https://blog.domski.pl/adc-initialization-order-bug-in-cubemx/">1</a>,<a href="https://community.st.com/s/question/0D50X0000BcS7tSSQS/stmcubeide-110-cubemx-540-switch-initialization-order">2</a>,<a href="https://community.st.com/s/question/0D53W00000GZpgtSAD/cubemx-adc-and-dma-initialized-in-the-wrong-order">3</a>,<a href="https://community.st.com/s/question/0D50X0000Bmob3uSQA/dma-not-working-in-cubemx-generated-code-order-of-initialization">4</a>) in HAL generated projects is that the DMA is not initialized in order, a simple solution will be to duplicate the DMA initialization call to the 'USER CODE BEGIN SysInit' section in main.c so whenever the project is regenerated, the change won't get lost.</div><div><br />
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><table><tbody><tr><td><pre style="line-height: 125%; margin: 0px;">1
2
3</pre></td><td><pre style="line-height: 125%; margin: 0px;"><span style="color: #888888;">/* USER CODE BEGIN SysInit */</span>
MX_DMA_Init();
<span style="color: #888888;">/* USER CODE END SysInit */</span>
</pre></td></tr></tbody></table></div>
</div><div><br /></div><div><br /></div>Once our project is generated, we'll add a circular buffer of choice, in this case I've chosen to use <a href="https://github.com/MaJerle">Tilen Majerle</a>'s <a href="https://github.com/MaJerle/lwrb">lwrb</a> - Lightweight ring buffer manager.</div><div><br /></div><div><div>Next in our usart.c, we'll add 2 semaphores for the tx and rx buffers, 2 aligned buffers for the DMA and 2 buffers for rx and tx, we'll use our "USER CODE BEGIN 0" for that so we'll keep them when the project is regenerated through STM32CubeMX/IDE. </div><div><br /></div><div>Feel free to change the buffer sizes, though for my needs I didn't see a reason to go higher.</div><div><br />
<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><table><tbody><tr><td><pre style="line-height: 125%; margin: 0px;"> 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38</pre></td><td><pre style="line-height: 125%; margin: 0px;"><span style="color: #888888;">/* USER CODE BEGIN 0 */</span>
<span style="color: #557799;">#include <cmsis_os.h></span>
<span style="color: #557799;">#include "lwrb/lwrb.h"</span>
<span style="color: #008800; font-weight: bold;">static</span> SemaphoreHandle_t readSemaphore;
<span style="color: #008800; font-weight: bold;">static</span> osSemaphoreId writeSemaphore;
<span style="color: #557799;">#define TX_DMA_BUFFER_SIZE 16</span>
__aligned(<span style="color: #0000dd; font-weight: bold;">32</span>) <span style="color: #333399; font-weight: bold;">uint8_t</span> TX_DMA_buffer[TX_DMA_BUFFER_SIZE];
<span style="color: #557799;">#define RX_DMA_BUFFER_SIZE 16</span>
__aligned(<span style="color: #0000dd; font-weight: bold;">32</span>) <span style="color: #333399; font-weight: bold;">uint8_t</span> RX_DMA_buffer[RX_DMA_BUFFER_SIZE];
<span style="color: #333399; font-weight: bold;">lwrb_t</span> rx_buffer;
<span style="color: #333399; font-weight: bold;">uint8_t</span> rx_buffer_container[<span style="color: #0000dd; font-weight: bold;">255</span>];
<span style="color: #333399; font-weight: bold;">lwrb_t</span> tx_buffer;
<span style="color: #333399; font-weight: bold;">uint8_t</span> tx_buffer_container[<span style="color: #0000dd; font-weight: bold;">255</span>];
<span style="color: #333399; font-weight: bold;">void</span> <span style="color: #0066bb; font-weight: bold;">initialize_buffers</span>(<span style="color: #333399; font-weight: bold;">void</span>) {
osSemaphoreDef(WRITESEM);
writeSemaphore <span style="color: #333333;">=</span> osSemaphoreCreate(osSemaphore(WRITESEM), <span style="color: #0000dd; font-weight: bold;">1</span>);
vSemaphoreCreateBinary(readSemaphore);
<span style="color: #008800; font-weight: bold;">if</span> (readSemaphore <span style="color: #333333;">==</span> <span style="color: #007020;">NULL</span>) {
Error_Handler();
}
<span style="color: #008800; font-weight: bold;">if</span> (lwrb_init(<span style="color: #333333;">&</span>rx_buffer, rx_buffer_container, <span style="color: #008800; font-weight: bold;">sizeof</span>(rx_buffer_container)) <span style="color: #333333;">!=</span> <span style="color: #0000dd; font-weight: bold;">1</span>){
Error_Handler();
}
<span style="color: #008800; font-weight: bold;">if</span> (lwrb_init(<span style="color: #333333;">&</span>tx_buffer, tx_buffer_container, <span style="color: #008800; font-weight: bold;">sizeof</span>(tx_buffer_container)) <span style="color: #333333;">!=</span> <span style="color: #0000dd; font-weight: bold;">1</span>){
Error_Handler();
}
}
<span style="color: #888888;">/* USER CODE END 0 */</span>
</pre></td></tr></tbody></table></div>
</div><div><br /></div><div>Note we included also our buffer initialization routine in the header.</div><div><p>Next we'll add the DMA start in our MX_USART2_UART_Init function in usart.c:</p><p>
<!--HTML generated using hilite.me--></p><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><table><tbody><tr><td><pre style="line-height: 125%; margin: 0px;">1
2
3
4</pre></td><td><pre style="line-height: 125%; margin: 0px;"> <span style="color: #888888;">/* USER CODE BEGIN USART2_Init 2 */</span>
HAL_UARTEx_ReceiveToIdle_DMA(<span style="color: #333333;">&</span>huart2, RX_DMA_buffer, RX_DMA_BUFFER_SIZE);
__HAL_DMA_DISABLE_IT(<span style="color: #333333;">&</span>hdma_usart2_rx, DMA_IT_HT);
<span style="color: #888888;">/* USER CODE END USART2_Init 2 */</span>
</pre></td></tr></tbody></table></div>
<p></p><p>Thanks for the tip about DMA_IT_HT from <a href="https://controllerstech.com/uart-dma-with-idle-line-detection/">ControllersTech</a>.</p><p>Next we'll add our USART tx/rx functions in usart.c. If you're wondering about the xSemaphoreGiveFromISR at line 23, its used to notify the waiting thread about new data rather than continuous polling that will either waste CPU time or cause a delay between received bytes until the thread realizes it.</p><p>
<!--HTML generated using hilite.me--></p><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><table><tbody><tr><td><pre style="line-height: 125%; margin: 0px;"> 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65</pre></td><td><pre style="line-height: 125%; margin: 0px;"><span style="color: #888888;">/* USER CODE BEGIN 1 */</span>
<span style="color: #008800; font-weight: bold;">static</span> <span style="color: #333399; font-weight: bold;">int</span> <span style="color: #0066bb; font-weight: bold;">tx_next_chunk</span>(<span style="color: #333399; font-weight: bold;">void</span>) {
<span style="color: #333399; font-weight: bold;">int</span> number_of_items_in_tx_buffer <span style="color: #333333;">=</span> lwrb_read(<span style="color: #333333;">&</span>tx_buffer, TX_DMA_buffer, TX_DMA_BUFFER_SIZE);
<span style="color: #008800; font-weight: bold;">if</span> (number_of_items_in_tx_buffer <span style="color: #333333;">></span> <span style="color: #0000dd; font-weight: bold;">0</span>) {
<span style="color: #008800; font-weight: bold;">if</span> (HAL_UART_Transmit_DMA(<span style="color: #333333;">&</span>huart2, TX_DMA_buffer,
number_of_items_in_tx_buffer) <span style="color: #333333;">!=</span> HAL_OK) {
assert(<span style="color: #0000dd; font-weight: bold;">0</span>);
}
__HAL_DMA_DISABLE_IT(<span style="color: #333333;">&</span>hdma_usart2_rx, DMA_IT_HT);
}
<span style="color: #008800; font-weight: bold;">return</span> number_of_items_in_tx_buffer;
}
<span style="color: #333399; font-weight: bold;">void</span> <span style="color: #0066bb; font-weight: bold;">HAL_UARTEx_RxEventCallback</span>(UART_HandleTypeDef <span style="color: #333333;">*</span>huart, <span style="color: #333399; font-weight: bold;">uint16_t</span> Size) {
<span style="color: #008800; font-weight: bold;">if</span> (huart<span style="color: #333333;">-></span>Instance <span style="color: #333333;">==</span> USART2) {
<span style="color: #008800; font-weight: bold;">if</span> (lwrb_write(<span style="color: #333333;">&</span>rx_buffer, RX_DMA_buffer, Size) <span style="color: #333333;">!=</span> Size ){
<span style="color: #888888;">//buffer overrun</span>
}
HAL_UARTEx_ReceiveToIdle_DMA(huart, RX_DMA_buffer, RX_DMA_BUFFER_SIZE);
BaseType_t xHigherPriorityTaskWoken;
xSemaphoreGiveFromISR(readSemaphore,<span style="color: #333333;">&</span>xHigherPriorityTaskWoken);
}
}
<span style="color: #333399; font-weight: bold;">int</span> <span style="color: #0066bb; font-weight: bold;">get_rx_data</span>(<span style="color: #333399; font-weight: bold;">uint8_t</span> <span style="color: #333333;">*</span>buffer, <span style="color: #333399; font-weight: bold;">size_t</span> buffer_length, <span style="color: #333399; font-weight: bold;">uint32_t</span> timeout) {
xSemaphoreTake(readSemaphore,pdMS_TO_TICKS(timeout ));
<span style="color: #008800; font-weight: bold;">return</span> lwrb_read(<span style="color: #333333;">&</span>rx_buffer, buffer, buffer_length);
}
<span style="color: #333399; font-weight: bold;">void</span> <span style="color: #0066bb; font-weight: bold;">HAL_UART_TxCpltCallback</span>(UART_HandleTypeDef <span style="color: #333333;">*</span>huart) {
<span style="color: #008800; font-weight: bold;">if</span> (huart<span style="color: #333333;">-></span>Instance <span style="color: #333333;">==</span> USART2) {
tx_next_chunk();
}
}
<span style="color: #333399; font-weight: bold;">void</span> <span style="color: #0066bb; font-weight: bold;">put_tx_data_with_wait</span>(<span style="color: #333399; font-weight: bold;">uint8_t</span> <span style="color: #333333;">*</span>buffer, <span style="color: #333399; font-weight: bold;">size_t</span> buffer_length) {
<span style="color: #333399; font-weight: bold;">int</span> retries <span style="color: #333333;">=</span> <span style="color: #0000dd; font-weight: bold;">1000</span>;
<span style="color: #008800; font-weight: bold;">while</span> (retries <span style="color: #333333;">></span> <span style="color: #0000dd; font-weight: bold;">0</span>) {
<span style="color: #333399; font-weight: bold;">int</span> pushed_bytes <span style="color: #333333;">=</span> put_tx_data(buffer, buffer_length);
buffer_length <span style="color: #333333;">-=</span> pushed_bytes;
buffer <span style="color: #333333;">+=</span> pushed_bytes;
<span style="color: #008800; font-weight: bold;">if</span> (buffer_length <span style="color: #333333;"><=</span> <span style="color: #0000dd; font-weight: bold;">0</span>) {
<span style="color: #008800; font-weight: bold;">break</span>;
}
osDelay(<span style="color: #0000dd; font-weight: bold;">1</span>);
retries<span style="color: #333333;">--</span>;
}
}
<span style="color: #333399; font-weight: bold;">int</span> <span style="color: #0066bb; font-weight: bold;">put_tx_data</span>(<span style="color: #333399; font-weight: bold;">uint8_t</span> <span style="color: #333333;">*</span>buffer, <span style="color: #333399; font-weight: bold;">size_t</span> buffer_length) {
<span style="color: #333399; font-weight: bold;">int</span> ret <span style="color: #333333;">=</span> <span style="color: #0000dd; font-weight: bold;">0</span>;
<span style="color: #008800; font-weight: bold;">if</span> (osSemaphoreWait(writeSemaphore, osWaitForever) <span style="color: #333333;">==</span> osOK) {
ret <span style="color: #333333;">=</span> lwrb_write(<span style="color: #333333;">&</span>tx_buffer, buffer, buffer_length);
osSemaphoreRelease(writeSemaphore);
}
<span style="color: #008800; font-weight: bold;">if</span> (huart2.gState <span style="color: #333333;">==</span> HAL_UART_STATE_READY) {
tx_next_chunk();
}
<span style="color: #008800; font-weight: bold;">return</span> ret;
}
<span style="color: #888888;">/* USER CODE END 1 */</span>
</pre></td></tr></tbody></table></div>
<p></p><p>And our function prototypes in usart.h:</p><p>
<!--HTML generated using hilite.me--></p><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><table><tbody><tr><td><pre style="line-height: 125%; margin: 0px;">1
2
3
4
5</pre></td><td><pre style="line-height: 125%; margin: 0px;"><span style="color: #888888;">/* USER CODE BEGIN Prototypes */</span>
<span style="color: #333399; font-weight: bold;">void</span> <span style="color: #0066bb; font-weight: bold;">put_tx_data_with_wait</span>(<span style="color: #333399; font-weight: bold;">uint8_t</span> <span style="color: #333333;">*</span>buffer, <span style="color: #333399; font-weight: bold;">size_t</span> buffer_length);
<span style="color: #333399; font-weight: bold;">int</span> <span style="color: #0066bb; font-weight: bold;">put_tx_data</span>(<span style="color: #333399; font-weight: bold;">uint8_t</span> <span style="color: #333333;">*</span>buffer, <span style="color: #333399; font-weight: bold;">size_t</span> buffer_length);
<span style="color: #333399; font-weight: bold;">int</span> <span style="color: #0066bb; font-weight: bold;">get_rx_data</span>(<span style="color: #333399; font-weight: bold;">uint8_t</span> <span style="color: #333333;">*</span>buffer, <span style="color: #333399; font-weight: bold;">size_t</span> buffer_length, <span style="color: #333399; font-weight: bold;">uint32_t</span> timeout);
<span style="color: #888888;">/* USER CODE END Prototypes */</span>
</pre></td></tr></tbody></table></div>
<p></p><p>And lastly we'll create our echo demo in StartDefaultTask in our freertos.c:</p><p>
<!--HTML generated using hilite.me--></p><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><table><tbody><tr><td><pre style="line-height: 125%; margin: 0px;"> 1
2
3
4
5
6
7
8
9
10
11</pre></td><td><pre style="line-height: 125%; margin: 0px;"><span style="color: #333399; font-weight: bold;">void</span> <span style="color: #0066bb; font-weight: bold;">StartDefaultTask</span>(<span style="color: #333399; font-weight: bold;">void</span> <span style="color: #008800; font-weight: bold;">const</span> <span style="color: #333333;">*</span> argument)
{
<span style="color: #888888;">/* USER CODE BEGIN StartDefaultTask */</span>
<span style="color: #008800; font-weight: bold;">while</span> (<span style="color: #0000dd; font-weight: bold;">1</span>){
<span style="color: #333399; font-weight: bold;">uint8_t</span> temp_buffer[<span style="color: #0000dd; font-weight: bold;">64</span>];
<span style="color: #333399; font-weight: bold;">size_t</span> read_bytes;
read_bytes <span style="color: #333333;">=</span>get_rx_data(temp_buffer, <span style="color: #008800; font-weight: bold;">sizeof</span>(temp_buffer), <span style="color: #0000dd; font-weight: bold;">100</span>);
put_tx_data_with_wait(temp_buffer,read_bytes);
}
<span style="color: #888888;">/* USER CODE END StartDefaultTask */</span>
}
</pre></td></tr></tbody></table></div>
<p></p><p>What the demo does is essentially waiting for up to 64 bytes or 100ms and transmitting back what it got. so this thread is waiting most of the time, the DMA does most of the work and the ring buffer is just there to make sure everything plays together nicely.</p><p>The demo project can be found here:</p><p><a href="https://github.com/drorgl/usart-boilerplate">https://github.com/drorgl/usart-boilerplate</a></p><p><br /></p><p><br /></p><p><br /></p></div></div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3077621944057198788.post-28891619819155218582019-11-05T02:59:00.003-08:002023-01-12T07:13:58.685-08:00Introduction to ESP32 Debugging<span style="color: red;">Note: This article is a prelude to a talk I'm having about ESP32 Unit Testing and Debugging on November 27th 2019.</span><br />
<br />
If you read any of my previous articles you could probably guess I'm not a big fan of debugging. I truly believe that production should not be debugged (with exceptions) and therefore, its better to change one's thought process and build beneficial logging abilities.<br />
<br />
But probably the most widely used debugging 'technology' is the printf way, by locating the crash or stack trace, the magical printf can tell us the current state which lead to the bug or crash and we can fix it. If we're thorough, we'll probably add a unit test to avoid that bug in the future.<br />
<br />
As much as I would like it to be, in embedded systems, logging is not always realistic, it can affect timing, occupy UARTS and kill <a href="https://hackaday.com/2019/10/17/worn-out-emmc-chips-are-crippling-older-teslas/">eMMC</a>.<br />
<br />
So what are our other options?<br />
<br />
Sometimes the we only need an indication something is not happening, an "if" statement we're not sure is actually happening, how about using a GPIO to turn a led on?<br />
<br />
Two more relatively fast options are I²C and SPI, we can use a very simple program that dumps the values being sent.<br />
<br />
To further improve the logging abilities of these facilities, you can encode only the values, rather than a text log message.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi8YBZp3eHX5lhMXsLP-P4HNvonlo3jnI5yZnLpCw2uhEl8Z7SdG4PUF4Y_t_Yb_jvJWYWFSjYta5hOkcgOYnI6ls4KrzSn63MhGaLWLxGbFJxBEdXYwlJ8Qk5iONU9b_IFn8nMUt_Htqo/s1600/debugging3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="651" data-original-width="1200" height="346" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi8YBZp3eHX5lhMXsLP-P4HNvonlo3jnI5yZnLpCw2uhEl8Z7SdG4PUF4Y_t_Yb_jvJWYWFSjYta5hOkcgOYnI6ls4KrzSn63MhGaLWLxGbFJxBEdXYwlJ8Qk5iONU9b_IFn8nMUt_Htqo/s640/debugging3.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<h3>
Debugging Port</h3>
<br />
JTAG has been around for quite some time (1990!), these days most MCUs have a debugging port, be it JTAG, SWD or debugWire (AVR).<br />
JTAG in particular is very capable, its designed to be chained across all the chips, processors and DSPs on the board, so a single port can be used to debug many components.<br />
<br />
<h4>
Logging</h4>
While printf debugging can provide a short term or a localized debugging option, as developers, we need to consider longer term and production problem solving and these solutions either keep state changes in a log file by either saving rolling logs or by having some sort of circular buffer of logging messages.<br />
<br />
While logging is pretty straight forward implementation, the ESP32 logging facilities provides a few interesting points:<br />
1. logs are divided by <a href="https://docs.espressif.com/projects/esp-idf/en/latest/api-reference/system/log.html#how-to-use-this-library">TAG</a>s<br />
2. logs can be turned <a href="https://docs.espressif.com/projects/esp-idf/en/latest/api-reference/system/log.html#_CPPv417esp_log_level_setPKc15esp_log_level_t">off/on/set logging level</a> by each tag<br />
3. internal esp-idf components also have a log tag<br />
4. printf is always sent to UART0<br />
5. <a href="https://docs.espressif.com/projects/esp-idf/en/latest/api-reference/system/log.html#_CPPv419esp_log_set_vprintf14vprintf_like_t">logs can be captured</a>, this is one of the more interesting features since it allows you, as a developer to have a device in the field that is misbehaving and you can turn on logging remotely and ask for the log files.<br />
<br />
While not directly related to logging, the ESP32 and FreeRTOS provides a few more interesting mechanisms for debugging problems:<br />
1. <a href="https://docs.espressif.com/projects/esp-idf/en/latest/api-reference/system/system.html#_CPPv416esp_reset_reasonv">get reset reason</a>, this is very important, think of it as extra information you can write to your logs when the device starts, did it reboot because of power failure? brownout? watchdog?<br />
2. <a href="https://docs.espressif.com/projects/esp-idf/en/latest/api-guides/core_dump.html">Core Dump</a>, when the device in the field, who is going to monitor the stack trace? it cannot be written to the log file, nothing is usually monitoring the UARTs, so where does it go? you can configure a core dump to place it on the flash, so next time you're asking for logs, you can also retrieve the core dump and analyze it.<br />
3.<a href="https://docs.espressif.com/projects/esp-idf/en/latest/api-reference/system/mem_alloc.html#_CPPv418heap_caps_get_infoP17multi_heap_info_t8uint32_t"> FreeRTOS Memory Analysis</a>, <a href="https://docs.espressif.com/projects/esp-idf/en/latest/api-reference/system/mem_alloc.html#_CPPv429heap_caps_check_integrity_allb">heap corruption</a>,<a href="https://docs.espressif.com/projects/esp-idf/en/latest/api-reference/system/freertos.html#_CPPv427uxTaskGetStackHighWaterMark12TaskHandle_t"> maximum stack use</a>, maximum heap use and even <a href="https://docs.espressif.com/projects/esp-idf/en/latest/api-reference/system/heap_debug.html#_CPPv416heap_trace_start17heap_trace_mode_t">memory tracing</a> similar to crtdbg.<br />
4. <a href="http://vtaskgetruntimestats/">FreeRTOS CPU Utilization</a><br />
<h3>
<br />PlatformIO Unified Debugging</h3>
PlatformIO became my favorite development platform, its simplistic, near zero configuration and simple extensibility gives one ability to do almost anything with very little effort.<br />
In ESP32 case, the openocd-esp32 and esp-idf are integrated with its unified debugger, making it so simple, I just had to add one line to platformio.ini:<br />
<span style="font-family: "courier new" , "courier" , monospace;"><a href="https://docs.platformio.org/en/latest/projectconf/section_env_debug.html#debug-tool">debug_tool</a> = esp-prog</span><br />
or<br />
<span style="font-family: "courier new" , "courier" , monospace;"><a href="https://docs.platformio.org/en/latest/projectconf/section_env_debug.html#debug-tool">debug_tool</a> = jlink</span><br />
<br />
But that will only get you so far, if you start the debugger, the esp32 debugging configuration is missing, so you'll need to add a debug env as well with build flags to add debug symbols to the firmware:<br />
<span style="font-family: "courier new" , "courier" , monospace;">build_flags = -<a href="https://gcc.gnu.org/onlinedocs/gcc/Debugging-Options.html#Debugging-Options">ggdb</a> </span><br />
<br />
<h3>
JTAG</h3>
JTAG is a standard debugging port, its common with most of the modern systems, it can help you to physically test a board using Boundary Scan, Stop and Start CPU cores, read and write variables and memory, add breakpoints, read and write registers, execute code and commands and even write firmware.<br />
<br />
Unfortunately ESP32 does not provide boundary scan capabilities, but you can achieve that and more if you have your test fixture flash MicroPython, script <a href="https://sigrok.org/">logic analyzer</a> commands and analyze the results.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEinaUyS8sE-dGACDww95lWhS4Iun8KQ_Ant4pvEtW7ifNcWAO-Ox5xbhLlWS3vhTHKdyt1emek0t-j9moxTv7NYqyvRHQxr_1XS49BxfNePEZzLeN0coI_9IsQ-TbG7UxA3wHBNpkB7HUc/s1600/esp-wroom-v1%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1600" data-original-width="1067" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEinaUyS8sE-dGACDww95lWhS4Iun8KQ_Ant4pvEtW7ifNcWAO-Ox5xbhLlWS3vhTHKdyt1emek0t-j9moxTv7NYqyvRHQxr_1XS49BxfNePEZzLeN0coI_9IsQ-TbG7UxA3wHBNpkB7HUc/s400/esp-wroom-v1%255B1%255D.jpg" width="266" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://www.espressif.com/en/products/hardware/production-testing-equipment">source</a></td></tr>
</tbody></table>
<br />
<br />
The rest of the features JTAG is enabling are great and on top of that the Tensilica TRAX module enhances debugging facilities by adding real-time log tracing and even FreeRTOS event tracing.<br />
<br />
<h3>
So what is TRAX?</h3>
TRAX is TRace Analyzer for Xtensa, is a module that the CPU and JTAG share to transfer data between the host and Tensilica Processor.<br />
<br />
With that in mind, we can use that data for almost anything, Espressif provided us with two interesting examples, <a href="https://docs.espressif.com/projects/esp-idf/en/latest/api-guides/app_trace.html#application-specific-tracing">trace logs</a> and <a href="https://docs.espressif.com/projects/esp-idf/en/latest/api-guides/app_trace.html#system-behavior-analysis-with-segger-systemview">FreeRTOS events</a>, but the sky is the limit.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjJuGIIULYhnFUMrL2Agy-LB7OaqBBesUAoLvkPf8g77OaxJdSybRf8THYVPpdR7Pzzc_Hqpiua4CZmGpfKIWgVJ1xov8ReMG0HxJ4dcdmN0W6db_FaMPPsHQ-GjoitOZh7I1d615RXWXM/s1600/TRAX.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="688" data-original-width="1114" height="245" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjJuGIIULYhnFUMrL2Agy-LB7OaqBBesUAoLvkPf8g77OaxJdSybRf8THYVPpdR7Pzzc_Hqpiua4CZmGpfKIWgVJ1xov8ReMG0HxJ4dcdmN0W6db_FaMPPsHQ-GjoitOZh7I1d615RXWXM/s400/TRAX.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<h3>
Getting Started</h3>
This is the ESP32-DevKitC, it's one of the most popular ESP32 development kits, its a low-footprint board with the essentials, it comes with either WROOM or WROVER modules. Its drawback is the lack of JTAG connectors, but you can add it by <a href="https://docs.espressif.com/projects/esp-idf/en/latest/api-guides/jtag-debugging/configure-other-jtag.html#configure-hardware">wiring directly to the pins</a>.<br />
<div style="direction: ltr;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLl7X8P4Vu8h-TbJAM3UbvINBbt0BPnIxMGIz9SV6E12jyB0j7MT1oWEwf_mnrTrbru60tVQAx003MGT89mRQaOdLGFXQm0LpfltN1pNOglM-gN3I3lOt4oIK-ynJLDzXqNSkhrzgmFsM/s1600/esp32-devkitc-functional-overview%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="540" data-original-width="960" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLl7X8P4Vu8h-TbJAM3UbvINBbt0BPnIxMGIz9SV6E12jyB0j7MT1oWEwf_mnrTrbru60tVQAx003MGT89mRQaOdLGFXQm0LpfltN1pNOglM-gN3I3lOt4oIK-ynJLDzXqNSkhrzgmFsM/s640/esp32-devkitc-functional-overview%255B1%255D.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="font-size: 12.8px;"><a href="https://docs.espressif.com/projects/esp-idf/en/latest/hw-reference/get-started-devkitc.html">source</a></td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_YCzTjxBrCWI5Uld8JADvZrZLP8GRXp7UQbzdT_wEBB6H6iAwasrQsIdNHAqC59N1E_yxUg8_b0ngLpDSn6iISrZX4e-Q9OruPnIv658SexsFyAqMPtWyPleYck9EpqaHcsiQzZAX_L4/s1600/20191029_215114_HDR.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1393" data-original-width="1600" height="347" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_YCzTjxBrCWI5Uld8JADvZrZLP8GRXp7UQbzdT_wEBB6H6iAwasrQsIdNHAqC59N1E_yxUg8_b0ngLpDSn6iISrZX4e-Q9OruPnIv658SexsFyAqMPtWyPleYck9EpqaHcsiQzZAX_L4/s400/20191029_215114_HDR.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://www.tindie.com/products/drorgl/esp32-devkit-jtag-debugging-adapter/">Get yours here</a></td></tr>
</tbody></table>
<br />
On the other hand, this adapter exposes the JTAG pins in both 10pin esp-prog format and 20pin standard JTAG / Segger J-Link format, it can stack between the DevKit and your breadboard or development PCB, it made me a lot less lazy connecting the debugger, is it a positive or a negative thing, you decide.<br />
<br />
In the end of this article you can find other options from Espressif.<br />
<br />
As a side note, I've experienced different problems with different debuggers, the Segger J-Link would freeze every once in a while, needing a complete disconnect and power down of both the debugger and devkit, the FT2232 based debuggers would succeed to upload the sketch through the J-Link interface but it was an inconsistent experience.<br />
<br />
So how to debug?<br />
<br />
1. compile and upload the firmware using -ggdb flag.<br />
2. in VSCode, go to Debug View, click PIO Debug (skip Pre-Debug), wait about 10-20 seconds and your first breakpoint will be caught.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgmH7W-ITKyXezL9rTfYU9lMztPrXjBH0lWLuUSnhUBvHw_7FRJ-YazyDvPnwU-3SnKyS9LfnquWoI_-1s_6fDX5UcKC-XBYuD91qC-vq4hZAa0-CxglmQY0tQPvmNlUZpuvCJ5pP0GE90/s1600/vscode.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="768" data-original-width="1024" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgmH7W-ITKyXezL9rTfYU9lMztPrXjBH0lWLuUSnhUBvHw_7FRJ-YazyDvPnwU-3SnKyS9LfnquWoI_-1s_6fDX5UcKC-XBYuD91qC-vq4hZAa0-CxglmQY0tQPvmNlUZpuvCJ5pP0GE90/s640/vscode.png" width="640" /></a></div>
<br />
We have a few interesting points here.<br />
1. The top left PIO Debug will start the debugger, you should switch to the lower right debug console tab to see progress and execute debugger commands.<br />
2. Debugger specific sidebar where Variables, Watch, Call Stack, Breakpoints etc' are visible.<br />
3. The gutter in the editor can set a breakpoint or conditional breakpoint, please note that since conditional breakpoint is implemented in the debugger, the execution will be paused each time the breakpoint is hit and evaluated, this affects timing and performance.<br />
4. Top right bar shows debugger controls, Continue, Step Over, Step In, etc', Note that they might not work if no hardware breakpoint is available.<br />
<br />
<br />
The Debug Console view in VSCode exposes GDB, I'm saving it for my next article, its going to be about ESP32 log tracing and event tracing abilities, exciting!<br />
<br />
<h3>
Debugging Supported ESP32 Development Kits</h3>
1. <a href="https://www.espressif.com/en/products/hardware/esp-wrover-kit/overview">ESP-WROVER-KIT</a> - JTAG on board (using FT2232HL chip)<br />
ESP-WROVER-KIT-VB is a highly integrated ultra-low-power development board which includes Flash and PSRAM with dual-core 240 MHz CPU.<br />
Create Internet cameras, smart displays or Internet radios by connecting LCDs, microphones and codecs to it.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKwzJgBht2xi2j77nwj11r4xBg5Ux5wBhyphenhyphenNEHaueNYjYxDdrdC4wM7vWnDFqDYU5lW8NMWeXnJG2MhQm2ZAdfHdjrCkL89ggalGqFth1gh36EwoZbkcPStb_FFwAtUO7UBBu8XgoCXEc8/s1600/esp-wrover-kit-v4.1-layout-front%255B1%255D.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="576" data-original-width="864" height="426" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKwzJgBht2xi2j77nwj11r4xBg5Ux5wBhyphenhyphenNEHaueNYjYxDdrdC4wM7vWnDFqDYU5lW8NMWeXnJG2MhQm2ZAdfHdjrCkL89ggalGqFth1gh36EwoZbkcPStb_FFwAtUO7UBBu8XgoCXEc8/s640/esp-wrover-kit-v4.1-layout-front%255B1%255D.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://docs.espressif.com/projects/esp-idf/en/latest/hw-reference/get-started-wrover-kit.html">source</a></td></tr>
</tbody></table>
<br />
<div>
2. <a href="https://www.espressif.com/en/products/hardware/esp32-lyratd-msc">ESP32-LyraTD-MSC</a> - JTAG connector</div>
<div>
<div>
Designed for smart speakers and AI applications. Supports Acoustic Echo Cancellation (AEC), Automatic Speech Recognition (ASR), Wake-up Interrupt and Voice Interaction.</div>
</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigrnbagiIgkZkoKCGeBxgTdi7nVHIvonTv7ZiUlJCQSk1F3YmY125xt8d326TVuOIGnHyLS3jfUUR07b77Lec8veYH8K3wGn3Eqz1OdvsDtUrKfxoRNHb67HCuhcfNpgXQOrCPUdpLpxI/s1600/esp32-lyratd-msc-v2.2-a-top%255B1%255D.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="672" data-original-width="960" height="448" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigrnbagiIgkZkoKCGeBxgTdi7nVHIvonTv7ZiUlJCQSk1F3YmY125xt8d326TVuOIGnHyLS3jfUUR07b77Lec8veYH8K3wGn3Eqz1OdvsDtUrKfxoRNHb67HCuhcfNpgXQOrCPUdpLpxI/s640/esp32-lyratd-msc-v2.2-a-top%255B1%255D.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://docs.espressif.com/projects/esp-adf/en/latest/get-started/get-started-esp32-lyratd-msc.html">source</a></td></tr>
</tbody></table>
<div>
3. <a href="https://docs.espressif.com/projects/esp-idf/en/latest/hw-reference/get-started-ethernet-kit.html">ESP32-Ethernet-Kit</a> - JTAG on board (using FT2232HL chip)</div>
<div>
Consists of two development boards, the Ethernet board A and the PoE board B</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_vwI4fOkzuVudF5e8snookDQ_ZeZZ-xSUdqppArLAaHxcuBBYFhI2W3x3NyVJIIhad668pvnooiDmwGYoafmop1atjlpwREShO8gtbs0mpV7qbVHJTlkJMAoovll2J-Wv3jIMs5IBNMY/s1600/esp32-ethernet-kit-a-v1.0-layout%255B1%255D.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="528" data-original-width="864" height="390" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_vwI4fOkzuVudF5e8snookDQ_ZeZZ-xSUdqppArLAaHxcuBBYFhI2W3x3NyVJIIhad668pvnooiDmwGYoafmop1atjlpwREShO8gtbs0mpV7qbVHJTlkJMAoovll2J-Wv3jIMs5IBNMY/s640/esp32-ethernet-kit-a-v1.0-layout%255B1%255D.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://docs.espressif.com/projects/esp-idf/en/latest/hw-reference/get-started-ethernet-kit.html">source</a></td></tr>
</tbody></table>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3077621944057198788.post-10981493753334518042019-07-17T13:04:00.001-07:002019-07-17T13:19:05.933-07:00Is ESP32 Ready for some AI?IoT is a passion of mine for quite some time, so imagine how happy I was to receive a gift from <a href="https://www.semix.co.il/">Semix</a>, the all new <a href="https://www.espressif.com/en/products/hardware/esp-eye/overview">ESP-EYE</a> v2.1!<br />
<br />
<a href="https://www.semix.co.il/">Semix</a> specializes in representation and distribution of world leading manufacturers of Electronic Components, Modules and Integrated solutions in Israel, in this case <a href="https://espressif.com/">Espressif</a> and <a href="https://www.macnica.eu/">Manica</a>.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMIVUsmaDxMV54RK7Citx8M5xDXLXaFSMxsfgvUD3-prZc2cHpWK6YLNJYxRvSfzSs4eOgmubA4GL5Rfna0eH3TYH1pbwxIqzvytyq2sMXXavFCwpxWMRt7HVvtyUiViPy8cUZwVSttFM/s1600/esp-eye-tf.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="843" data-original-width="1600" height="336" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMIVUsmaDxMV54RK7Citx8M5xDXLXaFSMxsfgvUD3-prZc2cHpWK6YLNJYxRvSfzSs4eOgmubA4GL5Rfna0eH3TYH1pbwxIqzvytyq2sMXXavFCwpxWMRt7HVvtyUiViPy8cUZwVSttFM/s640/esp-eye-tf.png" width="640" /></a></div>
<br />
<br />
While the board looks like it was pretty thought out (ferrite beads all over it!), it does lack GPIO connections, looking like it was directly made to demonstrate the ESP32 capabilities rather than a maker Swiss army knife. In the end of this article there are some other options if you're curious about combining these capabilities with your other crazy ideas. :-)<br />
<br />
I've started by looking up some <a href="https://github.com/espressif/esp-who/blob/master/docs/en/get-started/ESP-EYE_Getting_Started_Guide.md">information</a>, <a href="https://www.youtube.com/watch?v=_JwMuvFVMsM">videos</a>, <a href="https://www.espressif.com/sites/default/files/documentation/ESP-EYE_V2.1_Reference_Design_1.zip">design reference</a> and anything I can find on that module and eventually I've cloned the <a href="https://github.com/espressif/esp-who">esp-who project</a>.<br />
<br />
I've followed a few getting started examples but I really love what <a href="https://platformio.org/">PlatformIO</a> did with Visual Studio Code so I had to set it up to compile in PlatformIO. BTW, PlatformIO already has the <a href="https://docs.platformio.org/en/latest/frameworks/espidf.html">esp-idf</a> framework, which makes it very easy to use with ESP32!<br />
<br />
You'd be surprised how much faster a good IDE can help you understand a project structure!<br />
<br />
Eventually I got curious enough to see how they did it so I've begun to dig up a bit. the face recognition part seems to be based on <a href="http://cn.arxiv.org/ftp/arxiv/papers/1604/1604.02878.pdf">MTCNN</a>, where it's actually 3 separate networks integrated with algorithm glue.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg97HKV-DTC1hW8H-T7AFX3gBPKT6P6f80NzDP3PQ7Wvs0fB7zX4f-L-lHH46SXp6tiZsWAoGGFydUK88bZ5hp7JYo1pyLNnnps_HSIdc3u7zqsfMHJcwKBSNkxjNvzUxPimjH4DUjINIY/s1600/overview%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="758" data-original-width="1373" height="352" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg97HKV-DTC1hW8H-T7AFX3gBPKT6P6f80NzDP3PQ7Wvs0fB7zX4f-L-lHH46SXp6tiZsWAoGGFydUK88bZ5hp7JYo1pyLNnnps_HSIdc3u7zqsfMHJcwKBSNkxjNvzUxPimjH4DUjINIY/s640/overview%255B1%255D.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://github.com/espressif/esp-who">source</a></td></tr>
</tbody></table>
The audio keyword recognition seems to be very similar to TensorFlow demo I've seen, but during my research I've seen a few other examples that gave me the impression Espressif did not use TensorFlow.<br />
<br />
However, <a href="https://www.tensorflow.org/lite">TensorFlow lite</a> could be used for another project I was doing research for, so I've decided to take the plunge and see if I can compile it for ESP32.<br />
<br />
The getting started is pretty straight forward, download, compile, run and of-curse learn. but to really get started you need to get your feet wet and test the hardware compatibility since TensorFlow lite was not specifically ported to ESP32. So what do you do? you run the suite tests on the ESP32.<br />
<br />
Let me assure you, all the tests passed, some did take some time to complete but that's because it wasn't optimized for tensilica yet.<br />
<br />
I've also run the <a href="https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/experimental/micro/examples/micro_speech">micro speech</a> demo, but since it wasn't optimized, it took 360ms to process 100ms of audio. I did find some optimizations for the inference engine and boom, this thing is fast! (80ms for FFT + inference!!)<br />
<br />
If you'd like you can find more <a href="https://www.tensorflow.org/lite/models">pretrained models</a> and <a href="https://www.tensorflow.org/lite/examples">examples</a> in TensorFlow website.<br />
<br />
To me this little exploration opened a whole new world and ideas of AI on ESP32, if you had any doubts, you should definitely check out TensorFlow lite on ESP32!<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7Q1PeuvZfdjrl-iQ0tJhcz_cIYNokkDVEYPLKb1iPfsskAgCXsencLWIuL7CdQM6kq-FIgsmdDDcHNiHDEDUgHcId__Lw-IJUcN3PUnb3CCCA18Di73vdb8-vWNA8egoZ6tQqduL-fkM/s1600/esp-ai.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="819" data-original-width="1024" height="318" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7Q1PeuvZfdjrl-iQ0tJhcz_cIYNokkDVEYPLKb1iPfsskAgCXsencLWIuL7CdQM6kq-FIgsmdDDcHNiHDEDUgHcId__Lw-IJUcN3PUnb3CCCA18Di73vdb8-vWNA8egoZ6tQqduL-fkM/s400/esp-ai.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://www.flickr.com/photos/mikemacmarketing/30188200627">source</a></td></tr>
</tbody></table>
<br />
Please note that there are other variations of the ESP-EYE (or ESP-CAM) with different capabilities:<br />
<br />
1. <a href="http://s.click.aliexpress.com/e/KRomCEU">ESP32-CAM</a><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjEvfFR42JG49Yj_wL6Nm3ytB65viqZEETOIBOEPEW9xioS2VcYsawK5YWvgqixIIQ_DPdSVTPZacPcvYjieFvBZzQR64ERU8h9Rlvd0DtVXeFfyFw7iZPTQt6qj40bSTvdyQ69Tn_8lRs/s1600/ESP32-CAM-ESP-32S-WiFi-Module-ESP32-serial-to-WiFi-ESP32-CAM-Development-Board-5V-Bluetooth%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="800" data-original-width="800" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjEvfFR42JG49Yj_wL6Nm3ytB65viqZEETOIBOEPEW9xioS2VcYsawK5YWvgqixIIQ_DPdSVTPZacPcvYjieFvBZzQR64ERU8h9Rlvd0DtVXeFfyFw7iZPTQt6qj40bSTvdyQ69Tn_8lRs/s400/ESP32-CAM-ESP-32S-WiFi-Module-ESP32-serial-to-WiFi-ESP32-CAM-Development-Board-5V-Bluetooth%255B1%255D.jpg" width="400" /></a></div>
The ESP32-CAM also has 4MB of external PSRAM, it exposes some GPIOs for extensibility and even has an SD-CARD slot but no Mic.<br />
Notice there's no USB plug, so you'll need an external USB-TTL adapter to program this device.<br />
<br />
2. <a href="http://s.click.aliexpress.com/e/RIXA13S">M5Stack Official ESP32 Camera Development Board</a><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhIMbcUNRhlAxvf8H8uyoAa9srhdstrcKLwUoK_vENlfQ1gymz0sGb8i5Ikxr4P9kYlff0wKrskDnuT2agUSD-5uDmzfyMfg5DAQ3VlYgYSROGixVwH3YUNDcFNCPuB6OoGC8kB0LBlkVQ/s1600/M5Stack-Official-ESP32-Camera-Development-Board-OV2640-Camera-Type-C-Grove-Port-3D-Wifi-Antenna-Mini%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="700" data-original-width="700" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhIMbcUNRhlAxvf8H8uyoAa9srhdstrcKLwUoK_vENlfQ1gymz0sGb8i5Ikxr4P9kYlff0wKrskDnuT2agUSD-5uDmzfyMfg5DAQ3VlYgYSROGixVwH3YUNDcFNCPuB6OoGC8kB0LBlkVQ/s320/M5Stack-Official-ESP32-Camera-Development-Board-OV2640-Camera-Type-C-Grove-Port-3D-Wifi-Antenna-Mini%255B1%255D.jpg" width="320" /></a></div>
<br />
Almost looks like a copy,I couldn't find any reference of external PSRAM, so if anyone knows, leave your comments please.<br />
<br />
There are empty footprints for MPU6050, BME280, Mic and lithium battery connection, so it can be easily used for your wearable projects.<br />
<br />
Notice it has USB-C connector.<br />
<br />
3. <a href="http://s.click.aliexpress.com/e/cNlGKi8">TTGO T-Camera Plus</a><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzmWVBPnMa0PJSz7ul5_062YyCXu6JMHJOR3K_InrGs35bVqJMHk4WOWTcJwfrY0W67K18mP1y2F6UYNVVAFxgYT3W_uE1wHcAMLID2wkccKMTIv0X9kxVfEJGdVvhPkjozucHQFqOP1c/s1600/TTGO-T-Camera-Plus-ESP32-DOWDQ6-8MB-SPRAM-Camera-Module-OV2640-1-3-Inch-Display-Rear%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1000" data-original-width="1000" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzmWVBPnMa0PJSz7ul5_062YyCXu6JMHJOR3K_InrGs35bVqJMHk4WOWTcJwfrY0W67K18mP1y2F6UYNVVAFxgYT3W_uE1wHcAMLID2wkccKMTIv0X9kxVfEJGdVvhPkjozucHQFqOP1c/s320/TTGO-T-Camera-Plus-ESP32-DOWDQ6-8MB-SPRAM-Camera-Module-OV2640-1-3-Inch-Display-Rear%255B1%255D.jpg" width="320" /></a></div>
<br />
That thing is sweet!<br />
8MB of PSRAM(!!!)<br />
1.3 inch LCD<br />
Microphone<br />
SD card slot<br />
Battery connection/charger<br />
and a USB connection.<br />
<br />
4. <a href="http://s.click.aliexpress.com/e/EJL3yre">TTGO T-Camera ESP32-WROVER-B</a><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLnIYN_teH9-G6ZL_Hbly2nTQMaeTWn0Bf9thdPDyUM0nPX4smntroQeij4WzxfeqIo60KA-IDG4pM3Ht-UZaJbdqmf4APCsLdHl-KQnh84DJsoyaquxsRswSDbaYI6-6ssGqW5uMEml0/s1600/TTGO-T-Camera-ESP32-WROVER-B-OV2640-Camera-Module-ESP32-WROVER-PSRAM-Camera-Module-0-96%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1000" data-original-width="1000" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLnIYN_teH9-G6ZL_Hbly2nTQMaeTWn0Bf9thdPDyUM0nPX4smntroQeij4WzxfeqIo60KA-IDG4pM3Ht-UZaJbdqmf4APCsLdHl-KQnh84DJsoyaquxsRswSDbaYI6-6ssGqW5uMEml0/s400/TTGO-T-Camera-ESP32-WROVER-B-OV2640-Camera-Module-ESP32-WROVER-PSRAM-Camera-Module-0-96%255B1%255D.jpg" width="400" /></a></div>
Another notable module, very similar to the T-Camera Plus.<br />
We all know that video/imaging takes power, how are we expected to write low power applications when our MCU takes most of our power? well, if your particular application doesn't require you to always scan your camera, you can put your MCU to sleep and wake it up only when there's movement with a simple PIR sensor.<br />
<br />
<br />
<br />Unknownnoreply@blogger.com6tag:blogger.com,1999:blog-3077621944057198788.post-9599005359784814262019-03-08T12:39:00.001-08:002019-03-21T07:37:36.195-07:00A Million TimesA while ago someone at work approached me with an idea to build a replica of "<a href="https://www.humanssince1982.com/a-million-times">A Million Times</a>" by <a href="http://humanssince1982.com/">Humans Since 1982</a>, while the project did eventually die off as far as I know, the idea looked very interesting, many clocks, synchronized to display animation, text and time, what can be bad about it? or as the original creator wrote:<br />
<br />
<strong style="background-color: white; font-family: "Helvetica Neue", Arial, sans-serif; font-size: 13px; overflow-wrap: break-word;"><em style="overflow-wrap: break-word;">"Metaphorically speaking, we liberated the clock from its sole function of measuring and reporting the time by taking the clock hands out of their 'administrative' roles and turning them into dancers." </em></strong><span style="background-color: white; font-family: "helvetica neue" , "arial" , sans-serif; font-size: 13px;">– </span><em style="background-color: white; font-family: "Helvetica Neue", Arial, sans-serif; font-size: 13px; overflow-wrap: break-word;">Humans since 1982</em><br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZoxpI1qLexhyIlJ8QkcZ7McydNz-hvXhuN58AW0Jt-s2TtlWBERxL2F2tUYxazXnvMef7T4ZnQyzLX39lrVXKoP5SPHimgOJYCIhFMl_JOP3FX4BgUA_UAiuVMcTb5hpi0oqL8zbGlNA/s1600/HumansSince1982_Studio_Periphery%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1001" data-original-width="1500" height="426" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZoxpI1qLexhyIlJ8QkcZ7McydNz-hvXhuN58AW0Jt-s2TtlWBERxL2F2tUYxazXnvMef7T4ZnQyzLX39lrVXKoP5SPHimgOJYCIhFMl_JOP3FX4BgUA_UAiuVMcTb5hpi0oqL8zbGlNA/s640/HumansSince1982_Studio_Periphery%255B1%255D.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://www.humanssince1982.com/a-million-times-at-changi">A Million Times at Changi, 2014-2018</a></td></tr>
</tbody></table>
While they did not expose much of the design for their work, <a href="https://www.facebook.com/spirail">David Cox</a>, who is the engineer of this project, shared a few hints in his <a href="https://www.facebook.com/spirail">facebook</a>:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQRi4JzJXWhhKcYSe7_PRN3RKnjA1ZQLm9rS83X3kRqYYC8mKmrf71HIDZjljrhsNOYL_qbQmFlFVzrI4Bvp2mrC8QEW_mfUmUYhap-VKwTe4hMspvzGkYcPjMvUJ5ccLyOe5KH2uuFQo/s1600/1505503_10152708555028438_704475474_n%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="540" data-original-width="960" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQRi4JzJXWhhKcYSe7_PRN3RKnjA1ZQLm9rS83X3kRqYYC8mKmrf71HIDZjljrhsNOYL_qbQmFlFVzrI4Bvp2mrC8QEW_mfUmUYhap-VKwTe4hMspvzGkYcPjMvUJ5ccLyOe5KH2uuFQo/s640/1505503_10152708555028438_704475474_n%255B1%255D.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://www.facebook.com/photo.php?fbid=10152708555028438&set=a.10150919866383438&type=3&theater">Source</a></td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div style="text-align: center;">
<iframe height="480" src="https://drive.google.com/file/d/1VIjzOWBrwZTki4pv1kASBk88qjHXpF2b/preview" width="640"></iframe>
</div>
<br />
<div style="text-align: center;">
<span style="font-size: x-small;"><a href="https://www.facebook.com/spirail/videos/10152376091153438/">source</a></span></div>
<br />
From what I could deduce, the project is probably using 2 types of MCUs, one to control each of the motors (such as ATtiny85) and another to control the whole block (ATmega of sort) and then connected via USB to a PC to control the entire assembly.<br />
<br />
This article has been collecting links and paragraphs for quite some time (since July 2017!), I've decided to finish it after I've started to learn more about PCB design and actually took it off the breadboard.<br />
<br />
Assuming we would like to design our own, my first thought was that I'll need to use pipes, gears, Plexiglas and plenty of patience, just like Cornelius Franz tried:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLbQNCfYZ67gnJbw9jOVYT0AiKkDQcF5qXLOaD-F3GeCFSAN2ciPxLknrL3CVjyNkVMt8Opno-Htw9hU4vnN1y9m3cJgINyOUrfsEWx-kUNdivAcrYXAaHFVPoNE6nWHM4YCAdSiBlNok/s1600/cf+1st+attempt.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="373" data-original-width="800" height="298" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLbQNCfYZ67gnJbw9jOVYT0AiKkDQcF5qXLOaD-F3GeCFSAN2ciPxLknrL3CVjyNkVMt8Opno-Htw9hU4vnN1y9m3cJgINyOUrfsEWx-kUNdivAcrYXAaHFVPoNE6nWHM4YCAdSiBlNok/s640/cf+1st+attempt.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://hackaday.io/project/21433/logs">source</a></td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
</div>
He actually implemented what I thought to make, I've had a single motor with the same driver, I've had to add a step up (since the driver needed a minimum of 8v and used an ATmega328 instead of the STM32F103), I've had to use a a higher voltage than actually needed since the motor would miss steps and the maximum speed wasn't great since I've used AccelStepper which isn't very efficient. I've also thought about using the micro-switches for addressing but i don't think its a good use of available pins, even if used with resistor ladder.<br />
<br />
I've thought about the following options for controllers:<br />
- ATmega328 for board MCU, connected to a driver, hall effect sensor and canbus, which can be used without a transceiver on short distances.<br />
- ATmega328 for board MCU, connected to a driver, hall effect sensor and i2c/spi bus<br />
- ATmega32u2/4 for USB connectivity to i2c/spi<br />
<br />
My colleague, Allan Schwartz from <span id="goog_12409859"></span><span id="goog_12409860"></span><a href="http://whatimade.today/">whatimade.today</a> suggested the i2c route, I had doubts it will work over long distance, but as it turns out there have been uses for i2c over long distances with repeaters (such as PCA9515), I still didn't get around to test the long distance repeater solution, but the datasheet does specify you may not use more than one repeater, but does it also include parallel repeaters?...) Using the i2c as a bus for the entire assembly makes things simpler over communicating with 20 or more USB virtual com ports.<br />
<br />
Different layouts, either a PCB per motor or a PCB for 4 motors, which makes things a lot simpler on one hand but won't work if I wanted to add some more visual effects such as addressable Leds.<br />
<br />
For drivers, the following options:<br />
- no driver, these motors do not consume too much power (about 20ma VERIFY), so in theory the Arduino can power it, however, when I tried it, the motor produced inconsistent movement for various speeds, I suspect due to the fact I didn't implement micro-stepping in my source code. I did find out that Wojtek Kosak did make it work without any driver, so it might have been my fault it did not work.<br />
- 2 x ULN2003 ($0.2) or DRV8836 ($1.5) (there are many alternatives, just an h-bridge) per motor<br />
- 2 x <a href="https://www.sparkfun.com/datasheets/Robotics/A3967.pdf">A3967SLB</a> ($2) per motor<br />
- <a href="http://www.ti.com/lit/ds/symlink/drv8821.pdf">DRV8821</a> ($4.5), minimum 8v which might complicate things<br />
- L298N ($1) might work<br />
- <a href="https://drive.google.com/open?id=1hRCfesf2Ihe0oj4MilSolIwfDy6p6Cb7">X12.017 stepper driver</a> / <a href="https://drive.google.com/open?id=1137watWnvGXDF86R7FXzcHmI-cPZNRBh">VID6606</a> / <a href="https://drive.google.com/open?id=1u_Wc7srNpENFZd4ywsWFC-28VznXB_E9">BY892</a>0 / AX1201728SG ($1), should be able to control 4 motors (or two dual shaft motors) - <a href="https://guy.carpenter.id.au/gaugette/2012/01/19/x12-quad-driver-chip/">source</a>, I've tested the AX1201728SG and it was very stable even in high speeds as long as acceleration control is implemented.<br />
<br />
After testing a few drivers, the one that worked best is the AX1201728SG.<br />
<br />
And for motors it turns out there are dual shaft stepper motors, the following look pretty promising:<br />
- X40 8798 Stepper Motor ($6.8) - <a href="https://drive.google.com/open?id=11_yyuCV2hs6vu28_Fv8slDz59_XZ5f9G">datasheet</a><br />
- Sonceboz 6407 (27€)<br />
- vid28-05 Stepper Motor<br />
- BKA30D-R5 ($3.8) with a stop, but it turns out the manufacturer already realized that people would like to use them without a stop, so they started manufacturing them without a stop!!<br />
<br />
Last but not least is to use a real clock and modify the circuit, the motor is <a href="https://en.wikipedia.org/wiki/Lavet-type_stepping_motor">Lavet-type stepping motor</a> and <a href="https://hackaday.com/2013/02/12/lord-vetinaris-clock-strikes-again/">someone made a crazy clock with it</a>, I love it!<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/rijmudoj39w/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/rijmudoj39w?feature=player_embedded" width="320"></iframe></div>
<br />
<br />
All in all, I think Cornelius Franz did a some amazing work, it seems like he's on a good path to have a working replica!<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/g5Qzt3z111k/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/g5Qzt3z111k?feature=player_embedded" width="320"></iframe></div>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgeH0RYmC6rZHWVTqh5PjSaOq_5swdq7RK8ibwfAqUJSEz4iYkU7Ic4ZHqyCL6EfMiBv9OggzmxSdGPlD2-fqWWwAJxVGLPuJY3QXanOM7G4NUbUUadil8EyBWOTAH6R62MMYB01L45U-E/s1600/7265391521972479432%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="600" data-original-width="800" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgeH0RYmC6rZHWVTqh5PjSaOq_5swdq7RK8ibwfAqUJSEz4iYkU7Ic4ZHqyCL6EfMiBv9OggzmxSdGPlD2-fqWWwAJxVGLPuJY3QXanOM7G4NUbUUadil8EyBWOTAH6R62MMYB01L45U-E/s640/7265391521972479432%255B1%255D.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://hackaday.io/project/21433/logs">source</a></td></tr>
</tbody></table>
<br />
I've also found out that Wojtek Kosak Główczewski actually <a href="https://hackaday.io/project/163582-digital-clock-made-of-analog-clocks">completed a replica</a>, you might want to look at his <a href="https://cdn.hackaday.io/images/5283291549015151355.png">schematic</a> and his <a href="https://easyeda.com/wkosak/analog-clock-with-bka30d-motor-and-atmega328pb-microcontroller">project</a>:<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/MNlFprewN-A/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/MNlFprewN-A?feature=player_embedded" width="320"></iframe></div>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBaZJPtKcWShaBCB0JJH7dWd9l9RL0I4DRzxFpY7BF_oFQEiWgTzL_VoYLqzVzsk4w4scj8Ry5Kuf2bI-W8cMOjLCcQ3fLMnkVAzlHY_tq7EPcKJ7jpQsQxXU0HrO4RLIXWX17nPTL96A/s1600/document-d6ab1c0c80d34497a4c43c8af4698ce1.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="420" data-original-width="437" height="307" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBaZJPtKcWShaBCB0JJH7dWd9l9RL0I4DRzxFpY7BF_oFQEiWgTzL_VoYLqzVzsk4w4scj8Ry5Kuf2bI-W8cMOjLCcQ3fLMnkVAzlHY_tq7EPcKJ7jpQsQxXU0HrO4RLIXWX17nPTL96A/s320/document-d6ab1c0c80d34497a4c43c8af4698ce1.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://easyeda.com/wkosak/analog-clock-with-bka30d-motor-and-atmega328pb-microcontroller">source</a></td></tr>
</tbody></table>
<br />
Eventually I wanted to build my own, so I went with the parts I could source from Aliexpress as it was available and didn't cost $50 to ship unlike packages from DigiKey or Mouser.<br />
<br />
I got a recommendation to check the VID28-05, however it was harder to find, it seems like they are either no longer manufactured or perhaps I didn't look very hard after finding the replica BKA30D-R5, it's also a plus that the manufacturer is on Aliexpress, its a drop in replacement anyway.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkcJCALs0NUbC4nT0Eb7CaalBWwftumFn9WFrKWS_ek4EKBmsdIlQI0jg3-YA-OyDny0hQgdtiPmTcadmlz3Br4yM3u5dondIwzaeVpY4l-hIABFrrlJ3PBuLKYNQeHsB38K5d7N7J0yU/s1600/vid28-05.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="867" data-original-width="1227" height="452" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkcJCALs0NUbC4nT0Eb7CaalBWwftumFn9WFrKWS_ek4EKBmsdIlQI0jg3-YA-OyDny0hQgdtiPmTcadmlz3Br4yM3u5dondIwzaeVpY4l-hIABFrrlJ3PBuLKYNQeHsB38K5d7N7J0yU/s640/vid28-05.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://vidmotion.com.cn/UploadFile/pdf/product/20090905103959VID28-05.pdf">source</a></td></tr>
</tbody></table>
The original BKA30D-R5 had a hard stop, which should be removed if you want to rotate it 360 degrees<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBNJytGQV9FPwg2dVnQSKCtCdB_ZZdBjP_lVV1bHlHlTLcqxvwpcOWPOCkw2TGzPFyofefA9voZw8fJZpmGY6VCU6zugeIqTmRmXqYu_kBL7_Gb82h9sQmXct7fpMOQzx8HqRhoq8_6dg/s1600/remove+stop.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1067" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBNJytGQV9FPwg2dVnQSKCtCdB_ZZdBjP_lVV1bHlHlTLcqxvwpcOWPOCkw2TGzPFyofefA9voZw8fJZpmGY6VCU6zugeIqTmRmXqYu_kBL7_Gb82h9sQmXct7fpMOQzx8HqRhoq8_6dg/s640/remove+stop.jpg" width="426" /></a></div>
<br />
However, the manufacturer took it upon themselves to supply <a href="http://s.click.aliexpress.com/e/bGlsULmu">motors without stops</a>!<br />
<br />
The vid28 series comes with a thorough <a href="https://drive.google.com/open?id=1MkbhhxYfAghMTaWHI6ohhUeeSNj2-6Do">datasheet</a>, explaining how to drive the motors, the pinout, measurements and a lot more, if you're planning to use these motors, its definitely worth to read!<br />
<br />
I've made a small breadboard with the motor, an arduino and DRV8825 and it kinda worked, I had to jack up the voltage to 12v (outside the specs) so it won't miss any steps and I've even tried the A4988, but it produced a high pitched noise.<br />
<br />
<div style="text-align: center;">
<iframe height="480" src="https://drive.google.com/file/d/1_u3vZDUyw_4DepF8gMcdG2GEdhTkzwIA/preview" width="640"></iframe>
</div>
<br />
So Allan attempted the same thing using shift register (<a href="http://www.ti.com/lit/ds/symlink/sn74hc595.pdf">74HC595</a>), from a video he sent me, I saw it was missing some steps and made a-lot of noise, I suspect its due to the lack of micro-stepping.<br />
<br />
To zero the hands, I chose a hall effect sensor + 2mm magnets, I've attempted to use the <a href="https://drive.google.com/open?id=1NIMShmOPif4piAl3SteaVB_vxM94gruA">SS49E</a> but it turned out to be not sensitive enough (1.8mV/G), so I'm now attempting to use the <a href="https://drive.google.com/open?id=1IjhviBN7ugObsU_yG5Q7X5pQ6Us78_pg">SS495</a> and while its a bit more sensitive (3.3mV/G), its a lot more expensive, so perhaps using a larger magnet or a a sort of <a href="http://fluxtrol.com/inc/pdf/Specification-and-Use-of-a-Flux-Concentrator-presentation.pdf">magnetic flux concentrator</a> will be a better solution.<br />
<br />
It did work properly on one side, I'm not sure if its the N or P, so I'm thinking about building a 3D magnetic sensor (using MLX90393) to diagnose the problem more precisely.<br />
<br />
Alternative methods can be to use a <a href="http://www.vishay.com/docs/83760/tcrt5000.pdf">reflective optical sensor</a> or a reed switch, but the magnet needed for the reed is too big and heavy to fit on the hands.<br />
<br />
I think the research we did on this project makes it relatively simple to implement hardware wise, you may find software other developers wrote in the end of this article. in my opinion,probably the easiest build will be a combination of the original PCB form factor, X12.017 driver, a hall sensor and atmega328p, wire a few assemblies with i2c and to a PC via USB.<br />
<div>
<br />
I've started to design a more modular PCB, you can shape it into cubes, spheres, towers, what ever your imagination can conjure.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzu2FafFjf-r5wjE3KIfK5tXj4pq8w74VY2mFqLzg9yXtTELWha8pkOGrv2cyUG6EAR-7MWcrQmq8urlioK5WwflqVf0QcGk6Gx_Yj0xmrGaJQPlCQ2tO5DyOK97cqf33H3WI3eFpySwk/s1600/20190308_161130_HDRs.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="981" data-original-width="1024" height="612" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzu2FafFjf-r5wjE3KIfK5tXj4pq8w74VY2mFqLzg9yXtTELWha8pkOGrv2cyUG6EAR-7MWcrQmq8urlioK5WwflqVf0QcGk6Gx_Yj0xmrGaJQPlCQ2tO5DyOK97cqf33H3WI3eFpySwk/s640/20190308_161130_HDRs.jpg" width="640" /></a></div>
<br />
The first revision was a partial success, the motor fits perfectly, the lights work, the homing more or less works.<br />
<br />
A few design issues were discovered during the first test, the brownout trips when either the leds turn on or the motor driver resets, and although I covered most of the power requirements by adjusting the trace widths, you can guess what the problem was, decoupling capacitors for example and a main capacitor.<br />
<br />
I've also used relatively heavy acrylic hands with magnets on them, so the motors missed steps here and there, I've rewrote AccelStepper to use interrupts and <a href="https://www.pmdcorp.com/resources/get/mathematics-of-motion-control-profiles-article">s-curve</a> instead of linear acceleration, which was a lot of fun and should probably affect the overall life of the motors but no noticeable difference except for top speed.<br />
<br />
I've also added the famous <a href="https://www.seeedstudio.com/document/pdf/WS2812B%20Datasheet.pdf">WS2812B</a> addressable Leds to see what else can this board do.<br />
<br />
Eventually I've decided to learn KiCad and the redesign was done from the ground up while learning, at first, I've hated it, but now I'm really enjoying myself designing PCBs!<br />
<br />
I'm not sure where this little project is going, but my key take away are the interesting research, the world of stepper motors is not a complete stranger, but this is not NEMA, magnetism and of-curse KiCad!<br />
<br />
References:<br />
<a href="https://www.humanssince1982.com/studio">Humans since 1982</a> are Bastian Bischoff (b. 1982, Germany) and Per Emanuelsson (b. 1982, Sweden). Since meeting as postgraduate students at HDK Göteborg in 2008, the duo have produced works that defy easy categorisation, situated between visual art and product design. Creating objects and experiential installations, their work manages to be analytical with a healthy dose of escapism.<br />
Facebook: https://www.facebook.com/HumansSince1982/<br />
<br />
If you want to buy one and not interested in the engineering part, just head to <a href="https://store.moma.org/home/clocks/clockclock-24----white-edition/119937-119937.html">MoMA Design</a> and you can also get a <a href="https://store.moma.org/home/clocks/clockclock-24----black-edition/125153-125153.html">black</a> one!<br />
<br />
Another interesting idea which is somewhat related is <a href="https://vimeo.com/44044873">Clock by Christiaan Postma</a>, not sure if they are related in any way, still worth a look.<br />
<br />
If you're looking only for source code:<br />
Conor Hunt was also inspired by Humans since 1982 and wrote a javascript <a href="https://codepen.io/anon/pen/VLVxxX">demo</a> and shared the <a href="https://github.com/conorh/clockr">source code</a>.<br />
Jos Fabre was also inspired and made this <a href="https://codepen.io/josfaber/pen/EpwezQ">demo</a>.<br />
Dmitry Yakimenko wrote an iOS app and published the <a href="https://github.com/detunized/ClockClock">source code</a> and a <a href="https://www.youtube.com/watch?v=M_RQJLSM4bk">demo</a>.<br />
Carlos Cabo wrote a webGL <a href="https://htmlpreview.github.io/?https://github.com/carloscabo/a-million-times-webgl-demo/blob/master/index.html">demo</a> and shared the <a href="https://github.com/carloscabo/a-million-times-webgl-demo">source code</a>.<br />
Takahashi Yuto wrote a <a href="https://y-taka-23.github.io/elm-clockclock24/">demo</a> in elm and shared the <a href="https://github.com/y-taka-23/elm-clockclock24">source code</a>.<br />
Nicolas Daniel wrote a <a href="http://nicolasdaniel.fr/clocks">demo</a> and shared the <a href="https://github.com/nicolas-daniel/the-clocks">source code</a>.<br />
Malte Wessel wrote a <a href="https://malte-wessel.github.io/a-million-times/">demo</a> and shared the <a href="https://github.com/malte-wessel/a-million-times">source code</a>.<br />
Ubayd Rahmonzoda wrote a <a href="http://rahmonzoda.github.io/clockclock/">demo</a> and shared the <a href="https://github.com/rahmonzoda/clockclock">source code</a>.<br />
<br />
<br />
There are already existing projects for trying to build a replica:<br />
https://hackaday.com/2014/09/21/ask-hackaday-how-would-you-build-a-clock-clock/<br />
https://hackaday.io/project/7872-clock-clock-clone<br />
https://hackaday.io/project/4164-clock-clock<br />
<br />
You might want to take a look at a <a href="https://www.mikrocontroller.net/topic/363437">discussion</a> from mikrocontroller.net, its pretty old but there is some progress there and people are sharing their experiences.<br />
<br />
Also, there's a <a href="https://www.tindie.com/products/TheRengineer/analog-gauge-stepper-breakout-board/">single motor with breakout</a> that someone sells on tindie.<br />
<br /></div>
Unknownnoreply@blogger.com6tag:blogger.com,1999:blog-3077621944057198788.post-84683640133243445242019-02-13T14:19:00.000-08:002019-02-13T14:19:43.225-08:00ESP32 IoT Device Management using LWM2MDevice Management means to connect, configure, control, monitor and update devices, individually or collectively.<br />
<br />
The challenge can range from managing a single or multiple devices on the same location to managing thousands spread all over the world in other cases, to complicate things further, devices can connect in various ways, either through Ethernet, WiFi,Cellular, Lora, SMS and numerous other ways. It can use a slow or fast network, it can be connected all the time or just ping your servers once a day to save power, so many things to consider.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxkIq743CbsI5naZJHbeE-yaxUqi9zs9Q_AAzKvLSxyLPW8lmMX-b4UwSIudDqulc7-KI-Lxa5HiwhFc1XiFiuQHNAk2Pilfy27liIaHWN4LAi1X-0Ywq2aupgyp3nkFF2r61bXh1qP6A/s1600/OMA-129%252520Lightweight%252520M2M%252520Logo_RGB_full%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="650" data-original-width="1600" height="258" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxkIq743CbsI5naZJHbeE-yaxUqi9zs9Q_AAzKvLSxyLPW8lmMX-b4UwSIudDqulc7-KI-Lxa5HiwhFc1XiFiuQHNAk2Pilfy27liIaHWN4LAi1X-0Ywq2aupgyp3nkFF2r61bXh1qP6A/s640/OMA-129%252520Lightweight%252520M2M%252520Logo_RGB_full%255B1%255D.jpg" width="640" /></a></div>
<br />
<br />
At about 2015 Open Mobile Alliance realized these challenges and released the first version of Lightweight Machine to Machine standard, the standard describes device management using COAP protocol with UDP/DTLS or SMS transports, recently added TCP/TLS and even MQTT was added though its not part of the standard yet.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgsGXX1pBxbp5XNno3MtEbYipc8QtW1jzGNLCwdetMZCKh-QKlz0lpx66V7VHqLOmmLPkjL-kByN401bKiRazdclHMlugOzlE6xwUBssvy-Chk1kBgDF9DQKRnN2bssxI9G1MTn6CKLEz0/s1600/lwm2m+protocol.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="460" data-original-width="1180" height="248" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgsGXX1pBxbp5XNno3MtEbYipc8QtW1jzGNLCwdetMZCKh-QKlz0lpx66V7VHqLOmmLPkjL-kByN401bKiRazdclHMlugOzlE6xwUBssvy-Chk1kBgDF9DQKRnN2bssxI9G1MTn6CKLEz0/s640/lwm2m+protocol.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://www.openmobilealliance.org/release/LightweightM2M/V1_1-20180710-A/OMA-TS-LightweightM2M_Transport-V1_1-20180710-A.pdf">Source</a></td></tr>
</tbody></table>
<br />
So how does LWM2M help you with your device management needs?<br />
<br />
LWM2M standardizes the way your devices will talk to your servers, the clients are very lean and designed to work on constrained devices, the connectivity has very low bandwidth needs and doesn't even require the devices to stay connected and that can fit a very broad spectrum of products, it can help you track your temperature sensors throughout your facility or even a buoy in the middle of the ocean, just imagine you can plan a firmware update to a device you'll never see again with maximum safety!<br />
<br />
I've chose to implement firmware updates because I had an itch to try it on ESP32, while doing it I've also implemented a basic device that can turn a light on or off and report the current time and a few other properties.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgPCV_XkOXxhJZAJrs8s1trT0pdVGwWLfBB_9Hth87hRlXcUfB5V0QWjfn_zOm-H17dnHiYMsyzwjzwGKpPjuWBAsuqdQTJTLlf4at_fuM81hwVdFaau6qDijogBbr0vZc4kqvA0d66JuM/s1600/device+object.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="606" data-original-width="957" height="404" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgPCV_XkOXxhJZAJrs8s1trT0pdVGwWLfBB_9Hth87hRlXcUfB5V0QWjfn_zOm-H17dnHiYMsyzwjzwGKpPjuWBAsuqdQTJTLlf4at_fuM81hwVdFaau6qDijogBbr0vZc4kqvA0d66JuM/s640/device+object.png" width="640" /></a></div>
<br />
The way I did it was to port <a href="https://github.com/eclipse/wakaama">wakaama</a> and <a href="https://projects.eclipse.org/projects/iot.tinydtls">tinydtls</a> to ESP32, about 90% of it worked without any modification, I've added <a href="https://github.com/zhouhan0126/WIFIMANAGER-ESP32">WiFiManager</a> and <a href="https://platformio.org/lib/show/551/NTPClient">NTP Client</a> to the mix and it just worked. (not very efficiently in terms of size though, but good enough for my experiment, the firmware was about 1.2MB in case you're wondering, out of it DTLS and Wakaama were about 100kb-200kb each and took about 6-10kb of RAM, leaving me with about 210kb of RAM which is not bad.)<br />
<br />
As soon as it started connecting to my local <a href="https://www.eclipse.org/leshan/">Leshan Server</a>, I've hooked into the firmware object the update function I've wrote, most of it based on <a href="https://docs.espressif.com/projects/esp-idf/en/latest/api-reference/system/ota.html">esp-idf OTA</a> with custom certificate validation and I've had a complete solution, the firmware and the signature would be sent over http/https, the completed firmware will be validated against the stored certificate and if it worked, the device will attempt to boot into the new firmware,a few diagnostics will be executed and if the device achieved connectivity and stability, it will mark the new firmware as valid.<br />
<br />
The fun part started when I've sent the package URI (/5/0/1) to the device through Leshan Server and Executed the update (/5/0/2), the firmware was downloaded from a local web server, the device validated the firmware and finally rebooted into the new firmware.<br />
<br />
<h4>
Why firmware over-the-air is so important?</h4>
<br />
Lets agree that time to market is a critical business need, more than once we hear about superior devices and software being ditched because a significant player already got a grip on the market and trying to push a new product is hard if not impossible at times. Many times its the first product or the first player that wins the game.<br />
<br />
To achieve a significantly short time to market compromises needs to be made, sometimes a scaled down product, less features and even compromises on a thinner layer of security are essential to get the product out of the door. Once a product gains significant traction and more resources becomes available, a better product can be developed, a more robust firmware, features as well as better security or even security vulnerabilities needs to be deployed while there is no physical access to devices. More over, device recall can kill a business and most companies try to avoid it at any cost. literally.<br />
<br />
This is where firmware over-the-air comes in, assuming the device can connect to some kind of network, either by WiFI, or GPRS/3G or even SMS and LORA, it should be able to pull a firmware update when needed or it becomes available.<br />
<br />
<h4>
How does firmware over-the-air works?</h4>
<br />
Once a device has connectivity, it can either pull a firmware, for example from HTTP server or can be pushed a firmware, for example through COAP blockwise transfers, lets assume its relatively simple to implement it or already has a library available.<br />
<br />
To support firmware updates more than once, the device will keep track on two partitions and switch between them every time the firmware is updated, this way the old firmware is kept until the device determines the new firmware is good enough to switch to permanently.<br />
<br />
Lets drill down to the specifics of one way OTA can be implemented<br />
<br />
<br />
<ol>
<li>a device is configured with 2 application partitions, one for the factory application and a 2nd is left empty for future update.</li>
<li>a device is notified of an available firmware, the OTA process can start.</li>
<li>the device determines the next OTA partition for use, if it just went out of the factory, the 2nd partition is empty, if OTA was completed successfully, the first partition is available for the next update.</li>
<li>firmware is downloaded directly to the available partition.</li>
<li>the device validates the new firmware, for example with a checksum and/or a certificate.</li>
<li>the device boots into the partition and makes sure it works as designed, so it checks the various sensors, network connectivity can still be achieved and either marks the new firmware as valid or invalid, if it crashes, either intentionally or through a watchdog, it will reboot back into the old partition.</li>
</ol>
<div>
I would argue that adding a secure validation on the new firmware is very important even if its only so it will not become another zombie in a large botnet.</div>
<div>
<br /></div>
<div>
The way I've implemented security in my experiment is as follows:</div>
<div>
<br /></div>
<div>
1. Generate a self signed certificate:<br /><span style="font-family: Courier New, Courier, monospace;">openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365</span><br />2. Extract the public key:<br /><span style="font-family: Courier New, Courier, monospace;">openssl x509 -pubkey -noout -in cert.pem > pubkey.pem</span><br />3. Copy the public key into a certificate.h file<br />4. Create a new firmware and sign it:<br /><span style="font-family: Courier New, Courier, monospace;">openssl dgst -sha256 -sign key.pem -out firmware.signature firmware.bin</span><br />5. Copy both files to a web server, set the package URI and Execute the Update.</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
Though if you want to implement secure firmware on ESP32, you can do it "<a href="https://docs.espressif.com/projects/esp-idf/en/latest/security/secure-boot.html#secure-boot-generate-key">by the book</a>".</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
OMA LWM2M Includes many device management options and includes on-boarding procedures, which is very important once your production reaches a certain number of devices since you won't be able to do it manually anymore.</div>
<div>
<br /></div>
<div>
You can monitor, configure and control your devices, you can request to be notified using the Observe mechanism and you can query the device state, send commands so the device will turn on or off a certain actuator, reset to factory defaults or even remote wipe a device if it comes to it.</div>
<div>
<br /></div>
<div>
And we can't really talk about LWM2M without talking about <a href="http://openmobilealliance.org/wp/OMNA/LwM2M/LwM2MRegistry.html">IPSO Smart Objects</a> as well, which is a list of objects defining the structure of various sensors and actuators so you won't have to do it.</div>
<div>
<br /></div>
<div>
LWM2M is a lot of fun, you should definitely consider it in your next product!</div>
Unknownnoreply@blogger.com4tag:blogger.com,1999:blog-3077621944057198788.post-15820673080873979532018-11-03T11:46:00.002-07:002018-11-03T11:46:45.081-07:00Christopher Avery - The Responsibility Process®<div style="direction: ltr;">
I've had the honor of attending <a href="https://christopheravery.com/">Christopher Avery</a>'s The Responsibility Process® workshop on October 28-29th 2018. We all cringe when we hear excuses or blame but it eludes some of us that shame and obligation are not the nirvana of responsibility though they feel almost as bad. To some people shame and obligation IS responsible behavior. Christopher worked with all of the participants on understanding how to responsibility is not Obligation, Shame, Justifying or Blame and how team members and management can start working on their own responsibility and inspire responsibility in others.</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhN6TW9dQ_q6hm_1YkMwEZVZA7jDFOq01ux6kMgy-IiPtX9L1tkIWEnJKKm-RbI3p71cOa7OBqURj9dFlesN-zefkM_bXe1J_CJ_ChcaJwIyT-ci80ZgJYHeufu3h6wcNzFWKHQor7lj0U/s1600/The-Responsibility-Process%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1336" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhN6TW9dQ_q6hm_1YkMwEZVZA7jDFOq01ux6kMgy-IiPtX9L1tkIWEnJKKm-RbI3p71cOa7OBqURj9dFlesN-zefkM_bXe1J_CJ_ChcaJwIyT-ci80ZgJYHeufu3h6wcNzFWKHQor7lj0U/s320/The-Responsibility-Process%255B1%255D.jpg" width="267" /></a></div>
<br />
Christopher started the workshop by asking us to find the "Taxes" and "Dividends" of responsible teams and members, helping us to grasp what it is and what are the costs of not achieving responsibility within teams and our lives.<br />
<br />
He then proceeded to explain the difference between these states and how they hurt us from both a personal view, team members and especially in management positions.<br />
<br />
He explained why more accountability in organizations does not equal more responsibility, how to inspire responsible behavior and the 3 keys to responsibility:<br />
<br />
<ul>
<li>Clear Intention</li>
<li>Focused Attention</li>
<li>Effective Action</li>
</ul>
<div>
We discussed the circles of control and power, how clarity leads to trust and how power comes from being able to stop-think-act (I'm paraphrasing from scuba diving rule book) while control circle is actually hurting decision making by having the illusion of fixing the problem while doing nothing to really solve anything other than feeding the stressful behavior.</div>
<div>
<br /></div>
<div>
We talked about why responsible teams are more productive and why collaboration will produce much higher results than winning, but what is winning if not collaborating?... (did you ever hear "its not our problem" or "you can't touch that" from your supervisor?)</div>
<div>
<br /></div>
<div>
Later we discussed trust issues between teams and members, how to build and rebuild trust and how to avoid losing it since its so hard to rebuild.</div>
<div>
<br /></div>
<div>
In the end of the workshop we all understood why responsible people and teams do a better job than any other team and how we can build our own teams and companies to be responsible for all of our successes.</div>
<br />
<br />
I would like to say thanks to <a href="https://www.practical-agile.com/">Practical Agile</a> for enabling my attendance. Thank you <a href="https://www.linkedin.com/in/friedmanlior/">Lior</a>, <a href="https://www.linkedin.com/in/kirschi/">Ilan</a> and <a href="https://www.linkedin.com/in/dalit-dasht-7a7a1446/">Dalit</a>.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjzJ2abLI3zFydK0PvDGrNobMNAG-l23vaiF49phnHHMxr-Cz_cHsOKWGOvkO-g8UXTnOXqwV2IypIKPA8xSCmlYkopm6Eq8uctqJLELhrPanRY6-sGNQsQH-Ab9xhqXPN5LDHcUpazEkM/s1600/14ee25e3-14e8-42a5-947d-449762fb81e4-original%255B1%255D.jpeg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="550" data-original-width="1046" height="336" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjzJ2abLI3zFydK0PvDGrNobMNAG-l23vaiF49phnHHMxr-Cz_cHsOKWGOvkO-g8UXTnOXqwV2IypIKPA8xSCmlYkopm6Eq8uctqJLELhrPanRY6-sGNQsQH-Ab9xhqXPN5LDHcUpazEkM/s640/14ee25e3-14e8-42a5-947d-449762fb81e4-original%255B1%255D.jpeg" width="640" /></a></div>
<br />
<br />
Resources:<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/ijQ17ZEmC14/0.jpg" src="https://www.youtube.com/embed/ijQ17ZEmC14?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<br />Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-3077621944057198788.post-60013853805024748792018-06-12T12:41:00.001-07:002018-06-12T12:41:41.185-07:00Tiny Model for MNIST Dataset<a href="http://yann.lecun.com/exdb/mnist/">The MNIST database</a> is a database of handwritten digits, its used as an ideal beginner dataset for learning how to do simple image classification and as the dataset only contains 10 characters, its relatively easy to work with.<br />
<br />
It has 60000 tranining examples and 10000 testing examples and it is sufficiently large. as it has 60000 images of 28 x 28 grayscale images, it takes about 50MB, so it does not add complexity for batch generation since it can all fit in memory.<br />
<br />
Like I said, ideal.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhjVchkhmBIKmXLaePC9KubpJn3i7sHNBX_RDzFRDlnXVmpWLqCR3lxiRmAh21Dya1NOvYI7PgOUoUzZTnM6ZMwCNKAkdxJz6jgEUIj7LbSedN0ErbQDJ9qCMHn97vvEaDIn6tRks2_-pQ/s1600/how+low+can+you+go.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="334" data-original-width="480" height="277" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhjVchkhmBIKmXLaePC9KubpJn3i7sHNBX_RDzFRDlnXVmpWLqCR3lxiRmAh21Dya1NOvYI7PgOUoUzZTnM6ZMwCNKAkdxJz6jgEUIj7LbSedN0ErbQDJ9qCMHn97vvEaDIn6tRks2_-pQ/s400/how+low+can+you+go.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
I think the first lesson I did on image classification was with MNIST database as well. it was very amusing to see a 1.2 million parameters model for a 50MB dataset, so I've decided to see how low I can go.<br />
<br />
Lets start with the big one.<br />
<br />
1.2m parameters - <a href="https://gist.github.com/Thimira/89710258517bf1d72621aa9ffc294264">LeNet</a><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhRtnkG0i_0SpF8Vac-DVCz9CZJOLOm2DfWrzZDNpEnZkEb7foGGfChkaBrc5_Svyjy5OrAHqN0c5DpF9gWZIudI5oKS2zu6NcCtaXG1h-w2gVtnK5a_mdZYI-tLJqpRmRNcSHsIA-qCcE/s1600/larger+model.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="428" data-original-width="569" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhRtnkG0i_0SpF8Vac-DVCz9CZJOLOm2DfWrzZDNpEnZkEb7foGGfChkaBrc5_Svyjy5OrAHqN0c5DpF9gWZIudI5oKS2zu6NcCtaXG1h-w2gVtnK5a_mdZYI-tLJqpRmRNcSHsIA-qCcE/s400/larger+model.png" width="400" /></a></div>
<br />
<br />
CNN Error: 0.80%<br />
train_loss 0.0082<br />
train_acc 0.9984<br />
val_loss: 0.0552<br />
val_acc:0.9920<br />
The graph does look like its overfitting by a bit.<br />
<br />
<br />
My first experiment was using huge convolutions, I've managed to train 99% on 300k parameters, but I was not satisfied, surely there is a better way.<br />
<br />
This model at 0.992 accuracy, with only 36k parameters (!!)<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg2o2_q3SCXvTPb6FLzFH5Tm5rJqD-SP95DhJ9TSqHEgV0sa5SefmXWmmXbJSsAJlvRd4C7J5gFml7kKeT1OYl1rewGg-gwBEfh66ik0N0S_9GF2I-ORSY3Yo4Z2jjjh-8z0bMrwbvlw08/s1600/tiny+model.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="432" data-original-width="577" height="297" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg2o2_q3SCXvTPb6FLzFH5Tm5rJqD-SP95DhJ9TSqHEgV0sa5SefmXWmmXbJSsAJlvRd4C7J5gFml7kKeT1OYl1rewGg-gwBEfh66ik0N0S_9GF2I-ORSY3Yo4Z2jjjh-8z0bMrwbvlw08/s400/tiny+model.png" width="400" /></a></div>
<br />
CNN Error: 0.72%<br />
train_loss 0.0224<br />
train_acc 0.9928<br />
val_loss 0.0255<br />
val_acc 0.9928<br />
<br />
36k only? well, that's huge, I've looked around and found out its <a href="https://www.quora.com/What-is-the-smallest-model-in-terms-of-number-of-parameters-that-can-achieve-results-over-99-on-MNIST-dataset">possible with under 4k parameters</a>, so I set up for the challenge and came up with this model.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4bRZfIXss5hdERMRqz6oLqdqlb3G3Ru_83-5wFbH01mRTKs5XJFm-eE7Dp_XWUyDA95P8Lkf-IolI8s6mZVh020_OIccA41oEC_biVrVKnqwkUexgnwel5hoiJXyvdLp87H8uF8qZY7Q/s1600/Figure_1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="420" data-original-width="576" height="290" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4bRZfIXss5hdERMRqz6oLqdqlb3G3Ru_83-5wFbH01mRTKs5XJFm-eE7Dp_XWUyDA95P8Lkf-IolI8s6mZVh020_OIccA41oEC_biVrVKnqwkUexgnwel5hoiJXyvdLp87H8uF8qZY7Q/s400/Figure_1.png" width="400" /></a></div>
CNN Error: 0.90%<br />
train_loss: 0.0369<br />
train_acc: 0.9880<br />
val_loss: 0.0285<br />
val_acc: 0.9910<br />
<div>
<br /></div>
<div>
model is under 4k parameters (3.8)</div>
<div>
<br /></div>
<div>
<br /></div>
<br />
To summarise what I've learned from this exercise is that larger models will learn faster but also overfit faster, smaller models need more training to find a better fit.<br />
<h4>
<br />Credits</h4>
I would like to say thank you to <a href="https://elitedatascience.com/keras-tutorial-deep-learning-in-python">EliteDataScience.com for getting this little exercise started</a><br />
<br />
<br />
<br />
My 36k model:<br />
<script src="https://gist.github.com/drorgl/3dca097b11320e9cf21cececdb46b6c0.js"></script>
<br />
<br />
<br />
My 4k model:<br />
<script src="https://gist.github.com/drorgl/aecfa152a5c6943587370530d87600d8.js"></script>
<br />Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3077621944057198788.post-5156940810056565572018-04-27T12:24:00.001-07:002018-04-27T12:24:33.428-07:00Practical Sensing - RFWe, as humans, are so used to know who we are and where we are that we sometimes forget that it comes at a cost, a person sees where they are, they have an awareness (or general awareness) of their location in a room or on the street, they can also see a step (or feel if one is blind) and eventually know where obstacles are, get from a place to place and plan routes around obstacles.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3LDEn1AWpzdma_qFWRRRL_a0R_ZB9un_CIJN6Dkd2kvz4fMy8q9941YLZZpYHutg2EYxehf73T4V0d3JWgIUMabkv2te6fFx0hPubtrpxkHqPFYl2vB1Xtgd-Cl-KO6M6Uj5zK1fFbqQ/s1600/eye-669157_1920.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3LDEn1AWpzdma_qFWRRRL_a0R_ZB9un_CIJN6Dkd2kvz4fMy8q9941YLZZpYHutg2EYxehf73T4V0d3JWgIUMabkv2te6fFx0hPubtrpxkHqPFYl2vB1Xtgd-Cl-KO6M6Uj5zK1fFbqQ/s640/eye-669157_1920.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="font-size: 12.8px;"><a href="https://pixabay.com/en/eye-internet-forward-vision-669157/">https://pixabay.com/en/eye-internet-forward-vision-669157/</a></td></tr>
</tbody></table>
<br />
<h2>
What can computers do? </h2>
<h3>
Radio Sensing</h3>
<div>
Radio sensing has been invented 1904 and while only capable of detecting a presence it has since evolved into many other sensors.<br />
<br />
RF Radar has a few implementations, among them is presence, distance and movement, the principle is the same, an RF wave is transmitted, it is then reflected (or not) and the returned radio waves are detected, in more advanced scenarios, the received signal goes through FFT to detect the reflected timing/phase, which is further processed to get distance and/or speed of one or more objects.<br />
<br />
Another implementation of radio sensing is localization, in its simplest form triangulation and more advanced is GPS which uses clocks to indicate when the signal was sent to better localize the receiver. BLE have been used for indoor navigation as well.<br />
<br />
A different method of obtaining general location as long as you have network connectivity is using a service such as google geolocate which uses your IP and near by WIFI networks to guess the location.<br />
<br />
An attempt was made to discover the hackability of Bosch Radars (<a href="https://diydrones.com/forum/topics/repurpose-automotive-radar-for-uav-ugv">diydrones</a>, <a href="https://www.mikrocontroller.net/topic/271691">mikrocontroller</a>) but so far without success.<br />
<br />
Another form of electromagnetic sensing is a Geiger counter, which can be used to detect radiation, very useful if you want to detect Radon in your basement or take a relatively safe hike near Chernobyl.</div>
<div>
<br />
Lastly Radio can be used as a cheap way to find out if a certain device is near another device or even communicate a secret of some sort.<br />
<br />
<h4>
Radar</h4>
Doppler - Doppler type sensors detect change or movement, one such cheap sensor is the <a href="https://drive.google.com/open?id=1XkeTb-QWbUxe6LlWOHrop_pca5K76TOD">HB100</a>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhdxMBGK-6Ee4yCPGNlakC_bIepuLL7YkUBP7sEGW-qLJTm7avSlZ2JCosZk6vthmi-v0mdS7NsVQgvtgcoTz8ryt_px2svacUx_WitQ7CECEoBX-SnY4v7H0jmpBIrHJ0EJPz-sSZnRH4/s1600/HB100-Microwave-Doppler-Radar-Wireless-Sensor%255B1%255D.jpeg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1600" data-original-width="1600" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhdxMBGK-6Ee4yCPGNlakC_bIepuLL7YkUBP7sEGW-qLJTm7avSlZ2JCosZk6vthmi-v0mdS7NsVQgvtgcoTz8ryt_px2svacUx_WitQ7CECEoBX-SnY4v7H0jmpBIrHJ0EJPz-sSZnRH4/s320/HB100-Microwave-Doppler-Radar-Wireless-Sensor%255B1%255D.jpeg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://www.tindie.com/products/optimusdigital/hb100-microwave-sensor-module/">https://www.tindie.com/products/optimusdigital/hb100-microwave-sensor-module/</a></td></tr>
</tbody></table>
<br />
<br />
Distance - Sensors such as FM24-NP100 (<a href="https://www.aliexpress.com/item/FREE-SHIPPING-100-NEW-FM24-NP100-24GHz-Microwave-Ranging-Radar-24G-Radar-Body-Sensor-Level-Radar/32867262101.html">$110</a>) provide distance to the biggest reflection but also spectrum data which can be used to monitor multiple objects. These type of sensors measure the phase difference between two wavelengths.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhD31mRao8GsUqTeI7F6D9fVc1SOsCPDQH_Tctbl76p9UmeElr1HOm3wEHmU62YKnmGV_9qX16fochT0NCftBruHoAG2sZkY6QwuCTGQHoIEiuqs9GxGy70DQC3AHJWAo2pbQJ021zqhiM/s1600/FREE-SHIPPING-100-NEW-FM24-NP100-24GHz-Microwave-Ranging-Radar-24G-Radar-Body-Sensor-Level-Radar.jpg_640x640%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="580" data-original-width="610" height="304" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhD31mRao8GsUqTeI7F6D9fVc1SOsCPDQH_Tctbl76p9UmeElr1HOm3wEHmU62YKnmGV_9qX16fochT0NCftBruHoAG2sZkY6QwuCTGQHoIEiuqs9GxGy70DQC3AHJWAo2pbQJ021zqhiM/s320/FREE-SHIPPING-100-NEW-FM24-NP100-24GHz-Microwave-Ranging-Radar-24G-Radar-Body-Sensor-Level-Radar.jpg_640x640%255B1%255D.jpg" width="320" /></a></div>
<br />
<br />
Presence - Modules like the <a href="https://drive.google.com/open?id=13Q-4XHvmyTClgATdB0QP_Broln8Cgp3R">HW-MS03</a> (about <a href="https://www.aliexpress.com/item/New-High-Performance-24GHz-Radar-Sensors-The-Latest-Microwave-Radar-Module-HW-MS03-Small-Size-Mixture/32782742812.html">$2</a>) are in essence a Doppler radar combined with a timer to switch a pin/relay on or off.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEioIMC55feiY1tUKH1VPZ7Zer3OZQmnXWAzBg4_Doft65sP_R9-Fye_b58hSx45N1-pAM_U8U6xXSJEIcKW6lIySH-BvGCkKKrlpLNAHkd-qxq3WSX6gn53riKCIYBhswR2GenJw4RlaME/s1600/New-High-Performance-24GHz-Radar-Sensors-The-Latest-Microwave-Radar-Module-HW-MS03-Small-Size-Mixture%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="800" data-original-width="800" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEioIMC55feiY1tUKH1VPZ7Zer3OZQmnXWAzBg4_Doft65sP_R9-Fye_b58hSx45N1-pAM_U8U6xXSJEIcKW6lIySH-BvGCkKKrlpLNAHkd-qxq3WSX6gn53riKCIYBhswR2GenJw4RlaME/s320/New-High-Performance-24GHz-Radar-Sensors-The-Latest-Microwave-Radar-Module-HW-MS03-Small-Size-Mixture%255B1%255D.jpg" width="320" /></a></div>
<br />
<br />
Some radar modules (such as CFK024-5A for about <a href="https://www.aliexpress.com/item/FREE-SHIPPING-CFK024-5A-24GHZ-Radar-sensors-support-FMCW-FSK-CW-mode-speed-measurement-ranging-collision/32822497902.html">$50</a>) have <a href="https://github.com/khpeek/FMCW-radar">FMCW</a> tuning capabilities, which is very useful if you want to do a sweep which can be used to detect distance of multiple objects but it requires more than basic knowledge.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBnMBZXUNMgnGXoWbSsVsyFpGlQSELMpCQX_2njKHlzahyWMOQOTsTZpTVzSlNsgZmj-z92Tmi8nF69nOcNZeQSWFZq3-OHQHz1usiEhnSqJL51KEN5iDyQFhiQ5gKIM8IQC40To6SbfU/s1600/HTB1E6fSSpXXXXXTXFXXq6xXFXXXr%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="244" data-original-width="638" height="152" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBnMBZXUNMgnGXoWbSsVsyFpGlQSELMpCQX_2njKHlzahyWMOQOTsTZpTVzSlNsgZmj-z92Tmi8nF69nOcNZeQSWFZq3-OHQHz1usiEhnSqJL51KEN5iDyQFhiQ5gKIM8IQC40To6SbfU/s400/HTB1E6fSSpXXXXXTXFXXq6xXFXXXr%255B1%255D.jpg" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_qsk7D83fazXyMTZo-5pYyYPim6LQtPyhBcRh8GlpB8HnXoE3ZXDvn7R17Xdzyv9rcBTqc4GJzb8rPtInpgk8rbBzd2Nv2Qe8LXqdPH5_Ht6acjaZBIOR6EQQ4uNe8K4GlUzyZa6MFZA/s1600/HTB1TcH8SpXXXXbPXXXXq6xXFXXX5%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="181" data-original-width="689" height="105" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_qsk7D83fazXyMTZo-5pYyYPim6LQtPyhBcRh8GlpB8HnXoE3ZXDvn7R17Xdzyv9rcBTqc4GJzb8rPtInpgk8rbBzd2Nv2Qe8LXqdPH5_Ht6acjaZBIOR6EQQ4uNe8K4GlUzyZa6MFZA/s400/HTB1TcH8SpXXXXbPXXXXq6xXFXXX5%255B1%255D.jpg" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<h4>
GPS</h4>
GPS receivers determine location by triangulating the timestamps and signals received from satellites, the more satellites, the more accurate the location will be. but GPS technology is limited by atmospheric conditions, limiting the accuracy possible.<br />
<br />
GPS receivers have advanced over the years, GPS L1 and L2, GloNass, Galileo, Beidou and more, but the accuracy stayed more or less the same at this moment, the peak is around 2.5 meters accuracy for private use.<br />
<br />
To overcome the accuracy limitations, a few <a href="https://en.wikipedia.org/wiki/GNSS_augmentation">augmentations</a> were developed, some are over the air such as SBAS and QZSS, some are based on static base stations like <a href="https://en.wikipedia.org/wiki/Differential_GPS">DGPS</a> and <a href="https://en.wikipedia.org/wiki/Real_Time_Kinematic">RTK</a>.<br />
<br />
Commercial RTK solutions are provided by <a href="https://drotek.com/shop/en/rtk/762-l1-rtk-gnss.html">drotek</a> and <a href="https://emlid.com/reach/">Emlid</a> to name a few.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkXdQnBYv6tHqN04ifGJueoG2q4PwUFvk8hSh1Rpu4qxgn0TTEW7XVeEzS5Q_qGdAV6sHk28ChCbcyFY68lH4FfuIukR1HrWZxC2TMygLSUISvS0W7dgaQPzjk-7casrIkJgzMv_jA7wY/s1600/l1-rtk-gnss%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="800" data-original-width="800" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkXdQnBYv6tHqN04ifGJueoG2q4PwUFvk8hSh1Rpu4qxgn0TTEW7XVeEzS5Q_qGdAV6sHk28ChCbcyFY68lH4FfuIukR1HrWZxC2TMygLSUISvS0W7dgaQPzjk-7casrIkJgzMv_jA7wY/s320/l1-rtk-gnss%255B1%255D.jpg" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxd3NV-JINqgYyt2IO_jy5aYhb-txUZ0IfTyuGzcaR6IQMaWslSvDlCt-6K0fyA0NtTu4F4WbBYhT_kFvCVFgfcXVtoWgVGA6QXj0pxDNoofveodz3IRmQRURLi7PZBQJnJI6bPvtG62Y/s1600/reachm-new%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="743" data-original-width="645" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxd3NV-JINqgYyt2IO_jy5aYhb-txUZ0IfTyuGzcaR6IQMaWslSvDlCt-6K0fyA0NtTu4F4WbBYhT_kFvCVFgfcXVtoWgVGA6QXj0pxDNoofveodz3IRmQRURLi7PZBQJnJI6bPvtG62Y/s320/reachm-new%255B1%255D.jpg" width="277" /></a></div>
<br />
<h4>
BLE</h4>
<div>
BLE beacons are low energy (hence LE) devices which transmit a message once in a while, by reading the received power level (RSSI), it is possible to estimate the distance to the beacon. By knowing where the beacons are, it is possible to triangulate (called <a href="https://en.wikipedia.org/wiki/Trilateration">trilateration</a>) the location of the receiver.</div>
<div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgdfLK2yvW4GC0OdtZDuWBdSfBl5lMa-k3Ogh3lN95_4w3e2viUFSlIxy9O5ikTsO9-I8asBbV-aR2AS7yvADzgNvxpkIlQ5mbyWxZASlOm01x5oB-YYWZdMAIdRXPvhHyjWk-Pgt4SJr4/s1600/MobilePhoneWithBeacon%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="420" data-original-width="770" height="174" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgdfLK2yvW4GC0OdtZDuWBdSfBl5lMa-k3Ogh3lN95_4w3e2viUFSlIxy9O5ikTsO9-I8asBbV-aR2AS7yvADzgNvxpkIlQ5mbyWxZASlOm01x5oB-YYWZdMAIdRXPvhHyjWk-Pgt4SJr4/s320/MobilePhoneWithBeacon%255B1%255D.jpg" width="320" /></a></div>
<br />
<br /></div>
<div>
<a href="https://proximi.io/accurate-indoor-positioning-bluetooth-beacons/">proximi.io</a> (no affiliation) is one of the companies that does that kind of indoor positioning.</div>
<div>
<br /></div>
<h4>
Nuclear Radiation</h4>
<div>
Following <a href="https://en.wikipedia.org/wiki/Fukushima_Daiichi_nuclear_disaster">Fukushima</a> disaster and <a href="https://www.chernobylwel.com/">Chernobyl tourism</a>, personal radiation detection devices became more and more popular. </div>
<div>
<br /></div>
<div>
Dosimeters available in Film Badges, MOSFET and <a href="https://www.aliexpress.com/item/Assembled-DIY-Geiger-Counter-Kit-Nuclear-Radiation-Detector-Beta-Gamma-Ray-Build-Radiation-Monitoring-Station/32847268113.html">Geiger-Muler tubes</a> to name a few.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNCNQWySGV6isUbbt5wvxiXSH9mkXp9Mudm1kJPigiqme1VkVErooD7DbO6ehMcZ8U3M_7kjUfHpcNad3kH-UhO6sIJWMf0zqXKiqZgOAZXRukZUij7LcZAvHz9ROBjWB7ItzeJIVDv-0/s1600/Assembled-DIY-Geiger-Counter-Kit-Nuclear-Radiation-Detector-Beta-Gamma-Ray-Build-Radiation-Monitoring-Station.jpeg_640x640%255B1%255D.jpeg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="640" data-original-width="640" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNCNQWySGV6isUbbt5wvxiXSH9mkXp9Mudm1kJPigiqme1VkVErooD7DbO6ehMcZ8U3M_7kjUfHpcNad3kH-UhO6sIJWMf0zqXKiqZgOAZXRukZUij7LcZAvHz9ROBjWB7ItzeJIVDv-0/s320/Assembled-DIY-Geiger-Counter-Kit-Nuclear-Radiation-Detector-Beta-Gamma-Ray-Build-Radiation-Monitoring-Station.jpeg_640x640%255B1%255D.jpeg" width="320" /></a></div>
<br /></div>
<h4>
Proximity/Identity</h4>
<div>
Wireless identity devices are devices that contain a chip with a small coil, the coil is used to power up the chip and transmit data. two such devices are RFID and NFC devices.</div>
<div>
<br /></div>
<div>
Some devices have static data, some can store custom data, some can encrypt and authenticate but the principle is the same.</div>
<div>
<br /></div>
<div>
RFID more common types are the 125KHz and 13.56MHz and contain a 20 bytes ID and anywhere between 0 and 64 bytes of custom data.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgC79tn5JSi0zIrAEA-6Q4-PBnY77Oith59wAkXmhIwXz3t37lND83gCw5JS8kxnu74NHsvyRU0SJAxNypvz4fdiqDWgFYu_f-ARC7Wf4hc3Ixy4tGH1S7YFwzAw6WFJN9ytm6kbeLl72w/s1600/MFRC-522-RC-522-RC522-Antenna-RFID-IC-Wireless-Module-For-Arduino-SPI-Writer-Reader-IC.jpg_640x640%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="640" data-original-width="640" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgC79tn5JSi0zIrAEA-6Q4-PBnY77Oith59wAkXmhIwXz3t37lND83gCw5JS8kxnu74NHsvyRU0SJAxNypvz4fdiqDWgFYu_f-ARC7Wf4hc3Ixy4tGH1S7YFwzAw6WFJN9ytm6kbeLl72w/s320/MFRC-522-RC-522-RC522-Antenna-RFID-IC-Wireless-Module-For-Arduino-SPI-Writer-Reader-IC.jpg_640x640%255B1%255D.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://www.aliexpress.com/item/1PCS-RC522-Card-Read-Free-shipping-MFRC-522-RFID-RF-IC-card-sensor-module-to-send/32668850152.html">13.56MHz module</a></td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0BupjIVUp7p3pZc_6vWHg3CcR0S2P-zwQK6ZusPTNF0TEWN9TLwe-ro52n5KU-_f70e62vu37Q-iUGLhmxDP5Brtkk1CN7raATCeOLmg6id1OPMPEvcd9DWqHFmcJF5sdycPKSWH2kkU/s1600/1-set-125Khz-RFID-Reader-Module-RDM6300-UART-Output-Access-Control-System.jpg_640x640%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="640" data-original-width="640" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0BupjIVUp7p3pZc_6vWHg3CcR0S2P-zwQK6ZusPTNF0TEWN9TLwe-ro52n5KU-_f70e62vu37Q-iUGLhmxDP5Brtkk1CN7raATCeOLmg6id1OPMPEvcd9DWqHFmcJF5sdycPKSWH2kkU/s320/1-set-125Khz-RFID-Reader-Module-RDM6300-UART-Output-Access-Control-System.jpg_640x640%255B1%255D.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://www.aliexpress.com/item/UART-125Khz-EM4100-RFID-Card-Key-ID-Reader-Module-RDM6300-RDM630-For-Arduino/32711869610.html">125Khz module</a></td></tr>
</tbody></table>
<br />
One of the more interesting things about RFID is the tag sizes<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYENx1JKsEKe9cHLItittvPJ6_2R1KjcVYtqfEb1KFo0w5sdWcCvu38se_s1RCuVgLPowTO9s_koX_gyq1SOoq8cJ_sWnnY0JsXUUesjZ1Z2YzNnZCkaYruBZiG0T2AJwaNB2AzjFRAGk/s1600/10PCS-Lot-1-4x8mm134-2-125KHz-RFID-Glass-Tag-for-Pet-Identification-Tag-for-small-animal.jpg_640x640%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="412" data-original-width="597" height="275" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYENx1JKsEKe9cHLItittvPJ6_2R1KjcVYtqfEb1KFo0w5sdWcCvu38se_s1RCuVgLPowTO9s_koX_gyq1SOoq8cJ_sWnnY0JsXUUesjZ1Z2YzNnZCkaYruBZiG0T2AJwaNB2AzjFRAGk/s400/10PCS-Lot-1-4x8mm134-2-125KHz-RFID-Glass-Tag-for-Pet-Identification-Tag-for-small-animal.jpg_640x640%255B1%255D.jpg" width="400" /></a></div>
<br />
<br /></div>
<div>
<br /></div>
<div>
NFC works much in the same way, memory capacity is between 48 bytes and 32kb.</div>
<div>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhP4PrwPdL9YxW_rW_2jWDhnWtKJil-KuYgY6SKU8SK7nabNF-N0f24mQUAQTwi4uMycZozKhwbAjcHgiR9SYS_Q_lcoT97h0-d_yttgD42F0OKVixlsfP2D2oKGDWLb9JI6G9YxF-Xtqg/s1600/1Set-PN532-NFC-RFID-Wireless-Module-V3-User-Kits-For-Arduino-Android-Reader-Writer-Mode-IC%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="800" data-original-width="800" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhP4PrwPdL9YxW_rW_2jWDhnWtKJil-KuYgY6SKU8SK7nabNF-N0f24mQUAQTwi4uMycZozKhwbAjcHgiR9SYS_Q_lcoT97h0-d_yttgD42F0OKVixlsfP2D2oKGDWLb9JI6G9YxF-Xtqg/s320/1Set-PN532-NFC-RFID-Wireless-Module-V3-User-Kits-For-Arduino-Android-Reader-Writer-Mode-IC%255B1%255D.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://www.aliexpress.com/item/PN532-NFC-RFID-Module-V3-Kits-Reader-Writer/32646243367.html">NFC module</a></td></tr>
</tbody></table>
</div>
<h4>
Radio</h4>
<div>
While radio communication is not a sensor per say it is however a way to communicate and locate and can be used to sense location, state and various data.<br />
<br />
Standard modules come in various frequencies and modulations, some even implement protocols, error correction and buffers.<br />
<br />
Among the more popular ones are the 315Mhz, 433Mhz, 868Mhz, 915Mhz, depending on the country and local regulations and considered <a href="https://en.wikipedia.org/wiki/ISM_band">ISM band</a>, which can be used for anything from car remotes, multirotor telemetry and various remote switches.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAqCJPBlmMMF1Enp62ylB8jWv-b_gS7PpbMSKu4fn-RjrcKrzVuj1BwNmuNn6DDSOD_k8a94Zf3CwZOIW0aQuwjnUMeqGkXsLByvjbJq4AJJh339hZUoSuccPsXvE2ZwguxlkDZIJ2w6k/s1600/Smart-Electronics-433Mhz-RF-transmitter-and-receiver-Module-link-kit-For-arduino-ARM-MCU-WL-diy.jpg_640x640%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="640" data-original-width="640" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAqCJPBlmMMF1Enp62ylB8jWv-b_gS7PpbMSKu4fn-RjrcKrzVuj1BwNmuNn6DDSOD_k8a94Zf3CwZOIW0aQuwjnUMeqGkXsLByvjbJq4AJJh339hZUoSuccPsXvE2ZwguxlkDZIJ2w6k/s320/Smart-Electronics-433Mhz-RF-transmitter-and-receiver-Module-link-kit-For-arduino-ARM-MCU-WL-diy.jpg_640x640%255B1%255D.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://www.aliexpress.com/item/1set-2pcs-RF-wireless-receiver-module-transmitter-module-board-Ordinary-super-regeneration-433MHZ-DC5V-ASK-OOK/32606396563.html">433Mhz RF transmitter and receiver </a></td></tr>
</tbody></table>
<br />
<div style="direction: ltr;">
XBEE, Bluetooth and WIFI are also considered ISM band but more robust implementations are available and are mostly used for higher bandwidth application.</div>
<div style="direction: ltr;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjon5f_oJDoxEpA_Vn_K-uG5n1HLPHlqoofqKEWa6brygXPwmRidDcky-2ApfN0jnyb_lrp3YsVBN67DYlC36AY2TEqGR47ogGI5yGTqOlKHlSaZpvVW8D3b4G3Ie1J1_HchwDSJfkEaio/s1600/xbee.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="389" data-original-width="800" height="155" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjon5f_oJDoxEpA_Vn_K-uG5n1HLPHlqoofqKEWa6brygXPwmRidDcky-2ApfN0jnyb_lrp3YsVBN67DYlC36AY2TEqGR47ogGI5yGTqOlKHlSaZpvVW8D3b4G3Ie1J1_HchwDSJfkEaio/s320/xbee.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://www.digi.com/products/xbee-rf-solutions">XBee</a></td></tr>
</tbody></table>
<div style="direction: ltr;">
<br /></div>
<div style="direction: ltr;">
Some of the more popular 2.4Ghz modules are the NRF24L01, A7105 and CC2500 which are used in RC Toys.</div>
<div style="direction: ltr;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAx8iby_gN4wPY9hqWbc6_8VQgPlXcMJYCP6xDkaUI9iMs9ziTMXrwiFe5GMg-bLGmer4FQoOjP8NlVuUxztiDAEnB3T289PE1AbPpld997Jeza2TK0n1KJ-Yt4sp612P8sb1fvvTpW_Q/s1600/4%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1000" data-original-width="1000" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAx8iby_gN4wPY9hqWbc6_8VQgPlXcMJYCP6xDkaUI9iMs9ziTMXrwiFe5GMg-bLGmer4FQoOjP8NlVuUxztiDAEnB3T289PE1AbPpld997Jeza2TK0n1KJ-Yt4sp612P8sb1fvvTpW_Q/s320/4%255B1%255D.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://www.frsky-rc.com/product/taranis-q-x7s/">FrSky Taranis Q X7S</a></td></tr>
</tbody></table>
<div style="direction: ltr;">
<br /></div>
Lora, LoraWan, SigFox and NB-IOT are mostly used for smart appliances such as water meters, power meters and relatively long range and big coverage requirements and are very low bandwidth.<br />
<br />
Cell (2G/3G and up) are used nowadays for anything from messaging to video playback.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgFKunp8sIscmeZY06pf7jl5JwK8zLgWRT-xHdLjDpVqHMs_zXDJ_PCcw_V0HMODjeOvMoR0Yi7QpClt8S-8hPVbMdGeU9SUwBsoEMSemBx-cdLniOy4UntQY7wIGaFZJaCFq77RE3PsWk/s1600/Smallest-SIM800L-GPRS-GPS-GSM-Module-Micro-SIM-Card-Core-Board-Quad-band-Antenna-TTL-Serial.jpg_640x640%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="640" data-original-width="640" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgFKunp8sIscmeZY06pf7jl5JwK8zLgWRT-xHdLjDpVqHMs_zXDJ_PCcw_V0HMODjeOvMoR0Yi7QpClt8S-8hPVbMdGeU9SUwBsoEMSemBx-cdLniOy4UntQY7wIGaFZJaCFq77RE3PsWk/s320/Smallest-SIM800L-GPRS-GPS-GSM-Module-Micro-SIM-Card-Core-Board-Quad-band-Antenna-TTL-Serial.jpg_640x640%255B1%255D.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://www.aliexpress.com/item/Smallest-SIM800L-GPRS-GPS-GSM-Module-Micro-SIM-Card-Core-Board-Quad-band-Antenna-TTL-Serial/32674459695.html">SIM800L</a></td></tr>
</tbody></table>
<br />
<br /></div>
References:<br />
<br />
https://upcommons.upc.edu/bitstream/handle/2099.1/6629/PFC_MarcMirTutusaus.pdf?sequence=1<br />
https://fccid.io/NF3ACC2SCU/User-Manual/Installer-Instructions-413993<br />
https://etd.ohiolink.edu/!etd.send_file?accession=ohiou1304083389&disposition=inline<br />
https://hackaday.com/2017/05/24/radar-sensors-put-to-the-test/<br />
<br /></div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3077621944057198788.post-880911841054985062018-03-31T05:00:00.003-07:002018-03-31T05:08:46.467-07:00Practical Sensing - PhysicalWe, as humans, are so used to know who we are and where we are that we sometimes forget that it comes at a cost, a person sees where they are, they have an awareness (or general awareness) of their location in a room or on the street, they can also see a step (or feel if one is blind) and eventually know where obstacles are, get from a place to place and plan routes around obstacles.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3LDEn1AWpzdma_qFWRRRL_a0R_ZB9un_CIJN6Dkd2kvz4fMy8q9941YLZZpYHutg2EYxehf73T4V0d3JWgIUMabkv2te6fFx0hPubtrpxkHqPFYl2vB1Xtgd-Cl-KO6M6Uj5zK1fFbqQ/s1600/eye-669157_1920.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3LDEn1AWpzdma_qFWRRRL_a0R_ZB9un_CIJN6Dkd2kvz4fMy8q9941YLZZpYHutg2EYxehf73T4V0d3JWgIUMabkv2te6fFx0hPubtrpxkHqPFYl2vB1Xtgd-Cl-KO6M6Uj5zK1fFbqQ/s640/eye-669157_1920.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://pixabay.com/en/eye-internet-forward-vision-669157/">https://pixabay.com/en/eye-internet-forward-vision-669157/</a></td></tr>
</tbody></table>
<br />
<h2>
What can machines do? </h2>
<div>
This is a quick introduction about sensors, what are the available options for a beginner maker/developer.</div>
<h3>
Physical Sensing</h3>
"Sensors" such as switches, magnetic, sound (or ultrasonic), pressure, temperature, flexing (or bending), shear (or weight), piezo, gas, capacitive and resistive sensors have been around the block, simple vacuum robots have some of them, cars have them (not necessarily self driving ones, see <a href="https://en.wikipedia.org/wiki/Parking_sensor">parking sensors</a>) and even old refrigerators have them (someone has to turn on the light when you're hungry at 4AM).<br />
<br />
<h4>
Switches</h4>
Switches and buttons come in all shapes and sizes but the mechanical principle is the same, there are two (or more) spaced conductors, when a force is applied, one of the conductors (or a third) is moving toward them and makes a connection.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgyAYL8QKIc60Ndfjm3fUEbEQOac_8q5uYLPzDGeVK4kw1dhhyFiNvQzAa1MTdLzAVG7gYgK2F6YUWE_yWKJn9h5r9xQE2zipgZUlRVR57w1Jt8DWdYe2T5KBMTNOKD2En7qLPZEofr9uc/s1600/push-button-switches-make-easy-your-task-7-638.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="222" data-original-width="281" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgyAYL8QKIc60Ndfjm3fUEbEQOac_8q5uYLPzDGeVK4kw1dhhyFiNvQzAa1MTdLzAVG7gYgK2F6YUWE_yWKJn9h5r9xQE2zipgZUlRVR57w1Jt8DWdYe2T5KBMTNOKD2En7qLPZEofr9uc/s1600/push-button-switches-make-easy-your-task-7-638.jpg" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://www.omron-ecb.co.kr/web/en/special/switch/basic02-05">https://www.omron-ecb.co.kr/web/en/special/switch/basic02-05</a></td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
Other types of switches can be limit, rocker, toggle, DIP, push buttons, thumbwheels and more, but the principle is the same applied in different ways and are represented very similarly.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh0fpbxeN4GzWPIoM4aRkj-TX54pPPxYgGnUse86wuvXGuN8ccyBiGjO3xenPXuBVPrORza5RvIyxDrsbW9cFm_TOJInYbmOjTLsE7gTcYUI6CEGGMiw3VHPBPP9dlr1kKf_V38cwG2PEI/s1600/more-switches-relays%255B1%255D.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="412" data-original-width="523" height="500" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh0fpbxeN4GzWPIoM4aRkj-TX54pPPxYgGnUse86wuvXGuN8ccyBiGjO3xenPXuBVPrORza5RvIyxDrsbW9cFm_TOJInYbmOjTLsE7gTcYUI6CEGGMiw3VHPBPP9dlr1kKf_V38cwG2PEI/s640/more-switches-relays%255B1%255D.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://www.edrawsoft.com/switch-symbols.php">https://www.edrawsoft.com/switch-symbols.php</a></td></tr>
</tbody></table>
Among these mechanic switches there are also simple tilt switches, they work by the same principle where a metal ball is the connection point.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvNtBPHSmz3AYl8Ie1EnpPaWGLHjRMY9lc4THeloAfPN7LQclnMrFGYYVj6HxXbLIhaifIe03LwbzIegt53im5aqcOSe81lCkXNtS4uEAfL5f8obCU9B2Jv1Wox80Aub451Oyy2cnxsk8/s1600/tilt+switch.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="177" data-original-width="329" height="172" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvNtBPHSmz3AYl8Ie1EnpPaWGLHjRMY9lc4THeloAfPN7LQclnMrFGYYVj6HxXbLIhaifIe03LwbzIegt53im5aqcOSe81lCkXNtS4uEAfL5f8obCU9B2Jv1Wox80Aub451Oyy2cnxsk8/s320/tilt+switch.jpg" width="320" /></a></div>
<br />
<br />
Mercury tilt switches got infamous in the movies for less favoring reasons<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLF02YnBBkXYD8jCTW6YnQRnPZWvV9eBKOC2-repf4n4W0SfCcrlEoGcSK7TM-CkaY9TM9bPTJzTCZ8DtLC3_uBO919EADODT0LBUwnx4Mwiv1A6Ozw0W6NOx0Q2rFITYUwY7aKw9M874/s1600/800px-Mercury_Switch_without_housing%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="417" data-original-width="800" height="166" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLF02YnBBkXYD8jCTW6YnQRnPZWvV9eBKOC2-repf4n4W0SfCcrlEoGcSK7TM-CkaY9TM9bPTJzTCZ8DtLC3_uBO919EADODT0LBUwnx4Mwiv1A6Ozw0W6NOx0Q2rFITYUwY7aKw9M874/s320/800px-Mercury_Switch_without_housing%255B1%255D.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://en.wikipedia.org/wiki/Mercury_switch">Source</a></td></tr>
</tbody></table>
<br />
<br />
<br />
So what are switches and buttons good for?<br />
<br />
Sensing mechanical contact, bumping into walls, rotational forces, etc'<br />
<br />
What are they not good for?<br />
<br />
Anything that requires precision and detection of the amount of pressure, they are on-off, nothing more.<br />
<br />
<h4>
Magnetic</h4>
A magnetic sensor is a type of sensor capable of sensing magnetic fields, magnetic fields are a directional field but the direction can not always be sensed, for example a reed switch will close (N.O) or open (N.C) based on the presence of magnetic field. We can also sense the presence and direction of a magnetic field with Hall Effect sensors (1D) and electronic compass if we require a 3D sensing of electric fields.<br />
<br />
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2RVovQ_fjipQoJqYuBRVK9KZ1ZE-mzAvGfDnOgJ-FiNM8W6r1_61iqEtpvl3mylO2f3g9-w9pM4eKzA0DA_lpFXlmdyWHmwNa26lCMY6qp7ZCTnPUPYUUspPqqGOAdfSyhN_nmYekW20/s1600/6a00d8345225f869e20147e27ef7ec970b.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="768" data-original-width="1024" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2RVovQ_fjipQoJqYuBRVK9KZ1ZE-mzAvGfDnOgJ-FiNM8W6r1_61iqEtpvl3mylO2f3g9-w9pM4eKzA0DA_lpFXlmdyWHmwNa26lCMY6qp7ZCTnPUPYUUspPqqGOAdfSyhN_nmYekW20/s400/6a00d8345225f869e20147e27ef7ec970b.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://www.memsjournal.com/2011/02/motion-sensing-in-the-iphone-4-electronic-compass.html">http://www.memsjournal.com/2011/02/motion-sensing-in-the-iphone-4-electronic-compass.html</a></td></tr>
</tbody></table>
These electronic compasses are so cool and sensitive you can use them to <a href="https://hackaday.io/project/11865-3d-magnetic-field-scanner/log/39398-proof-of-concept">visualize a magnetic field</a>!<br />
<br />
Since hall sensors are so inexpensive and accurate, they can be used for rotational speed detection (odometry) and <a href="http://s.click.aliexpress.com/e/yFmeYRF">water</a>/air flow sensors.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBJlSlDvCVfV3YXcpGZuZYfe9ws2sfKWbZlJYyqqg2piVUKs6qxtdfeOG01s4iQ4ozjNI85fi2IUlMDgtsIPLcG0Q8U926xSEYaPpWqQrB12cSj-XhBIl-_d0dEQNi8VD783icm0iQzhw/s1600/1pc-1-30L-min-Water-Flow-Sensor-Flowmeter-Hall-Flow-Sensor-Water-Control-1-2-2%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1080" data-original-width="1080" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBJlSlDvCVfV3YXcpGZuZYfe9ws2sfKWbZlJYyqqg2piVUKs6qxtdfeOG01s4iQ4ozjNI85fi2IUlMDgtsIPLcG0Q8U926xSEYaPpWqQrB12cSj-XhBIl-_d0dEQNi8VD783icm0iQzhw/s320/1pc-1-30L-min-Water-Flow-Sensor-Flowmeter-Hall-Flow-Sensor-Water-Control-1-2-2%255B1%255D.jpg" width="320" /></a></div>
<br />
<br />
The principle is the same for most speed sensors, a moving part and a detector.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRoZFg8M-4xiYv2_sC_EHSkSZ5iUSiOS7AQablqNG-USQeH3s3wQ2DuwcSX1C3bolkegvzLk9WD1ZtjMOJ3281EVQKOC5cA9X4Fg18QGECsfE4wyBIOJi4PPNC74jxRcXRG4AsY-UP-as/s1600/rpm+sensor.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="799" data-original-width="567" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRoZFg8M-4xiYv2_sC_EHSkSZ5iUSiOS7AQablqNG-USQeH3s3wQ2DuwcSX1C3bolkegvzLk9WD1ZtjMOJ3281EVQKOC5cA9X4Fg18QGECsfE4wyBIOJi4PPNC74jxRcXRG4AsY-UP-as/s400/rpm+sensor.png" width="283" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://commons.wikimedia.org/wiki/File:Hall_effect_sensor_of_fan_cooler.svg">Source</a></td></tr>
</tbody></table>
<br />
<br />
Another very interesting application of hall sensor is 3D absolute sensing like the <a href="https://www.melexis.com/-/media/files/documents/datasheets/mlx90333-datasheet-melexis.pdf">MLX90333</a>:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxgL5K41r9sr-SRB2ZQwRxWvFFQrZ2gEESz5JbOBY94LZqoIMbbVsLb6O_pIIL5AGYGs_0jF9q49e8pb7HQ54_6Ak32xfKCN-3x9RnrdrjLlHAGpXxhh9t8eVOMxHuitkQavbt52IAAH4/s1600/1626331842%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="297" data-original-width="419" height="226" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxgL5K41r9sr-SRB2ZQwRxWvFFQrZ2gEESz5JbOBY94LZqoIMbbVsLb6O_pIIL5AGYGs_0jF9q49e8pb7HQ54_6Ak32xfKCN-3x9RnrdrjLlHAGpXxhh9t8eVOMxHuitkQavbt52IAAH4/s320/1626331842%255B1%255D.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://www.vogel.com.cn/top/2012munihei/prods_t_view.html?id=28221">Source</a></td></tr>
</tbody></table>
<br />
<br />
Another type of magnetic sensing is induction sensing, this is what metal detectors do, the way they work is by generating a strong magnetic field and detecting changes in the feedback, induction sensors are used in 3D printers to detect the build plate, they are pretty accurate and repeatable, but there are better sensors for that purpose as well and they are limited by the fact that they can only sense metals.<br />
<br />
Common usages:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYowNJVnEPj3QeIKolvsyeoRsFw8sVJkgmy_e2IcpXYBn3hcr1puvy42NWQ8bxGxYMjBPyyKzrnEZyu77u-7a-OcOpMdO0PG02tLdH5-iDoHRWV7dGVqa4-tASm5BwY0F_uWCy4WpJo84/s1600/neato+mag+sensor.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="208" data-original-width="306" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYowNJVnEPj3QeIKolvsyeoRsFw8sVJkgmy_e2IcpXYBn3hcr1puvy42NWQ8bxGxYMjBPyyKzrnEZyu77u-7a-OcOpMdO0PG02tLdH5-iDoHRWV7dGVqa4-tASm5BwY0F_uWCy4WpJo84/s1600/neato+mag+sensor.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Neato Boundary Sensor</td></tr>
</tbody></table>
<br />
<br />
So what are they good for?<br />
The reed switches are used as an on-off switch when a magnet is near by, the common example are windows and doors sensors for alarms. The hall effect sensors are used to precisely detect a magnetic object and the compasses are used to orient against magnetic north or artificial magnetic field.<br />
<br />
What are they not good for?<br />
If the environment is magnetically "dirty", many motors and magnets, both the hall effect and the compass are going to have a hard time "locking", also, electronic compasses are relatively slow, so they can not be used to keep the orientation for a moving object, for that purpose they are combined with accelerometers and gyros and their data is combined (or "fused" - see <a href="https://learn.adafruit.com/ahrs-for-adafruits-9-dof-10-dof-breakout/sensor-fusion-algorithms">sensor fusion</a>) to keep the robot oriented until the compass catches on.<br />
<br />
<h4>
Sound</h4>
Sound has very interesting properties with regard to robots, it bounces easily, its travelling relatively slow and sound sensors are abundant (microphones for example).<br />
<br />
These leads us to a very simple implementation: distance sensors, they send a ping, wait for the response and report the time it took, Mechatronics has a nice <a href="https://howtomechatronics.com/tutorials/arduino/ultrasonic-sensor-hc-sr04/">tutorial</a>.<br />
<br />
Another very interesting usage is direction detection, I've seen <a href="http://s.click.aliexpress.com/e/v3rBM7a">3 mic arrays</a>, <a href="https://www.cnx-software.com/2018/03/13/respeaker-core-v2-is-a-6-mic-array-audio-development-kit-powered-by-rockchip-rk3229-processor/">6 mic array</a>s and they come with SDKs and can do noise reduction and direction detection.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8Z3nc2OzHcZn1OAtOEdkLvfXQ_5mgna4k9BfiZvbFbTU13QDJvrLqtc_Izi7GilOKQLbtRtg6GVTlZGusCh1P9oW9_2tzoIWBZTO6jfLV4z1huQD5sXa1AJmRKMx3H_M0GChOB-3PTl4/s1600/Rockchip-RK3229-Smart-Speaker-Development-Kit%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="267" data-original-width="640" height="265" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8Z3nc2OzHcZn1OAtOEdkLvfXQ_5mgna4k9BfiZvbFbTU13QDJvrLqtc_Izi7GilOKQLbtRtg6GVTlZGusCh1P9oW9_2tzoIWBZTO6jfLV4z1huQD5sXa1AJmRKMx3H_M0GChOB-3PTl4/s640/Rockchip-RK3229-Smart-Speaker-Development-Kit%255B1%255D.jpg" width="640" /></a></div>
<br />
Another interesting use for ultrasonic sound is indoor navigation, in theory, one can triangulate the source of a sound pulse by the time it takes to get to the transceivers.<br />
<br />
Thanks to <a href="https://marvelmind.com/">Marvelmind </a>its no longer a theory:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8-G_Rn6YJAu6c4mwJ_DMHxdYfmC4r95v02OLHUIwq7DX_navOXhQ03RwfNrpI-z0D1r0Yq_XBfKpLsc36MQtQNqGds9Vl8dYvzOIIthWfv0_lY7BDwxah_PMZftFufq3VhdMW9_WKcAc/s1600/dsc_9926-1-e1506092225632-307x380%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="380" data-original-width="307" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8-G_Rn6YJAu6c4mwJ_DMHxdYfmC4r95v02OLHUIwq7DX_navOXhQ03RwfNrpI-z0D1r0Yq_XBfKpLsc36MQtQNqGds9Vl8dYvzOIIthWfv0_lY7BDwxah_PMZftFufq3VhdMW9_WKcAc/s320/dsc_9926-1-e1506092225632-307x380%255B1%255D.jpg" width="258" /></a></div>
<br />
<br />
Common usage would be <a href="http://s.click.aliexpress.com/e/uRf6QRB">distance sensors</a>, <a href="http://s.click.aliexpress.com/e/iEaujQf">atomizers</a>, <a href="http://s.click.aliexpress.com/e/rNFeam2">voice commands</a>.<br />
<br />
So what is it good for?<br />
<br />
When you need to detect distance to a flat surface (rememebr, sound bounces..), when you need to reduce noise in a noisy environment or when you need to detect the direction of where the sound is coming from.<br />
<br />
What is it not good for?<br />
<br />
When the surfaces are not perpendicular to the sensor, sound will bounce all over the place, you should be thankful for any reading.<br />
<div>
<br /></div>
<div>
Interesting libraries:</div>
<div>
<a href="https://playground.arduino.cc/Code/NewPing">NewPing</a></div>
<br />
<h4>
Pressure</h4>
Pressure sensors are useful for many things, for hydraulic systems they can detect leaks, for water systems they can detect presence or water pump quality or even regulating pressure with a PID loop.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBhTr5QVkZR4xt3ISbQc3wbKNjutXC11td9gWq3WZKG7JoJ3vp2BMWA9zZV3Oc5BhOy8QjXgPnOfANe2kU5ekSHPd5bKkrSXlZsPuYFsGXxRcQiJkMzMq_GsB3PEWLPLommh29jx2x0qE/s1600/Pressure-Sensor-Transmitter-DC-5V-G1-4-0-1-2-MPa-0-174-PSI-For-Water%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="800" data-original-width="800" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBhTr5QVkZR4xt3ISbQc3wbKNjutXC11td9gWq3WZKG7JoJ3vp2BMWA9zZV3Oc5BhOy8QjXgPnOfANe2kU5ekSHPd5bKkrSXlZsPuYFsGXxRcQiJkMzMq_GsB3PEWLPLommh29jx2x0qE/s320/Pressure-Sensor-Transmitter-DC-5V-G1-4-0-1-2-MPa-0-174-PSI-For-Water%255B1%255D.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://s.click.aliexpress.com/e/zzZnAia">Water Pressure Sensor</a></td></tr>
</tbody></table>
<br />
<br />
Water pressure sensors can detect depth in submersible robots.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgp88Zz-uelHLVlv7GY_WP-zGew_WJadJxil9fD4R6ZhE1bOE6mTVvwueieHyTbez9W1CZ45ZzWCRB7wUSZEbtT1VoxkQruwQiQ7ZiXh9AGb14hIuPqUN-Rg_O-83z6fhEVu2eigUbPfxc/s1600/bar30-high-resolution-300m-depth-pressure-sensor%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="900" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgp88Zz-uelHLVlv7GY_WP-zGew_WJadJxil9fD4R6ZhE1bOE6mTVvwueieHyTbez9W1CZ45ZzWCRB7wUSZEbtT1VoxkQruwQiQ7ZiXh9AGb14hIuPqUN-Rg_O-83z6fhEVu2eigUbPfxc/s320/bar30-high-resolution-300m-depth-pressure-sensor%255B1%255D.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://www.robotshop.com/en/bar30-high-resolution-300m-depth-pressure-sensor.html">300m Depth/Pressure Sensor</a></td></tr>
</tbody></table>
<br />
<br />
Air Pressure sensors are also very useful for weather stations as they can detect <a href="https://en.wikipedia.org/wiki/Weather_front">weather fronts</a> and predict rain.<br />
<br />
Air Pressure sensors can also help multi-rotors keep a certain height, not accurately though, but good enough. Just don't count on them to work on bad weather, I've seen a multi-rotor drop 10 meters on a bad weather day and go back up in a blink of an eye, you can't blame the electronics..<br />
<br />
A common type of air pressure sensor is the <a href="http://s.click.aliexpress.com/e/6YZFMVf">BMP280</a>:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjiIZLMgQrV7Gp1AIk5-dCw2PpMrSBtzmULCvpJn-qa7LAmVuJRDsuXXKR20C-7lKmTUWw6cb9N02yLHKlvepaSyDzSKxEdq_Xf5kuIRW9vX_DhiY07uHkhPryZanRufj2LczjKZANythc/s1600/BME280-Digital-Sensor-Temperature-Humidity-Barometric-Pressure-Sensor-Module-I2C-SPI-1-8-5V-GY-BME280%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="968" data-original-width="968" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjiIZLMgQrV7Gp1AIk5-dCw2PpMrSBtzmULCvpJn-qa7LAmVuJRDsuXXKR20C-7lKmTUWw6cb9N02yLHKlvepaSyDzSKxEdq_Xf5kuIRW9vX_DhiY07uHkhPryZanRufj2LczjKZANythc/s320/BME280-Digital-Sensor-Temperature-Humidity-Barometric-Pressure-Sensor-Module-I2C-SPI-1-8-5V-GY-BME280%255B1%255D.jpg" width="320" /></a></div>
Air Pressure sensors are also used to <a href="http://s.click.aliexpress.com/e/UB2RZBi">detect speed</a> in aircrafts.<br />
<br />
All pressure sensors measure from a certain reference point, gauge, absolute and differential are relative and <a href="http://www.machinedesign.com/pneumatics/what-s-difference-between-gauge-absolute-differential-and-sealed-pressure">you should know what you need</a>.<br />
<br />
Make sure the pressure sensor is suitable for your medium and amount of stress its going to take, note the breakdown pressure so no one will lose an eye/finger and if dealing with high pressure, a suitable burst valve is always recommended.<br />
<br />
<h4>
Temperature</h4>
Temperature sensors are useful in cases when you need to know the temperature, for example, is your motor running hot? did the water boil already? is the oil too hot? is the printer's hotend at the right temperature? is it hot outside?<br />
<br />
Like all other sensors, temperature sensors come in different flavors, but they divide into 2 groups: conduction and radiation sensors.<br />
<br />
A common air/weather convection temperature sensor is the <a href="http://s.click.aliexpress.com/e/2z727QJ">DHT11</a> and <a href="http://s.click.aliexpress.com/e/EIeii6u">DHT22</a>, these are very cheap:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzjpzRQ3lhcN6mSUDUY80e0N8ahC0QB3ks2C8JrEIz0FECst-eHWVUFnjfDGoJIyDQ_mN8tOqVFYtdvnBlDMXEzSnKwtHDtMtY7jAJKLnfzaTp6HdpVSQoC2qsjuhrU-T63GtuRkqqmvk/s1600/New-DHT11-Temperature-And-Relative-Humidity-Sensor-Module.jpg_640x640%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="640" data-original-width="640" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzjpzRQ3lhcN6mSUDUY80e0N8ahC0QB3ks2C8JrEIz0FECst-eHWVUFnjfDGoJIyDQ_mN8tOqVFYtdvnBlDMXEzSnKwtHDtMtY7jAJKLnfzaTp6HdpVSQoC2qsjuhrU-T63GtuRkqqmvk/s320/New-DHT11-Temperature-And-Relative-Humidity-Sensor-Module.jpg_640x640%255B1%255D.jpg" width="320" /></a></div>
<br />
<br />
A common type of conduction temperature sensor is the NTC 100k thermistor<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjldssD-gCB7kDHEk50RSV7_tkPge34B5gvGHF_MHqawCUeds5jyKVCbJYfH9tbM1I9UUU4p2y-DLfXxuKwts6Mc_39yhN3FkNtdcVjrKmFqeshb-__rQjGu91P4T0jSoSDMBCTbDSDAU8/s1600/thermistores-100kohm-ntc-%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="600" data-original-width="600" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjldssD-gCB7kDHEk50RSV7_tkPge34B5gvGHF_MHqawCUeds5jyKVCbJYfH9tbM1I9UUU4p2y-DLfXxuKwts6Mc_39yhN3FkNtdcVjrKmFqeshb-__rQjGu91P4T0jSoSDMBCTbDSDAU8/s320/thermistores-100kohm-ntc-%255B1%255D.jpg" width="320" /></a></div>
Another common type of sensor is the contact-less IR temperature sensor, <a href="https://www.melexis.com/en/products/sense/temperature-sensors">Melexis</a> makes some of them:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjnk9UVFXn6YNTyWWdFTurQ8-SEN5kd0xg2MEYrkcg8MZdI0aZ71wW_6hXNTzU7NDRLuTgo7_uBfMnUNeh9IALmIBBDeM6mj0mZMkM3DYilYoMvIFkNVovOsK7xVqsrkheLaMua2il4tBs/s1600/09570-01%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="600" data-original-width="600" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjnk9UVFXn6YNTyWWdFTurQ8-SEN5kd0xg2MEYrkcg8MZdI0aZ71wW_6hXNTzU7NDRLuTgo7_uBfMnUNeh9IALmIBBDeM6mj0mZMkM3DYilYoMvIFkNVovOsK7xVqsrkheLaMua2il4tBs/s320/09570-01%255B1%255D.jpg" width="320" /></a></div>
BTW, the same principle applies for IR cameras, which produce these cool images:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh92tlaF2KCxuHUUzV7NbCB4jiPw4kY_qWxirTbI_Kg2SJn__xYIEJUWkuaIpteFUyPNHxmcRusbBcNqadSCpFaXOSusc3-hcbr73pdcPVAfhXP0MbSWypy4AWt0r_6QqUWTa8R9Qe0O9Q/s1600/800px-Whisky_Distillery_Thermal_Image%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="600" data-original-width="800" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh92tlaF2KCxuHUUzV7NbCB4jiPw4kY_qWxirTbI_Kg2SJn__xYIEJUWkuaIpteFUyPNHxmcRusbBcNqadSCpFaXOSusc3-hcbr73pdcPVAfhXP0MbSWypy4AWt0r_6QqUWTa8R9Qe0O9Q/s320/800px-Whisky_Distillery_Thermal_Image%255B1%255D.jpg" width="320" /></a></div>
<br />
Two consumer grade companies makes these sensors, <a href="https://www.thermal.com/compact-series.html">Seek </a>and <a href="http://www.flir.eu/flirone/">FLIR</a>, both cost around $200 for the cheap versions and of-curse you can get sensors only, but that is a different adventure.<br />
<br />
<br />
Another kind of temperature sensor is the thermostats, which can be used for thermal protection, no sensing, just turn on the fan when its too hot, or turn off the power. they are not accurate but they do a good job for their designed purpose.<br />
<br />
I've used the <a href="http://s.click.aliexpress.com/e/YFaAqrb">KSD9700 </a>to quiet down a power supply that had a fan always on:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjyZ7ZuhJvK7gLyL4aMfAhR0mAkSEi79xfmGKm4k5ex7zAFM2bGVxPa5asKCzhEj467xtUus1Ap5Y8Jnfu16bYpc_mfMztYsuKiY0GfBaooJzgzFv2hFPVcQRDawrvu2CMremggtZNFZrE/s1600/KSD9700-250V-5A-15-155-Degree-Bimetal-Disc-Temperature-Switch-Normal-Close-Closed-Thermostat-Thermal-Protector.jpg_640x640%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="640" data-original-width="640" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjyZ7ZuhJvK7gLyL4aMfAhR0mAkSEi79xfmGKm4k5ex7zAFM2bGVxPa5asKCzhEj467xtUus1Ap5Y8Jnfu16bYpc_mfMztYsuKiY0GfBaooJzgzFv2hFPVcQRDawrvu2CMremggtZNFZrE/s320/KSD9700-250V-5A-15-155-Degree-Bimetal-Disc-Temperature-Switch-Normal-Close-Closed-Thermostat-Thermal-Protector.jpg_640x640%255B1%255D.jpg" width="320" /></a></div>
<br />
Lastly the PTC thermistors, they are a type of resistor that when they get too hot, their resistance jumps significantly, they are used to protect over-current and are regarded as self resetting fuse as the system cools off, the power returns to normal.<br />
<br />
This is <a href="http://s.click.aliexpress.com/e/yjUzBqR">a nice kit to get started</a>:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg64rq2N5LWAOfQGsNU0o62tblZVEBJ4o3bAHV-O_Hf8_E8fIuaK8iQgBQTemeYlkmbBFsp5a4WBVog3WA3luVb2N4eH37Xu6ydpz3AT0FUtjAwrJSKALWzhsdtUEeP7QfioTjloLSDtpo/s1600/Free-shipping-80pc-lot-DIP-resettable-fuse-kit-0-1A-10A-fuse-assortment-Diy-kit.jpg_640x640%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="424" data-original-width="640" height="212" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg64rq2N5LWAOfQGsNU0o62tblZVEBJ4o3bAHV-O_Hf8_E8fIuaK8iQgBQTemeYlkmbBFsp5a4WBVog3WA3luVb2N4eH37Xu6ydpz3AT0FUtjAwrJSKALWzhsdtUEeP7QfioTjloLSDtpo/s320/Free-shipping-80pc-lot-DIP-resettable-fuse-kit-0-1A-10A-fuse-assortment-Diy-kit.jpg_640x640%255B1%255D.jpg" width="320" /></a></div>
<br />
<h4>
Flexing </h4>
Flexing (or bending) sensors are basically resistors that change their resistance when they are bent, they are great for human interaction since, if you think about it, we have many joints that flex and tracking these flexing movements is not trivial by other means, so gloves, sleeves, tights, if it bends, you can detect it with these little sensors.<br />
<br />
Be careful though, these sensors are not cheap and the material melts away if attempting to solder it, you might be able to fix it with <a href="http://s.click.aliexpress.com/e/vnQbybU">conductive silver paint</a>, avoid heat as much as possible and avoid rubbing it in the exposed areas.<br />
<br />
Here's a<a href="http://s.click.aliexpress.com/e/B2jYjMV"> 2.2" flex sensor</a>, they come at various lengths:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEZR3GHjd3vQVq-yeVQyGJAJsbCrQy_EM3viD1qVcMzp_MHMNyA9VSk1S2GvD-8c4YVBPMeyjK9xn0bcsaXCrOVUfAa_lDN3cpMAms4JVgUxJErM5490R4I6JRZbNazH04A2uTZCoCURI/s1600/2-2-Inch-Bend-Flex-Sensor-for-Robotic-Arm-Power-Glove-etc.jpg_640x640%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="640" data-original-width="640" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEZR3GHjd3vQVq-yeVQyGJAJsbCrQy_EM3viD1qVcMzp_MHMNyA9VSk1S2GvD-8c4YVBPMeyjK9xn0bcsaXCrOVUfAa_lDN3cpMAms4JVgUxJErM5490R4I6JRZbNazH04A2uTZCoCURI/s320/2-2-Inch-Bend-Flex-Sensor-for-Robotic-Arm-Power-Glove-etc.jpg_640x640%255B1%255D.jpg" width="320" /></a></div>
<h4>
Shear / Load Cell / Weight</h4>
Shear beam, Load Cells and Weight sensors all come fromt he same family of sensors, they change resistance according to the deformation that is sensed in the metal. They are used in scales and testing machines. One might want to use them to test a load on a mechanical arm for example so they can stop a motor if the weight becomes dangerous or avoid breaking something if an arm is attempting to open a door.<br />
<br />
Here's an example of a <a href="http://s.click.aliexpress.com/e/QfyjIyv">10KG weight sensor</a>:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5lrFxt4IDe90OxXOnoWpcx82HXxidqNKwxHppWZM1tQrOh_6h3fnOZtUorbGHqV6AVZuZMzS2xsdz-qp913uPDG62_sosQW2x4bWxg3WLKwn9J7slIicLRY0V0A-egHUdQQ7lwkYOPBc/s1600/Digital-Portable-Electronic-Kitchen-Scale-Load-Cell-Weight-Weighing-Sensor-Pressure-sensor-20KG-Driver-Voltage-3%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1000" data-original-width="1000" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5lrFxt4IDe90OxXOnoWpcx82HXxidqNKwxHppWZM1tQrOh_6h3fnOZtUorbGHqV6AVZuZMzS2xsdz-qp913uPDG62_sosQW2x4bWxg3WLKwn9J7slIicLRY0V0A-egHUdQQ7lwkYOPBc/s320/Digital-Portable-Electronic-Kitchen-Scale-Load-Cell-Weight-Weighing-Sensor-Pressure-sensor-20KG-Driver-Voltage-3%255B1%255D.jpg" width="320" /></a></div>
<h4>
Piezo</h4>
Piezo sensors have many uses, a piezo crystal is either flexing when power is applied to it, thus can make a sound or it can generate power when a force is applied to it. due to these features piezo are versatile.<br />
<br />
Piezo have been used as microphones, especially where vibration plays a big role, for example, as a guitar pickup microphone.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhaN2FQIIMddsB77tx9sAX7KVRsCEhljNCLnzbb0azz2LbgzywjN3w7eWvbkq_zocK0AXfL0SgRfTp1iWoMC5rs_8sXPeGecWmLjFqEpHavTHUkzBm7Dbq42u1cQG-ZIcr0nLEz1w-7MBA/s1600/Professional-Piezo-Contact-Microphone-Pickup-For-Guitar-Violin-Banjo-Mandolin-Ukulel-Guitar-Accessories.jpg_640x640%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="640" data-original-width="640" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhaN2FQIIMddsB77tx9sAX7KVRsCEhljNCLnzbb0azz2LbgzywjN3w7eWvbkq_zocK0AXfL0SgRfTp1iWoMC5rs_8sXPeGecWmLjFqEpHavTHUkzBm7Dbq42u1cQG-ZIcr0nLEz1w-7MBA/s320/Professional-Piezo-Contact-Microphone-Pickup-For-Guitar-Violin-Banjo-Mandolin-Ukulel-Guitar-Accessories.jpg_640x640%255B1%255D.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://s.click.aliexpress.com/e/aQJAQR7">Source</a></td></tr>
</tbody></table>
Wait what? guitars? aren't we talking about robotics? well, lets repurpose these sensors a bit, say you have a motor running, 24/7, that motor have bearings and like all bearings they wear off, you can claim that if your motor is running 24/7 you can calculate the service times, but what happens if the motor is under load only a few times a day and this changes the service time significantly. by detecting the vibrations coming off the motor, you can predict mechanical failure.<br />
<br />
Since piezo generate power on change, they can be used to detect "knocks" (or bumps or clicks) or even <a href="http://www.ohnitsch.net/2015/03/18/measuring-heart-rate-with-a-piezoelectric-vibration-sensor/">heartbeat</a>!<br />
<h4>
Gas</h4>
<div>
Gas sensors mostly split into two groups, spectroscopic sensing and chemical reaction sensing. while spectroscopic have a very long life, the chemical reaction ones have a relatively short lifespan and the catalyst will eventually deplete and no longer detect anything or detection accuracy will be low enough to make it useless.</div>
<div>
<br /></div>
<div>
Another type gas sensor is dust sensor, such as <a href="http://s.click.aliexpress.com/e/vnaaaie">GP2Y1014AU0F</a>:</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgry-d1z5aH9aTyloh2C2AFhybrs9fIoQSt-YISojTvACJUDBQ34BIXoeqB0ndgF0Ik1jl2e-CUVpO1_c4cFG9BywEYCgC80MMOp1n5vDdc8vHZDn69ozbo3bAuVOUm1qdmkf05DS3ZH6g/s1600/3_full%255B1%255D.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="358" data-original-width="500" height="229" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgry-d1z5aH9aTyloh2C2AFhybrs9fIoQSt-YISojTvACJUDBQ34BIXoeqB0ndgF0Ik1jl2e-CUVpO1_c4cFG9BywEYCgC80MMOp1n5vDdc8vHZDn69ozbo3bAuVOUm1qdmkf05DS3ZH6g/s320/3_full%255B1%255D.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://eereview.com/article/gp2y1014au0f-pm25-optical-dust-density-sensor">Source</a></td></tr>
</tbody></table>
<div>
CO2 sensors for example, come in a few different ways, for example, chemical (MG811) and spectroscopic (MH-Z19):</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgpBokiOwU5VfxISUs84OGNkvdFFA7zJcDptONc_9KOG79eRtRblt0ft4ytA-uYQ5ZuCslugD0xSC9A6UZ1BtRTUxkji4zMjl810rxvHR5j_6PEqL2Q2Oq1IkUyziYWk7g0H1S8NcJBsrg/s1600/1PCS-module-MH-Z19-infrared-co2-sensor-for-co2-monitor-Free-shipping-new-stock-best-quality%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="449" data-original-width="444" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgpBokiOwU5VfxISUs84OGNkvdFFA7zJcDptONc_9KOG79eRtRblt0ft4ytA-uYQ5ZuCslugD0xSC9A6UZ1BtRTUxkji4zMjl810rxvHR5j_6PEqL2Q2Oq1IkUyziYWk7g0H1S8NcJBsrg/s320/1PCS-module-MH-Z19-infrared-co2-sensor-for-co2-monitor-Free-shipping-new-stock-best-quality%255B1%255D.jpg" width="316" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://s.click.aliexpress.com/e/qvrrb2F">MH-Z19 - NDIR sensor</a></td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh6-4NY3LgSn61OBsBNtcizokQXRCmojfjX3gYhaEOrC6PlcsLVan_NafF-akiRJUYkEnNFLxUcFo1O_uwUqU2kuFRCyZyND0GcR15iwBRTlPYQPfgIpIPi0-aEjKw0ySlk_yCX_5uKW14/s1600/MG811-carbon-dioxide-sensor-CO2-sensor-gas-sensor-module-free-shipping%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="288" data-original-width="312" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh6-4NY3LgSn61OBsBNtcizokQXRCmojfjX3gYhaEOrC6PlcsLVan_NafF-akiRJUYkEnNFLxUcFo1O_uwUqU2kuFRCyZyND0GcR15iwBRTlPYQPfgIpIPi0-aEjKw0ySlk_yCX_5uKW14/s1600/MG811-carbon-dioxide-sensor-CO2-sensor-gas-sensor-module-free-shipping%255B1%255D.jpg" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://s.click.aliexpress.com/e/EuJM3Vz">MG811 - Chemical Sensor</a></td></tr>
</tbody></table>
<div>
<br /></div>
<div>
Other types of sensors are </div>
<div>
Combustible Gas ( LPG, Propane, Hydrogen, Methane and other combustible steam) - <a href="https://www.pololu.com/file/0J309/MQ2.pdf">MQ-2</a></div>
<div>
Alcohol/Ethanol and Benzine - <a href="http://www.china-total.com/Product/meter/gas-sensor/MQ-3.pdf">MQ-3</a></div>
<div style="direction: ltr;">
Combustible gas (Methane, Propane and Butane) - <a href="https://www.pololu.com/file/0J311/MQ4.pdf">MQ-4</a></div>
<div>
LPG, natural gas , town gas - <a href="https://www.parallax.com/sites/default/files/downloads/605-00009-MQ-5-Datasheet.pdf">MQ-5</a></div>
<div>
LPG, iso-butane, propane - <a href="http://www.china-total.com/Product/meter/gas-sensor/MQ-6.pdf">MQ-6</a></div>
<div>
CO - <a href="https://www.pololu.com/file/0J313/MQ7.pdf">MQ-7</a></div>
<div>
Hydrogen - <a href="https://dlnmh9ip6v2uc.cloudfront.net/datasheets/Sensors/Biometric/MQ-8.pdf">MQ-8</a></div>
<div>
LPG, CO, and Methane - <a href="http://www.haoyuelectronics.com/Attachment/MQ-9/MQ9.pdf">MQ-9</a></div>
Ozone - <a href="http://www.sensorsportal.com/DOWNLOADS/MQ131.pdf">MQ131 </a><br />
<div>
Ammonia, nitrogen oxide, alcohols, aromatic compounds, sulfide and smoke - <a href="https://www.robotshop.com/media/files/pdf/MQ-135-Gas-Sensor-UserManual.pdf">MQ-135</a><br />
Hydrogen sulfide - <a href="http://www.sensorica.ru/pdf/MQ-136.pdf">MQ136</a></div>
Ammonia - <a href="http://eph.ccs.miami.edu/precise/GasSensorSpecs/NH3.pdf">MQ137 </a><br />
Toluene, Acetone, Ethanol and Formaldehyde - <a href="http://www.china-total.com/Product/meter/gas-sensor/MQ138.pdf">MQ138 </a><br />
Freon - <a href="http://p.globalsources.com/IMAGES/PDT/SPEC/811/K1080325811.pdf">MQ139</a><br />
Methane LPG, i-butane, Propane - <a href="https://www.mysensors.org/dl/57c3ebeb071cb0e34c90057a/design/1341.pdf">MQ-214</a><br />
Alcohol - <a href="https://raw.githubusercontent.com/SeeedDocument/Grove-Alcohol_Sensor/master/res/MQ303A.pdf">MQ303A </a> / <a href="https://www.openhacks.com/uploadsproductos/2012916141216631.pdf">MQ303B </a><br />
CO2 Low Power - <a href="http://www.winsen-sensor.com/d/files/PDF/Solid%20Electrolyte%20CO2%20Sensor/MG812%20CO2%20Manual%20V1.1.pdf">MG-812</a><br />
<br />
Natural gas, LPG, Coal gas, alkane ect combustible gas, and gasoline, Alcohol, ketone, benzene ect organic solvent - <a href="http://www.china-total.com/Product/meter/gas-sensor/MC113-ShowRange.pdf">MC113</a><br />
<br />
CO2 - <a href="https://eph.ccs.miami.edu/precise/GasSensorSpecs/CO2.pdf">MG811 </a><br />
<br />
Methane - <a href="https://www.sgxsensortech.com/content/uploads/2015/01/DS-0152-MP-7217-TC-Datasheet-V2.pdf">MP7217</a><br />
Alcohol,smoke,formaldehyde, toluene, acetone, benzene, lighter gas, paint - <a href="https://www.compel.ru/item-pdf/a29dd20b796ddbf7069a627811de91a1/pn/winsen~mp901-air-quality-gas-sensor.pdf">MP901</a><br />
Formaldehyde - <a href="http://www.winsen-sensor.com/d/files/4-series-electrochemical-toxic-gas-sensor/me3-ch2o/me3-ch2o-0-10ppm(ver1_2)-manual.pdf">ME3M-CH2O</a><br />
Ammonia - <a href="http://www.winsen-sensor.com/d/files/PDF/Electrochemical%20Gas%20Sensor/4-series%20Electrochemical/ME3-NH3%200-100ppm(ver1.2)%20Manual.pdf">ME3-NH</a><br />
<br />
<br />
Organic Solvent Vapors - <a href="http://www.figarosensor.com/products/822pdf.pdf">TGS822</a><br />
Carbon Monoxide - <a href="https://cdn.sos.sk/productdata/af/2e/9901fb15/tgs-2442.pdf">TGS2442 </a><br />
Air Contaminants - hydrogen and carbon monoxide - <a href="http://www.figarosensor.com/products/2600pdf.pdf">TGS2600</a><br />
Air Contaminants - odorous gases
such as ammonia and H2S - <a href="http://www.figarosensor.com/products/2602pdf.pdf">TGS2602</a><br />
Air Contaminants - odorous gases
such as amine-series and sulfurous odors - <a href="http://www.figaro.co.jp/en/product/docs/tgs2603_product_information_rev02.pdf">TGS2603</a><br />
LP gas - <a href="http://midondesign.com/Documents/Figaro2610GasSensor.pdf">TGS2610</a><br />
Methane - <a href="http://www.figarosensor.com/products/2611pdf.pdf">TGS2611</a><br />
Solvent Vapors - <a href="http://www.figarosensor.com/products/2620pdf.pdf">TGS2620</a><br />
Carbon Monoxide - <a href="https://cdn.sos.sk/productdata/af/2e/9901fb15/tgs-2442.pdf">TGS2442</a><br />
Carbon Dioxide - <a href="https://cdn.sos.sk/productdata/62/d9/f2bb36a6/tgs-4161.pdf">TGS4161 </a><br />
Carbon Monoxide - <a href="http://www.figarosensor.com/products/5042pdf.pdf">TGS5042</a><br />
<br />
VOCs gases (toluene, formaldehyde, benzene, ect.) - <a href="https://cdn.instructables.com/ORIG/FZB/W1GP/IPTZRTHJ/FZBW1GPIPTZRTHJ.pdf">MS1100</a><br />
<br />
Methane Butane Hydrogen - <a href="http://www.kosmodrom.com.ua/data/gas/MR511.pdf">MR511 </a><br />
<br />
Most combustible gases and vapors - <a href="http://web.sensor-ic.com:8000/ZLXIAZAI/solidsense/Solidsenes_Classic_Line_CLE_4_and_7_Series_Gas_Sensors_rev072712.pdf">CLE-0951-400</a><br />
Nitric Oxide - <a href="https://euro-gasman.com/media/wysiwyg/Gas_Sensors/SOLIDSENSE/4_NO_2000.pdf">4NO-2000</a><br />
<br />
<div>
Oxygen - <span style="background-color: white; font-family: inherit; font-size: 18px; font-style: inherit; font-weight: inherit;">O2-A2 / AO2 / 2FO-N / KE-25 / ZE07-CO / ME2-O2 ME3-O2 / 4OXV O2 / ME3-C2H4O / OOM201 </span><br />
<span style="background-color: white; font-family: inherit; font-size: 18px; font-style: inherit; font-weight: inherit;"><br /></span>
<span style="background-color: white; font-family: inherit; font-size: 18px; font-style: inherit; font-weight: inherit;">So there's practically a sensor for almost any need, most of them are decently priced.</span></div>
<h4>
Resistive</h4>
In general, resistive sensors are a large class of sensors, among them are the previously written topics of load cells, flexing, temperature and various pressure sensors, they property of the material changes conductivity based on the forces and temperatures applied on them, some of these sensors need some kind of temperature compensation as resistance changes with temperature as well.<br />
<br />
Other types of resistive sensors are resistive touch screens, which work by changing the resistance with the force and place the mechanical stress is applied, that's why they work best with a stylus since a finger might apply force on more than a small location, thus changing the sensed resistance and eventually leading to bad localization. Another problem with resistive touch screens is temperature which affects accuracy.<br />
<br />
In the end resistive touch screens were abandoned but that technology is not completely useless, here's an example for a <a href="http://s.click.aliexpress.com/e/qfay3Bi">foot pressure point sensor</a>:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvwc0rRZSPm74nV-IffZVr89Q6FI1kE0PThQwx0XbF7kytXmtCfzBu-1X-KhqDwPaA8qi1Q60L73QKJVV117AJSdReSBUSFSeSH8-toaRapTBNC3h99pYVmANp8zC6-8ZKdeKkOKO-Nz0/s1600/foot+pressure+sensor.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="842" data-original-width="855" height="313" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvwc0rRZSPm74nV-IffZVr89Q6FI1kE0PThQwx0XbF7kytXmtCfzBu-1X-KhqDwPaA8qi1Q60L73QKJVV117AJSdReSBUSFSeSH8-toaRapTBNC3h99pYVmANp8zC6-8ZKdeKkOKO-Nz0/s320/foot+pressure+sensor.png" width="320" /></a></div>
<br />
Resistive sensors can also detect rotation angle, for example, <a href="http://s.click.aliexpress.com/e/3j6ImAa">potentiometers</a>, and slide, as <a href="http://s.click.aliexpress.com/e/nUNJiyR">slide potentiometers</a>.<br />
<br />
Analog servo motors use potentiometers, but the same principle applies, you can use a <a href="http://s.click.aliexpress.com/e/nUNJiyR">multi turn potentiometer</a> to achieve multiple turn servo!<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgF8YZ8GNL6DIWaydOWUj4jfOlxD2_MhMlsbgwvg-4s8HE5awSE8W6l8vJi9FqHIVtR3ZjZODIgGG9IkXZKU2VpuRugnleUn_dktXfRKHNSyiQnzzf0I3Fj1oN7ByKkiCacwfWu8z66gUA/s1600/article_12s%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="235" data-original-width="314" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgF8YZ8GNL6DIWaydOWUj4jfOlxD2_MhMlsbgwvg-4s8HE5awSE8W6l8vJi9FqHIVtR3ZjZODIgGG9IkXZKU2VpuRugnleUn_dktXfRKHNSyiQnzzf0I3Fj1oN7ByKkiCacwfWu8z66gUA/s1600/article_12s%255B1%255D.jpg" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://www.rcsails.com/sailwinch.html">Source</a></td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg16nbcy7virwjhL0KebBWoZmIxy0YfMe-c_GwrsSySHbmGRUDL_T5HynkyOHg1Vb88eBdaAZgUSbsD8VA-GF6AMdfgMx8a8NACSsEbl3YCYcqXUL_L1Ka_lM8gdFJJQCvxD2WX97b2V6Q/s1600/free-shipping-2pcs-lot-1-5-g-1-5g-servo-micro-digital-servo-loading-two-linear.jpg_640x640%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="600" data-original-width="600" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg16nbcy7virwjhL0KebBWoZmIxy0YfMe-c_GwrsSySHbmGRUDL_T5HynkyOHg1Vb88eBdaAZgUSbsD8VA-GF6AMdfgMx8a8NACSsEbl3YCYcqXUL_L1Ka_lM8gdFJJQCvxD2WX97b2V6Q/s320/free-shipping-2pcs-lot-1-5-g-1-5g-servo-micro-digital-servo-loading-two-linear.jpg_640x640%255B1%255D.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://s.click.aliexpress.com/e/iQfIIIU">Linear Servo</a></td></tr>
</tbody></table>
<br />
<br />
<h4>
Capacitive</h4>
Capacitive sensors work by detecting conductivity different than air which makes them useful for many things and they don't suffer from the same problems resistance sensors.<br />
<br />
Common usage is touchpads, phone screens, touch buttons, capacitive proximity sensors and capacitive soil moisture sensor which doesn't corrode as easily as resistive ones.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjaC_O2X2bfoHkTsnRHcGxCrdyxWMcLYTV9eZZ9AO8Nl4R4z838obnhWjjYeA5ytIz9mwsaPDo21AE6r9YN73wWngacodBmTcHF1cOvEfHjNJ_4IjCOcgtegPE5ke-MxFjuvg3Dx5DZG0/s1600/Electrode-grid-capacitive-touch-IC-diagram.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="359" data-original-width="619" height="370" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjaC_O2X2bfoHkTsnRHcGxCrdyxWMcLYTV9eZZ9AO8Nl4R4z838obnhWjjYeA5ytIz9mwsaPDo21AE6r9YN73wWngacodBmTcHF1cOvEfHjNJ_4IjCOcgtegPE5ke-MxFjuvg3Dx5DZG0/s640/Electrode-grid-capacitive-touch-IC-diagram.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Capacitive Touchpad<br />
<a href="http://www.cirque.com/capacitive-touch/">http://www.cirque.com/capacitive-touch/</a></td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOuB7bcLW1qYan12nub8lEjKhUcTg_07JXbWnKYdXDYWJDyed_tzHdu8qIlnfF-qChCWSrP8FNEZ5VFDqAWxVIfepo3OuC_YhMwjDWKXXiizLJ9kJWBCpJfoZG8rnlimrUVpWC44TR7QI/s1600/TTP223B-1-Channel-Jog-Digital-Touch-Sensor-Capacitive-Touch-Touch-Switch-Modules-Accessories-for-arduino-DIY.jpg_640x640%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="640" data-original-width="640" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOuB7bcLW1qYan12nub8lEjKhUcTg_07JXbWnKYdXDYWJDyed_tzHdu8qIlnfF-qChCWSrP8FNEZ5VFDqAWxVIfepo3OuC_YhMwjDWKXXiizLJ9kJWBCpJfoZG8rnlimrUVpWC44TR7QI/s320/TTP223B-1-Channel-Jog-Digital-Touch-Sensor-Capacitive-Touch-Touch-Switch-Modules-Accessories-for-arduino-DIY.jpg_640x640%255B1%255D.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://s.click.aliexpress.com/e/i6iAMRz">Touch Sensor</a></td></tr>
</tbody></table>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhdkRVcQqFIFPEMP73dvK0RYy9cGOHBv270SQNFplu-BDVl7rzHMyPZKQd6xVsriYn_us-9YqJTMAY96hkOfuAg0yI4lAvxN6iMrOEgPxPR_vM87UME2AQeoBzKY8o7Jl1VWfd557__Dwg/s1600/NEW-Capacitive-soil-moisture-sensor-not-easy-to-corrode-wide-voltage-wire-for-arduino.jpg_640x640%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="640" data-original-width="640" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhdkRVcQqFIFPEMP73dvK0RYy9cGOHBv270SQNFplu-BDVl7rzHMyPZKQd6xVsriYn_us-9YqJTMAY96hkOfuAg0yI4lAvxN6iMrOEgPxPR_vM87UME2AQeoBzKY8o7Jl1VWfd557__Dwg/s320/NEW-Capacitive-soil-moisture-sensor-not-easy-to-corrode-wide-voltage-wire-for-arduino.jpg_640x640%255B1%255D.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://s.click.aliexpress.com/e/EAIEMBE">Soil Moisture Sensor</a></td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiy2tdSENamlwGQ6Rkq08WyeO11obz2KYP8xIYkTCMpqrT3YV27cs17qwLJutQnfLkBWeIMJjyv9dyZqtvvRIM0LTj2zDb1fGak1DIbltZetjBRrjf7hKje0AMjaCzcaO_PQhj4IMj5s0Q/s1600/High-Quality-LJC18A3-H-Z-BX-1-10mm-Capacitance-Proximity-Sensor-Switch-NPN-NO-DC-6%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1100" data-original-width="1100" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiy2tdSENamlwGQ6Rkq08WyeO11obz2KYP8xIYkTCMpqrT3YV27cs17qwLJutQnfLkBWeIMJjyv9dyZqtvvRIM0LTj2zDb1fGak1DIbltZetjBRrjf7hKje0AMjaCzcaO_PQhj4IMj5s0Q/s320/High-Quality-LJC18A3-H-Z-BX-1-10mm-Capacitance-Proximity-Sensor-Switch-NPN-NO-DC-6%255B1%255D.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://s.click.aliexpress.com/e/i6iAMRz">Proximity Sensor</a></td></tr>
</tbody></table>
<br />
<br />
<h4>
Current/Voltage</h4>
Current sensors/voltage sensors are very important to keep a limited system within the power supply's boundaries or keeping a Lithium battery alive when the power runs out, so the battery voltage won't get below 3.2v (or its limit) per cell.<br />
<br />
DC Voltage sensing can use a very simple <a href="https://en.wikipedia.org/wiki/Voltage_divider">voltage divider</a>. and AC Voltage can use the same principle except that we need to put a <a href="https://en.wikipedia.org/wiki/Rectifier">rectifier</a> in front of it.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjX1hMOaOicm78JYVnTOzOB_NHHF0T4mj5xUW5GGUn7oP-UfxOlY3dz00RfE1tUA32MSRi3y-Va5dp0FOdOZximFjW_ubTRC3_Byh0Vpw5txxNdebkZCUCNVTJBydoOSUHJpGg_aqfYKBo/s1600/1PCS-Voltage-Sensor-Voltage-Detection-Module-DC0-25V-with-Code-Electronic-Building-Blocks%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1000" data-original-width="1000" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjX1hMOaOicm78JYVnTOzOB_NHHF0T4mj5xUW5GGUn7oP-UfxOlY3dz00RfE1tUA32MSRi3y-Va5dp0FOdOZximFjW_ubTRC3_Byh0Vpw5txxNdebkZCUCNVTJBydoOSUHJpGg_aqfYKBo/s320/1PCS-Voltage-Sensor-Voltage-Detection-Module-DC0-25V-with-Code-Electronic-Building-Blocks%255B1%255D.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://s.click.aliexpress.com/e/3NJEami">Source</a></td></tr>
</tbody></table>
Current sensing on AC works by induction, while DC current sensor works by measuring the voltage across a very low reistance component, such as a short wire, a 0.1/0.001 ohm resistor or dedicated sensors such as ACS758.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjzGIfZy8Q1na31wkHRz-R0IA6flTlBbBtnJAGDN9QE-qzWUQ73g7cAgLEZwxtQA7hPVOJ5_9pY_39LqyeNKw7xz3Yn_ICu12pXSEgtChYOQt_Nai3O9Zb7db6kDDOAsZMFu3w6s6AR5lQ/s1600/TENSTAR-ROBOT-30A-SCT-013-030-Non-invasive-AC-current-sensor-Split-Core-Current-Transformer-SCT.jpg_640x640%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="640" data-original-width="640" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjzGIfZy8Q1na31wkHRz-R0IA6flTlBbBtnJAGDN9QE-qzWUQ73g7cAgLEZwxtQA7hPVOJ5_9pY_39LqyeNKw7xz3Yn_ICu12pXSEgtChYOQt_Nai3O9Zb7db6kDDOAsZMFu3w6s6AR5lQ/s320/TENSTAR-ROBOT-30A-SCT-013-030-Non-invasive-AC-current-sensor-Split-Core-Current-Transformer-SCT.jpg_640x640%255B1%255D.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://s.click.aliexpress.com/e/zvRzNnY">Non-invasive AC current sensor</a></td></tr>
</tbody></table>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBoKwyzWMnWIz8oNinMGfoMu2lUTA_RY3LwcnOUXe5W1sv1ULBNCll-DcR4B3Ytm5Vq2A-Bi5OHzglptGWPLQi7uzae60sPjBWkAuvWLvgARbFugygSDCudV1H8SG0fOlQyyNThNA0TjQ/s1600/Panel-Mount-100Amp-AC-DC-Current-Sensor-Module-Board-based-on-ACS758%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1600" data-original-width="1600" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBoKwyzWMnWIz8oNinMGfoMu2lUTA_RY3LwcnOUXe5W1sv1ULBNCll-DcR4B3Ytm5Vq2A-Bi5OHzglptGWPLQi7uzae60sPjBWkAuvWLvgARbFugygSDCudV1H8SG0fOlQyyNThNA0TjQ/s320/Panel-Mount-100Amp-AC-DC-Current-Sensor-Module-Board-based-on-ACS758%255B1%255D.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://s.click.aliexpress.com/e/neYJuZB">100Amp AC/DC Current Sensor Module Board, based on ACS758</a></td></tr>
</tbody></table>
<br />
<h4>
Biological</h4>
Biological sensors are useful for detecting or authenticating a person, so if your robot depends on someone's pulse or blood oxygen level, you can use a <a href="https://en.wikipedia.org/wiki/Pulse_oximetry">pulse oximeter</a> for that, its a non invasive sensor that measures the difference between IR and red light absorbance of skin.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh53Z2r7ZByW7TX4_m2Rath8VaHaZy653o726oaE-zUuks_owuDZzElFQsJeSUqNW6Ae3skRkmJgjbeyv5i_39HxsEvJjiAhn9APHy-tIzeQuFySXO8WclhM8Q89tjQk4JGpmy45tYTdss/s1600/Heart-Rate-Click-MAX30100-modules-Sensor-for.jpg_640x640%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="640" data-original-width="640" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh53Z2r7ZByW7TX4_m2Rath8VaHaZy653o726oaE-zUuks_owuDZzElFQsJeSUqNW6Ae3skRkmJgjbeyv5i_39HxsEvJjiAhn9APHy-tIzeQuFySXO8WclhM8Q89tjQk4JGpmy45tYTdss/s320/Heart-Rate-Click-MAX30100-modules-Sensor-for.jpg_640x640%255B1%255D.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://s.click.aliexpress.com/e/yVzFA2Z">MAX30100 Pulse Oximeter</a></td></tr>
</tbody></table>
<br />
<br />
Another thing someone might want to sense is muscle/heart/brain signals, which can be done with <a href="https://millar.com/research/applications/telemetry-biopotential">EMG/ECG/EEG</a> sensors.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgdRaoC3QjIJbPIKQmRLytF0D8DV3fpfF6DQauK-VpnnILt2vm0C4QSaPOilzPFimUTdyvnZQxjNkok0cAttlvL0rxffyySB5GVryGZj2Var-zy2QsTzSxMrcwxHFjxFmCjZ9oJa6fwX64/s1600/Smart-Electronics-Muscle-Signal-EMG-Sensor-Module-Grove-interface-Diy-Kit.jpg_640x640%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="500" data-original-width="500" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgdRaoC3QjIJbPIKQmRLytF0D8DV3fpfF6DQauK-VpnnILt2vm0C4QSaPOilzPFimUTdyvnZQxjNkok0cAttlvL0rxffyySB5GVryGZj2Var-zy2QsTzSxMrcwxHFjxFmCjZ9oJa6fwX64/s320/Smart-Electronics-Muscle-Signal-EMG-Sensor-Module-Grove-interface-Diy-Kit.jpg_640x640%255B1%255D.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://s.click.aliexpress.com/e/7mUznQB">Muscle Signal EMG Sensor</a></td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhJX7XdiWEUY7xfZ9_ApfveGcXNQl5TseQTsE91UotZf8_e9wb9tsMaEsXJTtaZDZUdtj1AKjNTmGt9jZvYAjCPksIiQp5JNnWIGDKuecUP9ZghkmUrPhU-9WA3AjQzzvS2feq8ELRC_lw/s1600/WAVGAT-Ecg-module-AD8232-ecg-measurement-pulse-heart-ecg-monitoring-sensor-module-kit.jpg_640x640%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="640" data-original-width="640" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhJX7XdiWEUY7xfZ9_ApfveGcXNQl5TseQTsE91UotZf8_e9wb9tsMaEsXJTtaZDZUdtj1AKjNTmGt9jZvYAjCPksIiQp5JNnWIGDKuecUP9ZghkmUrPhU-9WA3AjQzzvS2feq8ELRC_lw/s320/WAVGAT-Ecg-module-AD8232-ecg-measurement-pulse-heart-ecg-monitoring-sensor-module-kit.jpg_640x640%255B1%255D.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://s.click.aliexpress.com/e/76AUrjE">ECG module AD8232</a></td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1RlsElzcaah0dhoEnbbBue46znhaSI2AFj1yZHBwzq5wWylI1Ej4zhyZVfKH0u_SQGJFfdl1Yb_qk8tecF6ca9egox0THBbh17gRu0Ck1oiC6qaojkfZFNocjP0AXsm8E_8JQ8ANGQdQ/s1600/eeg.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="640" data-original-width="480" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1RlsElzcaah0dhoEnbbBue46znhaSI2AFj1yZHBwzq5wWylI1Ej4zhyZVfKH0u_SQGJFfdl1Yb_qk8tecF6ca9egox0THBbh17gRu0Ck1oiC6qaojkfZFNocjP0AXsm8E_8JQ8ANGQdQ/s320/eeg.jpg" width="240" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://s.click.aliexpress.com/e/Un62Jy3">8bit EEG brain wave module -8 Channel </a></td></tr>
</tbody></table>
<br />
Fingerprints, lately it seems fingerprint sensors are all over the place, you can get them very cheaply and in various sizes. A very fun project I've seen someone do is build thor's hammer with fingerprint reader.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjf4GHHfPUGaSId3kd7gmLZYZUURYP1DJvDAibWDeUEf9rEGzbSDpo5_HKVLhTH9KsdHWpUb54tpeG9gRP2GBZfd4eaftqNXnjlgS0FAq2QhNCqqOb6BwQbpHaf6ltYMf3v_CRZw1iiN4w/s1600/FPM10A-Fingerprint-Reader-Sensor-Module-Optical-fingerprint-Fingerprint-Module-For-Arduino-Locks%255B1%255D.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1000" data-original-width="1000" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjf4GHHfPUGaSId3kd7gmLZYZUURYP1DJvDAibWDeUEf9rEGzbSDpo5_HKVLhTH9KsdHWpUb54tpeG9gRP2GBZfd4eaftqNXnjlgS0FAq2QhNCqqOb6BwQbpHaf6ltYMf3v_CRZw1iiN4w/s320/FPM10A-Fingerprint-Reader-Sensor-Module-Optical-fingerprint-Fingerprint-Module-For-Arduino-Locks%255B1%255D.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://s.click.aliexpress.com/e/m6EEAY7">FPM10A Fingerprint Reader Sensor</a></td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/0_8Xhzt5YQI/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/0_8Xhzt5YQI?feature=player_embedded" width="320"></iframe></div>
<br />
<br />
<br />
<br />
<br />Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3077621944057198788.post-19116221184465621882018-03-07T11:25:00.000-08:002018-03-07T11:25:17.638-08:00RTree v1.1.0<a href="https://github.com/drorgl/cspatialindexrt">RTree</a> was one of the <a href="http://uhurumkate.blogspot.com/2009/04/rtree.html">first</a> projects I've contributed to open source, its the seed that started this blog and a tool for a past thought.<br />
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjR9xtZzGOZCHKW8v4kFLqkcf7T_qCB0kjAXGwM1qztP5iA1DMkMBSW71y7Q1F_sIoamxyc7IiRIH4B-nr-BcMECEr466ugv4ZAejQAjOMTSQxqo-o1ea6s7LNzSmZfRmJrO6hqAhnUmTg/s1600/rtree+nuget.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="170" data-original-width="264" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjR9xtZzGOZCHKW8v4kFLqkcf7T_qCB0kjAXGwM1qztP5iA1DMkMBSW71y7Q1F_sIoamxyc7IiRIH4B-nr-BcMECEr466ugv4ZAejQAjOMTSQxqo-o1ea6s7LNzSmZfRmJrO6hqAhnUmTg/s1600/rtree+nuget.png" /></a></div>
<br />
That being said, it was written for .NET framework and while looking at nuget stats, once a day is no where near popular, its already accumulating dust, doesn't support threads or .net core.<br />
<br />
So, time for an upgrade.<br />
<br />
The project has been converted to .NET Standard 2.0<br />
<br />
Added support for multithreading by removing a limitation and adding ReaderWriterLock and fixing a bug that makes deletes impossible.<br />
<br />
<a href="https://www.nuget.org/packages/RTree/">https://www.nuget.org/packages/RTree/</a><br />
<br />
<br />Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3077621944057198788.post-53380924824100093952018-01-23T09:43:00.001-08:002018-01-23T09:43:35.651-08:00Reusing a Laptop CameraA few months ago I came across <a href="http://www.instructables.com/id/Reuse-old-laptop-webcam/">this instructable</a>, its showing how to reuse a laptop camera module as a usb device, which some of them use. so later when I've decided to clean up and get rid of old and defunct laptops, I've found two camera modules and attempted to repurpose them.<br />
<br />
The module's quality is nothing to be impressed by but they are UVC compatible, which should be good for linux use as well, but I'll test them at a later time.<br />
<br />
The connectors seem like a standard, the following diagram is from the instructable, one of my modules even had the pinouts on the silkscreen!<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgrdpzZNdM3dkRsywl37p0VvIsNcLZy6MHmHD3RGMxMx4ELjhWfl8lAl3OKrZ7C2n3vzTJtZZjAhEc4G1V_wbeMHNaxcUr9o93dqGXwyZTRIEdXQRm5m24xv_2CTsFAY77D6eIr8XMsCYE/s1600/F9JDREBIAP6YFJD.LARGE%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="472" data-original-width="1024" height="295" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgrdpzZNdM3dkRsywl37p0VvIsNcLZy6MHmHD3RGMxMx4ELjhWfl8lAl3OKrZ7C2n3vzTJtZZjAhEc4G1V_wbeMHNaxcUr9o93dqGXwyZTRIEdXQRm5m24xv_2CTsFAY77D6eIr8XMsCYE/s640/F9JDREBIAP6YFJD.LARGE%255B1%255D.jpg" width="640" /></a></div>
<span id="goog_1331323513"></span><span id="goog_1331323514"></span><br />
While my devices are:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhM5bFLgTFlSEzzlNf6MdGxmxCxE0N_5WwL8rY5svDyNlJHH2XdsR7-Rmwi-D7FY7TB9xTvwzK_0W6DS8yKIqWu_FTmE72DSfx4kBTKaNa_nhZ6kjIgNrpAbEcOIzKTsobxGxfByDXF5Ik/s1600/20180123_191327.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="733" data-original-width="1600" height="292" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhM5bFLgTFlSEzzlNf6MdGxmxCxE0N_5WwL8rY5svDyNlJHH2XdsR7-Rmwi-D7FY7TB9xTvwzK_0W6DS8yKIqWu_FTmE72DSfx4kBTKaNa_nhZ6kjIgNrpAbEcOIzKTsobxGxfByDXF5Ik/s640/20180123_191327.jpg" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEik2IDa6KAQjZsBD-BDFpXHcrS9YY5znabv8SNvf9pSomIX-6UgoCRfjYi1dmsqhVS82neRFCXguhLEfaMmsj0JRjwPkC5duYorIES3sATffq6mwO-CRmyqJF5BiPtgWlxpcm-1Ne5mub0/s1600/20180123_191402.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="710" data-original-width="1600" height="282" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEik2IDa6KAQjZsBD-BDFpXHcrS9YY5znabv8SNvf9pSomIX-6UgoCRfjYi1dmsqhVS82neRFCXguhLEfaMmsj0JRjwPkC5duYorIES3sATffq6mwO-CRmyqJF5BiPtgWlxpcm-1Ne5mub0/s640/20180123_191402.jpg" width="640" /></a></div>
<br />
<br />
<br />
I've had a few USB connectors laying around, so I've looked up the pinouts<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8MSIY5bZtaqv4C4v9Fnse8h6fsGOEq5OLqAbm2ja68BmKqB0eVTEOtlPYSg4IlFeBiJfBM8lYF5nwdZUICadMriLjyLkM21eHMsRG9MoSCxmDx1fzWyQ3uwxCNmJHOZoGcjIVRiZXEZw/s1600/F6UY6LMIN0FGCY7.LARGE%255B1%255D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="877" data-original-width="1024" height="171" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8MSIY5bZtaqv4C4v9Fnse8h6fsGOEq5OLqAbm2ja68BmKqB0eVTEOtlPYSg4IlFeBiJfBM8lYF5nwdZUICadMriLjyLkM21eHMsRG9MoSCxmDx1fzWyQ3uwxCNmJHOZoGcjIVRiZXEZw/s200/F6UY6LMIN0FGCY7.LARGE%255B1%255D.jpg" width="200" /></a></div>
<br />
<br />
<br />
Soldered the connectors<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgL1R8763QHFwhEngJjSsHyCiFtNHb4j9AaqiGtUku_LeYnMo_RLQgKHTD8PrZYK66uozjKP6Oa9bjdjWXL0XMWIxJucinAz-VwnMhaHghbHqgY0NdDvVI6kbzr4UDNF-XZT5wHnA941xU/s1600/20180123_191443.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="630" data-original-width="1600" height="248" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgL1R8763QHFwhEngJjSsHyCiFtNHb4j9AaqiGtUku_LeYnMo_RLQgKHTD8PrZYK66uozjKP6Oa9bjdjWXL0XMWIxJucinAz-VwnMhaHghbHqgY0NdDvVI6kbzr4UDNF-XZT5wHnA941xU/s640/20180123_191443.jpg" width="640" /></a></div>
<br />
<br />
and tested the cameras are not a short circuit risk with a <a href="https://www.aliexpress.com/item/7-in-1-USB-tester-DC-Digital-voltmeter-amperimetro-current-voltage-meter-amp-volt-ammeter-detector/32845359274.html">USB tester</a><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEig3lQT9XTSDND9J4i6YOt6XFDWxwa5FpDSsjU8awJBj6uj6orWntD2WoyW8JPuNPUg12ISYIbXUD0L9-X4i3jf8w5X7SK7ZzjyJ_ZRayLDWYl51qypf5ApMqVmcinGDU9abrU_Dvl3KCo/s1600/usb+tester.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="220" data-original-width="505" height="139" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEig3lQT9XTSDND9J4i6YOt6XFDWxwa5FpDSsjU8awJBj6uj6orWntD2WoyW8JPuNPUg12ISYIbXUD0L9-X4i3jf8w5X7SK7ZzjyJ_ZRayLDWYl51qypf5ApMqVmcinGDU9abrU_Dvl3KCo/s320/usb+tester.png" width="320" /></a></div>
<br />
<br />
The first showed up as VID_05E3&PID_0505, which is a Genesys Logic USB2.0 PC Camera, its power consumption is around 80ma.<br />
<br />
The second one showed up as VID_04F2&PID_B106, which is a Chicony® USB M UVC WebCam, its power consumption is around 110ma.<br />
<br />
Next, testing them with an Orange PI....<br />
<br />Unknownnoreply@blogger.com1