300x250 AD TOP

Search This Blog

Pages

Featured Post

Tiny Model for MNIST Dataset

The MNIST database is a database of handwritten digits, its used as an ideal beginner dataset for learning how to do simple image classific...

Paling Dilihat

Powered by Blogger.

Feature Label Area

Tuesday, June 12, 2018

Tiny Model for MNIST Dataset

The MNIST database is a database of handwritten digits, its used as an ideal beginner dataset for learning how to do simple image classification and as the dataset only contains 10 characters, its relatively easy to work with.

It has 60000 tranining examples and 10000 testing examples and it is sufficiently large. as it has 60000 images of 28 x 28 grayscale images, it takes about 50MB, so it does not add complexity for batch generation since it can all fit in memory.

Like I said, ideal.


I think the first lesson I did on image classification was with MNIST database as well. it was very amusing to see a 1.2 million parameters model for a 50MB dataset, so I've decided to see how low I can go.

Lets start with the big one.

1.2m parameters - LeNet



CNN Error: 0.80%
train_loss 0.0082
train_acc 0.9984
val_loss: 0.0552
val_acc:0.9920
The graph does look like its overfitting by a bit.


My first experiment was using huge convolutions, I've managed to train 99% on 300k parameters, but I was not satisfied, surely there is a better way.

This model at 0.992 accuracy, with only 36k parameters (!!)


CNN Error: 0.72%
train_loss 0.0224
train_acc 0.9928
val_loss 0.0255
val_acc 0.9928

36k only? well, that's huge, I've looked around and found out its possible with under 4k parameters, so I set up for the challenge and came up with this model.

CNN Error: 0.90%
train_loss: 0.0369
train_acc: 0.9880
val_loss: 0.0285
val_acc: 0.9910

model is under 4k parameters (3.8)



To summarise what I've learned from this exercise is that larger models will learn faster but also overfit faster, smaller models need more training to find a better fit.


Credits

I would like to say thank you to EliteDataScience.com for getting this little exercise started



My 36k model:



My 4k model:

Tags:

Friday, April 27, 2018

Practical Sensing - RF

We, as humans, are so used to know who we are and where we are that we sometimes forget that it comes at a cost, a person sees where they are, they have an awareness (or general awareness) of their location in a room or on the street, they can also see a step (or feel if one is blind) and eventually know where obstacles are, get from a place to place and plan routes around obstacles.

https://pixabay.com/en/eye-internet-forward-vision-669157/

What can computers do? 

Radio Sensing

Radio sensing has been invented 1904 and while only capable of detecting a presence it has since evolved into many other sensors.

RF Radar has a few implementations, among them is presence, distance and movement, the principle is the same, an RF wave is transmitted, it is then reflected (or not) and the returned radio waves are detected, in more advanced scenarios, the received signal goes through FFT to detect the reflected timing/phase, which is further processed to get distance and/or speed of one or more objects.

Another implementation of radio sensing is localization, in its simplest form triangulation and more advanced is GPS which uses clocks to indicate when the signal was sent to better localize the receiver. BLE have been used for indoor navigation as well.

A different method of obtaining general location as long as you have network connectivity is using a service such as google geolocate which uses your IP and near by WIFI networks to guess the location.

An attempt was made to discover the hackability of Bosch Radars (diydrones, mikrocontroller) but so far without success.

Another form of electromagnetic sensing is a Geiger counter, which can be used to detect radiation, very useful if you want to detect Radon in your basement or take a relatively safe hike near Chernobyl.

Lastly Radio can be used as a cheap way to find out if a certain device is near another device or even communicate a secret of some sort.

Radar

Doppler - Doppler type sensors detect change or movement, one such cheap sensor is the HB100.

https://www.tindie.com/products/optimusdigital/hb100-microwave-sensor-module/


Distance - Sensors such as FM24-NP100 ($110) provide distance to the biggest reflection but also spectrum data which can be used to monitor multiple objects. These type of sensors measure the phase difference between two wavelengths.



Presence - Modules like the HW-MS03 (about $2) are in essence a Doppler radar combined with a timer to switch a pin/relay on or off.



Some radar modules (such as CFK024-5A for about $50) have FMCW tuning capabilities, which is very useful if you want to do a sweep which can be used to detect distance of multiple objects but it requires more than basic knowledge.


GPS

GPS receivers determine location by triangulating the timestamps and signals received from satellites, the more satellites, the more accurate the location will be. but GPS technology is limited by atmospheric conditions, limiting the accuracy possible.

GPS receivers have advanced over the years, GPS L1 and L2, GloNass, Galileo, Beidou and more, but the accuracy stayed more or less the same at this moment, the peak is around 2.5 meters accuracy for private use.

To overcome the accuracy limitations, a few augmentations were developed, some are over the air such as SBAS and QZSS, some are based on static base stations like DGPS and RTK.

Commercial RTK solutions are provided by drotek and Emlid to name a few.


BLE

BLE beacons are low energy (hence LE) devices which transmit a message once in a while, by reading the received power level (RSSI), it is possible to estimate the distance to the beacon. By knowing where the beacons are, it is possible to triangulate (called trilateration) the location of the receiver.



proximi.io (no affiliation) is one of the companies that does that kind of indoor positioning.

Nuclear Radiation

Following Fukushima disaster and Chernobyl tourism, personal radiation detection devices became more and more popular. 

Dosimeters available in Film Badges, MOSFET and Geiger-Muler tubes to name a few.


Proximity/Identity

Wireless identity devices are devices that contain a chip with a small coil, the coil is used to power up the chip and transmit data. two such devices are RFID and NFC devices.

Some devices have static data, some can store custom data, some can encrypt and authenticate but the principle is the same.

RFID more common types are the 125KHz and 13.56MHz and contain a 20 bytes ID and anywhere between 0 and 64 bytes of custom data.

13.56MHz module
125Khz module

One of the more interesting things about RFID is the tag sizes




NFC works much in the same way, memory capacity is between 48 bytes and 32kb.

Radio

While radio communication is not a sensor per say it is however a way to communicate and locate and can be used to sense location, state and various data.

Standard modules come in various frequencies and modulations, some even implement protocols, error correction and buffers.

Among the more popular ones are the 315Mhz, 433Mhz, 868Mhz, 915Mhz, depending on the country and local regulations and considered ISM band, which can be used for anything from car remotes, multirotor telemetry and various remote switches.

433Mhz RF transmitter and receiver 

XBEE, Bluetooth and WIFI are also considered ISM band but more robust implementations are available and are mostly used for higher bandwidth application.

XBee

Some of the more popular 2.4Ghz modules are the NRF24L01, A7105 and CC2500 which are used in RC Toys.

FrSky Taranis Q X7S

Lora, LoraWan, SigFox and NB-IOT are mostly used for smart appliances such as water meters, power meters and relatively long range and big coverage requirements and are very low bandwidth.

Cell (2G/3G and up) are used nowadays for anything from messaging to video playback.

SIM800L


References:

https://upcommons.upc.edu/bitstream/handle/2099.1/6629/PFC_MarcMirTutusaus.pdf?sequence=1
https://fccid.io/NF3ACC2SCU/User-Manual/Installer-Instructions-413993
https://etd.ohiolink.edu/!etd.send_file?accession=ohiou1304083389&disposition=inline
https://hackaday.com/2017/05/24/radar-sensors-put-to-the-test/

Tags: ,

Saturday, March 31, 2018

Practical Sensing - Physical

We, as humans, are so used to know who we are and where we are that we sometimes forget that it comes at a cost, a person sees where they are, they have an awareness (or general awareness) of their location in a room or on the street, they can also see a step (or feel if one is blind) and eventually know where obstacles are, get from a place to place and plan routes around obstacles.

https://pixabay.com/en/eye-internet-forward-vision-669157/

What can machines do? 

This is a quick introduction about sensors, what are the available options for a beginner maker/developer.

Physical Sensing

"Sensors" such as switches, magnetic, sound (or ultrasonic), pressure, temperature, flexing (or bending), shear (or weight), piezo, gas, capacitive and resistive sensors have been around the block, simple vacuum robots have some of them, cars have them (not necessarily self driving ones, see parking sensors) and even old refrigerators have them (someone has to turn on the light when you're hungry at 4AM).

Switches

Switches and buttons come in all shapes and sizes but the mechanical principle is the same, there are two (or more) spaced conductors, when a force is applied, one of the conductors (or a third) is moving toward them and makes a connection.

https://www.omron-ecb.co.kr/web/en/special/switch/basic02-05

Other types of switches can be limit, rocker, toggle, DIP, push buttons, thumbwheels and more, but the principle is the same applied in different ways and are represented very similarly.

https://www.edrawsoft.com/switch-symbols.php
Among these mechanic switches there are also simple tilt switches, they work by the same principle where a metal ball is the connection point.



Mercury tilt switches got infamous in the movies for less favoring reasons

Source



So what are switches and buttons good for?

Sensing mechanical contact, bumping into walls, rotational forces, etc'

What are they not good for?

Anything that requires precision and detection of the amount of pressure, they are on-off, nothing more.

Magnetic

A magnetic sensor is a type of sensor capable of sensing magnetic fields, magnetic fields are a directional field but the direction can not always be sensed, for example a reed switch will close (N.O) or open (N.C) based on the presence of magnetic field. We can also sense the presence and direction of a magnetic field with Hall Effect sensors (1D) and electronic compass if we require a 3D sensing of electric fields.



http://www.memsjournal.com/2011/02/motion-sensing-in-the-iphone-4-electronic-compass.html
These electronic compasses are so cool and sensitive you can use them to visualize a magnetic field!

Since hall sensors are so inexpensive and accurate, they can be used for rotational speed detection (odometry) and water/air flow sensors.



The principle is the same for most speed sensors, a moving part and a detector.

Source


Another very interesting application of hall sensor is 3D absolute sensing like the MLX90333:

Source


Another type of magnetic sensing is induction sensing, this is what metal detectors do, the way they work is by generating a strong magnetic field and detecting changes in the feedback, induction sensors are used in 3D printers to detect the build plate, they are pretty accurate and repeatable, but there are better sensors for that purpose as well and they are limited by the fact that they can only sense metals.

Common usages:
Neato Boundary Sensor


So what are they good for?
The reed switches are used as an on-off switch when a magnet is near by, the common example are windows and doors sensors for alarms. The hall effect sensors are used to precisely detect a magnetic object and the compasses are used to orient against magnetic north or artificial magnetic field.

What are they not good for?
If the environment is magnetically "dirty", many motors and magnets, both the hall effect and the compass are going to have a hard time "locking", also, electronic compasses are relatively slow, so they can not be used to keep the orientation for a moving object, for that purpose they are combined with accelerometers and gyros and their data is combined (or "fused" - see sensor fusion) to keep the robot oriented until the compass catches on.

Sound

Sound has very interesting properties with regard to robots, it bounces easily, its travelling relatively slow and sound sensors are abundant (microphones for example).

These leads us to a very simple implementation: distance sensors, they send a ping, wait for the response and report the time it took, Mechatronics has a nice tutorial.

Another very interesting usage is direction detection, I've seen 3 mic arrays, 6 mic arrays and they come with SDKs and can do noise reduction and direction detection.


Another interesting use for ultrasonic sound is indoor navigation, in theory, one can triangulate the source of a sound pulse by the time it takes to get to the transceivers.

Thanks to Marvelmind its no longer a theory:



Common usage would be distance sensors, atomizers, voice commands.

So what is it good for?

When you need to detect distance to a flat surface (rememebr, sound bounces..), when you need to reduce noise in a noisy environment or when you need to detect the direction of where the sound is coming from.

What is it not good for?

When the surfaces are not perpendicular to the sensor, sound will bounce all over the place, you should be thankful for any reading.

Interesting libraries:

Pressure

Pressure sensors are useful for many things, for hydraulic systems they can detect leaks, for water systems they can detect presence or water pump quality or even regulating pressure with a PID loop.

Water Pressure Sensor


Water pressure sensors can detect depth in submersible robots.

300m Depth/Pressure Sensor


Air Pressure sensors are also very useful for weather stations as they can detect weather fronts and predict rain.

Air Pressure sensors can also help multi-rotors keep a certain height, not accurately though, but good enough. Just don't count on them to work on bad weather, I've seen a multi-rotor drop 10 meters on a bad weather day and go back up in a blink of an eye, you can't blame the electronics..

A common type of air pressure sensor is the BMP280:

Air Pressure sensors are also used to detect speed in aircrafts.

All pressure sensors measure from a certain reference point, gauge, absolute and differential are relative and you should know what you need.

Make sure the pressure sensor is suitable for your medium and amount of stress its going to take, note the breakdown pressure so no one will lose an eye/finger and if dealing with high pressure, a suitable burst valve is always recommended.

Temperature

Temperature sensors are useful in cases when you need to know the temperature, for example, is your motor running hot? did the water boil already? is the oil too hot? is the printer's hotend at the right temperature? is it hot outside?

Like all other sensors, temperature sensors come in different flavors, but they divide into 2 groups: conduction and radiation sensors.

A common air/weather convection temperature sensor is the DHT11 and DHT22, these are very cheap:



A common type of conduction temperature sensor is the NTC 100k thermistor
Another common type of sensor is the contact-less IR temperature sensor, Melexis makes some of them:

BTW, the same principle applies for IR cameras, which produce these cool images:


Two consumer grade companies makes these sensors, Seek and FLIR, both cost around $200 for the cheap versions and of-curse you can get sensors only, but that is a different adventure.


Another kind of temperature sensor is the thermostats, which can be used for thermal protection, no sensing, just turn on the fan when its too hot, or turn off the power. they are not accurate but they do a good job for their designed purpose.

I've used the KSD9700 to quiet down a power supply that had a fan always on:


Lastly the PTC thermistors, they are a type of resistor that when they get too hot, their resistance jumps significantly, they are used to protect over-current and are regarded as self resetting fuse as the system cools off, the power returns to normal.

This is a nice kit to get started:


Flexing 

Flexing (or bending) sensors are basically resistors that change their resistance when they are bent, they are great for human interaction since, if you think about it, we have many joints that flex and tracking these flexing movements is not trivial by other means, so gloves, sleeves, tights, if it bends, you can detect it with these little sensors.

Be careful though, these sensors are not cheap and the material melts away if attempting to solder it, you might be able to fix it with conductive silver paint, avoid heat as much as possible and avoid rubbing it in the exposed areas.

Here's a 2.2" flex sensor, they come at various lengths:

Shear / Load Cell / Weight

Shear beam, Load Cells and Weight sensors all come fromt he same family of sensors, they change resistance according to the deformation that is sensed in the metal. They are used in scales and testing machines. One might want to use them to test a load on a mechanical arm for example so they can stop a motor if the weight becomes dangerous or avoid breaking something if an arm is attempting to open a door.

Here's an example of a 10KG weight sensor:

Piezo

Piezo sensors have many uses, a piezo crystal is either flexing when power is applied to it, thus can make a sound or it can generate power when a force is applied to it. due to these features piezo are versatile.

Piezo have been used as microphones, especially where vibration plays a big role, for example, as a guitar pickup microphone.

Source
Wait what? guitars? aren't we talking about robotics? well, lets repurpose these sensors a bit, say you have a motor running, 24/7, that motor have bearings and like all bearings they wear off, you can claim that if your motor is running 24/7 you can calculate the service times, but what happens if the motor is under load only a few times a day and this changes the service time significantly. by detecting the vibrations coming off the motor, you can predict mechanical failure.

Since piezo generate power on change, they can be used to detect "knocks" (or bumps or clicks) or even heartbeat!

Gas

Gas sensors mostly split into two groups, spectroscopic sensing and chemical reaction sensing. while spectroscopic have a very long life, the chemical reaction ones have a relatively short lifespan and the catalyst will eventually deplete and no longer detect anything or detection accuracy will be low enough to make it useless.

Another type gas sensor is dust sensor, such as GP2Y1014AU0F:

Source
CO2 sensors for example, come in a few different ways, for example, chemical (MG811) and spectroscopic (MH-Z19):

MH-Z19 - NDIR sensor
MG811 - Chemical Sensor

Other types of sensors are 
Combustible Gas ( LPG, Propane, Hydrogen, Methane and other combustible steam) - MQ-2
Alcohol/Ethanol and Benzine - MQ-3
Combustible gas (Methane, Propane and Butane) - MQ-4
LPG, natural gas , town gas - MQ-5
LPG, iso-butane, propane - MQ-6
CO - MQ-7
Hydrogen - MQ-8
LPG, CO, and Methane - MQ-9
Ozone - MQ131
Ammonia, nitrogen oxide, alcohols, aromatic compounds, sulfide and smoke - MQ-135
Hydrogen sulfide - MQ136
Ammonia - MQ137 
Toluene, Acetone, Ethanol and Formaldehyde - MQ138 
Freon - MQ139
Methane LPG, i-butane, Propane - MQ-214
Alcohol - MQ303A  / MQ303B 
CO2 Low Power - MG-812

Natural gas, LPG, Coal gas, alkane ect combustible gas, and gasoline, Alcohol, ketone, benzene ect organic solvent - MC113

CO2 - MG811

Methane - MP7217
Alcohol,smoke,formaldehyde, toluene, acetone, benzene, lighter gas, paint - MP901
Formaldehyde - ME3M-CH2O
Ammonia - ME3-NH


Organic Solvent Vapors - TGS822
Carbon Monoxide - TGS2442 
Air Contaminants - hydrogen and carbon monoxide - TGS2600
Air Contaminants - odorous gases such as ammonia and H2S - TGS2602
Air Contaminants - odorous gases such as amine-series and sulfurous odors - TGS2603
LP gas - TGS2610
Methane - TGS2611
Solvent Vapors - TGS2620
Carbon Monoxide - TGS2442
Carbon Dioxide - TGS4161 
Carbon Monoxide - TGS5042

VOCs gases (toluene, formaldehyde, benzene, ect.) - MS1100

Methane Butane Hydrogen - MR511 

Most combustible gases and vapors  - CLE-0951-400
Nitric Oxide - 4NO-2000

Oxygen - O2-A2 / AO2 / 2FO-N / KE-25 / ZE07-CO / ME2-O2 ME3-O2 / 4OXV O2 /  ME3-C2H4O / OOM201 

So there's practically a sensor for almost any need, most of them are decently priced.

Resistive

In general, resistive sensors are a large class of sensors, among them are the previously written topics of load cells, flexing, temperature and various pressure sensors, they property of the material changes conductivity based on the forces and temperatures applied on them, some of these sensors need some kind of temperature compensation as resistance changes with temperature as well.

Other types of resistive sensors are resistive touch screens, which work by changing the resistance with the force and place the mechanical stress is applied, that's why they work best with a stylus since a finger might apply force on more than a small location, thus changing the sensed resistance and eventually leading to bad localization. Another problem with resistive touch screens is temperature which affects accuracy.

In the end resistive touch screens were abandoned but that technology is not completely useless, here's an example for a foot pressure point sensor:


Resistive sensors can also detect rotation angle, for example, potentiometers, and slide, as slide potentiometers.

Analog servo motors use potentiometers, but the same principle applies, you can use a multi turn potentiometer to achieve multiple turn servo!

Source
Linear Servo


Capacitive

Capacitive sensors work by detecting conductivity different than air which makes them useful for many things and they don't suffer from the same problems resistance sensors.

Common usage is touchpads, phone screens, touch buttons, capacitive proximity sensors and capacitive soil moisture sensor which doesn't corrode as easily as resistive ones.

Capacitive Touchpad
http://www.cirque.com/capacitive-touch/
Touch Sensor

Soil Moisture Sensor
Proximity Sensor


Current/Voltage

Current sensors/voltage sensors are very important to keep a limited system within the power supply's boundaries or keeping a Lithium battery alive when the power runs out, so the battery voltage won't get below 3.2v (or its limit) per cell.

DC Voltage sensing can use a very simple voltage divider. and AC Voltage can use the same principle except that we need to put a rectifier in front of it.
Source
Current sensing on AC works by induction, while DC current sensor works by measuring the voltage across a very low reistance component, such as a short wire, a 0.1/0.001 ohm resistor or dedicated sensors such as ACS758.

Non-invasive AC current sensor

100Amp AC/DC Current Sensor Module Board, based on ACS758

Biological

Biological sensors are useful for detecting or authenticating a person, so if your robot depends on someone's pulse or blood oxygen level, you can use a pulse oximeter for that, its a non invasive sensor that measures the difference between IR and red light absorbance of skin.

MAX30100 Pulse Oximeter


Another thing someone might want to sense is muscle/heart/brain signals, which can be done with EMG/ECG/EEG sensors.
Muscle Signal EMG Sensor
ECG module AD8232
8bit EEG brain wave module -8 Channel 

Fingerprints, lately it seems fingerprint sensors are all over the place, you can get them very cheaply and in various sizes. A very fun project I've seen someone do is build thor's hammer with fingerprint reader.

FPM10A Fingerprint Reader Sensor





Tags: