Cheap Home Air Quality Monitoring

I’ve wanted to monitor room temperature and humidity in my home for quite some time and I recently came across the ESP8266 Wi-Fi microcontrollers that were ideal for the job. Prior to the ESP8266 becoming mainstream, I was considering running a 1wire network over spare phone wires, similar to this weather station setup.

ds18b20_log

Thankfully, the ESP8266 chips are cheap enough that fitting one in each room of a house is practical. On top of that, people had already made inroads to getting code running on the ESP, including a webserver returning readings from a DHT22 temperature and humidity sensor.

The ESP8266 chip itself includes a Tensilica Diamond Standard core, although documentation is a little sparse. Fortunately, setting up a toolchain is relatively straight forward, and there is a (proprietary) SDK available that allows custom firmware to be built and flashed onto the modules.

Hardware

The hardware I threw together is very simple, and hooks up a DS18B20 1wire temperature sensor and a DHT11 temperature and humidity sensor up to the two GPIOs hooked up to pins on the module. Since I wanted a handful of the the boards, I opted to design the circuit in Fritzing and got a pack of boards from DirtyPCBs. The schematic and board are below:

Scheamatic

Temp Mon PCB

The schematic is straight-forward; 5V comes in from a USB socket and goes through a 250mA fuse and gets dropped to 3.3V by a TS1086CZ or AMS1117 module. The switch S1 allows the GPIO pin to be grounded to program the module, with the UART pins broken out to a separate header.

Software – ESP8266

The software is quite simple. Using sprite_tm’s webserver code and building on Martin’s additions for the DHT sensor, my code adds a DS18B20 driver and support for the cheaper DHT11 sensor.

In addition to serving the sensor values over HTTP, the code periodically sends a JSON packet over UDP to a logging host (a Raspberry Pi) which stores it in a MySQL database and an rrdtool database.

I noticed after a few days of running, the ESP boards can get quite warm. Although I don’t think it affected the sensors, it seemed sensible to avoid leaving the ESP8266 online if it only needed to periodically log data. Fortunately, there is a “deep sleep” function in the SDK, but unfortunately this requires the RESET pin to be soldered to the RTC pin to trigger wakeups. With the chips spending most of the time asleep, they don’t get warm at all.

Software – Logging

A python script listens on the UDP port for incoming packets, deserialises them, and logs them. The readings from the DHT sensor are susceptible to noise (possibly a bug in the driver), so the script includes logic to reject samples that are outside a window.

RRDTool expects updates for every value at once, so it’s not possible to use a single RRD to store the data from multiple hosts as it comes in at different times. Instead, each host needs its own RRD, so I use a script to create them as I add hosts to the network.

Software – Visualising

I initially used a Shiny app to load the data from the SQL database, but this became unworkable quite quickly; without a lot of spare RAM, MySQL resorts to creating temporary tables on the SD card which is tragically slow.

Shiny Dashboard

For quick at-a-glance readouts, I switched to RRDTool. Although it’s not as pretty as ggplot2, it’s lightweight enough to render graphs regularly and serve them as static images, which fits far more comfortably on the Raspberry pi. The script just iterates over each host and plots it for various intervals, saving the images to a webserver’s directory.

Bill of Materials & Cost

  • PCBs: $14/12 – £0.78ea
  • USB B Connector: £2.51/5 – £0.50ea
  • Fuse holder: £1.29/10 – £0.13ea
  • Fuse: ~£0.10ea
  • ESP8266: £8.36/4 – £2.09ea
  • DS18B20: £4.99/5 – £1.00ea
  • DHT11: £3.04/5 – £0.61ea
  • AMS1117 module: £2.62/5 – £0.52ea
  • Switch: £1.99/5 – £0.40ea
  • 4.7kR resistor: 2x~£0.02ea
  • Project box: £3.36/5 – £0.67ea

That works out at approximately £6.84 (amortising the cost of the PCBs), or £7.93 each for the 5 nodes I built. I’m ignoring the cost of power supplies as I have at least one USB power source and cable in the rooms I’m monitoring. Given there are more GPIOs on board to the ESP8266, there’s also scope for adding more sensors to them later on.

Advertisements

Visualising Bitcoin Prices with R

Vircurex is an online exchange for various cryptocurrencies. They have an accessible API which is easily queried with a small bit of R code.

Get Data

Pulling data from HTTP URLs is straightforward in R. HTTPS support is a slightly different story, but the RCurl package easily solves the problem, using the getURL function. Querying the various endpoints becomes fairly straightforward:

url <- paste0("https://api.vircurex.com/api/", request_type, ".json", parameters)
fromJSON(json_str=getURL(url))

Almost all API calls follow the same sequence; build a query string and send it. R provides some interesting reflection capabilities that make implementing this particularly easy. Firstly, match.call() gives the call for the function (name and arguments). In addition, named arguments can be extracted using formals(). Between those two functions, we can determine argument names and values dynamically. These will be the parameters used in the query.

query_trade <- function(base = "BTC", alt="USD")
    request(as.list(match.call())[-1], formals())

This defines a function, query_trade that calls out to the Vircurex API (via request). Since most of the API calls take the same arguments, we can reuse this function for most of the calls by abusing sys.calls() which gives access to the call stack.

caller_details <- sys.calls()[[sys.nframe()-1]]
request_type <- caller_details[[1]]

Included in the call stack is the name of the calling functions. Using the right entry in the stack allows the following definitions to “just work”:

get_last_trade <- query_trade
get_lowest_ask <- query_trade
get_highest_bid <- query_trade
get_highest_bid <- query_trade
get_volume <- query_trade
get_info_for_1_currency <- query_trade
orderbook <- query_trade
trades <- query_trade

Visualising MtGox’s impact on BTC prices

Of the various API calls supports, the trades function gives all the transactions completed in the last 7 days for a given currency pair. The fromJSON function returns a list of lists which need to be munged into a data frame. The easiest way to do this is to convert it to a matrix, then to a data frame.

trade_list <- trades()
trade.df <- data.frame(matrix(unlist(trade_list), nrow=length(trade_list), byrow=T))
#Convert columns from factors to numerics:
for(col in 1:ncol(trade.df))
    trade.df[,col] <- as.numeric(levels(trade.df[,col]))[trade.df[,col]]
#Give columns names:
names(trade.df) <- c("time","id","amount","price")
#Convert the time column:
trade.df$time <- as.POSIXct(as.numeric(df$time), origin="1970-01-01")

Now the data is in a data frame with the correct types, it’s straightforward to visualise it using ggplot2:

ggplot(trade.df, aes(time, price, size=amount)) + geom_point() + geom_smooth(aes(weight=amount)) + theme_minimal()

This plots each trade as a dot, scaled by the amount (in Bitcoin). The line is fitted using LOESS regression, weighted by the transaction amount.

Vircurex: BTC USD

There’s an interesting dip that corresponds to Feb 25, about the time that MtGox went offline.

Code for this entry is available in this GitHub Gist.