Wednesday, March 20, 2019

Things I Learned by Recording and Monitoring Our Indoor Air Metrics


I live in a state with frequent air pollution issues during the winter, and throughout my life have started to feel the effects of it in the development of seasonal asthma. My state is known for producing lots of asthma sufferers! Recently, I've been involved in doing some hobbyist-level air science, thanks in no small part to my friend Brad Midgley and his Geothunk project, which really helped me get started. I've been more closely monitoring our indoor air. It's really fun!

For the technical details, I've deployed several sensors in and outside of my house to monitor PM levels (particulate matter, IE smoke, dust, pollen, mold, etc), humidity, VOCs (volatile organic compounds), temperature, and air pressure. I record values every 30 seconds and ultimately log them to a database, InfluxDB. Then I visualize and expore the data using Grafana. All of this infrastructure runs in my basement on a Raspberry PI with USB RAID controller for storage.

My Kitchen Air Quality Sensor: ESP8266 microcontroller, BME680, PMS5003 Particle Sensor

I've learned a lot of things by watching our data and learning what kinds of things contribute to different pollutants. While none of my discoveries are new discoveries, it was fun to discover them, and, I'm not sure they are common knowledge.

A Note on Air Pollutants

I'm monitoring the following air pollutant metrics in my home:
  • PM (Particulate matter) levels. Primarily, PM1, PM2.5, and PM10.
  • VOCs (Volatile Organic Compounds)
PM2.5 particles are hazardous for our health because they are too small for your mainline defenses, since the cilia in lungs cannot deal with them [1]. This means our immune system has to deal with it. And how? Inflammatory reactions. Inflammation in our lungs over an extended period of time will lead to tissue damage. The kind of tissue damage, I think, that leads to statistically increased rates of lung and cardiovascular disease in more polluted areas.

VOCs are vapors in the air that can cause the room to feel stuffy. Some common sources are living creatures, decaying plant matter, cleaning products, fumes, etc. They accumulate the worst in poorly ventilated areas, and while some are actively harmful to your health, at a certain level of saturation it will be more difficult to breath, since the vapors will crowd out the available oxygen. People with impaired lung capacities (Asthma, COPD, etc.) will notice high-VOC concentrations earlier than others, since their lungs absorb oxygen less efficiently.

Lessons Learned

Humidifiers and Tap Water

We have an essential oil diffuser from döTERRA, which some household members use to put essential oils in the air. While these devices are purchased and used for purported health benefits, I found that these things, when used with tap water, are much worse polluters than I'd imagined!

Would you like a side of lung-injuring particles with that?

döTERRA diffuser pollutants released with tap water

In the above graph, you can see pretty clearly when the diffuser was turned on, at about 23:05. It was running in our master bedroom (~40 square meters total). In 30 minutes, that innocent looking device converted the air in our bedroom up to 25 PM2.5 μg/m³. I stopped the diffuser for this repeat-experiment; however, in the past I have seen levels of 50-75 PM2.5, a level well above the limit per the EPA 24-hour fine particles standard for outdoor air [4] (no indoor standards have been set by the EPA yet).

This effect is not limited to diffusers; any humidifier that use an ultrasonic mist maker will happily convert the mineral content in tap water into fine particulate matter.

If you own and use any ultrasonic mist humidifier / diffuser devices, use water with the mineral content removed, such as distilled water or water filtered by reverse osmosis. It's a potential misconception to think that manufacturers recommend distilled water just to help protect the humidifier equipment. Really, you should be more worried about protecting your lungs. If a humidifier says it's "filter free" because it kills bacteria, this says nothing about the pollutants it will put in the air. Don't put mineral-containing water in ultrasonic mist humidifiers or diffusers.

As an alternative, if you want convenience of using tap water, boiling humidifiers can be a better, more convenient choice since they leave minerals behind in the water (which is why the water will turn black after several runnings without cleaning). However, aside from the fact that this will heat your home (a fine side effect during the winter), another fun fact is that tap water also contains VOCs (volatile organic compounds) in it (as seen in a city water report, IE [2]). When you boil the water, these organic compounds are released into the air. VOCs, in high quantities, can crowd out the availability of oxygen and can make the room feel stuffy and make it more difficult to breath. This effect is made worse by the fact that humidifiers usually require the windows and doors to be closed for them to do their job. Personally, for a boiling humidifier, I find that targeting 35-45% humidity is a nice compromise between convenience, keeping my room from getting too dry, and keeping the VOC levels under control.

    Slow cooking meats electrically can release a lot of VOCs

    Slow cooking meats, indoors, in a crockpot, can create hazardous air situations as meat releases VOCs when it is cooked (all that delicious smell!). If you're not ventilating well (because, for example, it's cold outside), you might consider cooking the meat outside.

    A few days ago, we cooked corned beef in a crockpot, indoors, for several hours and it sent our indoor VOC levels to near-hazardous levels. It got the point that several household members were feeling light-headed and about to faint! Here's a graph of the data showing the incident (the crockpot running, the point at which breathing difficulties are experienced, our back door opening to ventilate the house, etc. can all be clearly seen in the data):

    (click to zoom)

    Since it was winter, the windows were closed. Because the crockpot was warming the house, the HVAC wasn't running. Both of these facts contributed towards creating a bad air situation, all from one electronic crockpot!

    Cooking indoors with oils are a common source of PM pollution, even without visible smoke

    Cooking with oils will release PM2.5 before smoke is visible. It's wise to pay attention to smoking points, and be conservative. Coconut oil, with all of its hype, can be particularly bad for the air when used for cooking eggs, tortillas, or other things in a skillet! We've started to prefer grape seed oil, instead, which is more stable at higher heats.

    When cooking things in a fry pan, I've found it is better to target lower temperatures when possible. Even when running the hood-range over my stove, if the oil starts to smoke even a little bit, then I either need to open some windows or deal with the fact that my home is going to be full of pollution for several hours.

    Burning candles indoors are a source of pollutants

    That nice smelling candle can be a big offender for releasing both VOCs and PMs [3]! What looks like a little bit of smoke can actually be really hazardous if not ventilated well. One time, I blew out several candles in a room that was poorly ventilated, and watched my PM2.5 levels climbed all the way to 70 μg / m³, a level that is considered hazardous for long-term exposure.


      It's very easy to make your indoor air worse than outdoor air.
        Indoor air is super important and I wonder how many people really think about it! If you are not mindful of it, then your indoor air can be worse than the air outside on a bad air quality day! When it comes to supporting good health, we should be as concerned about the air we breath as with the food we eat! I'm under the opinion that monitoring your indoor air quality should be a more common thing, especially in families with asthma sufferers. Having the raw values helps to answer the question, "is my indoor air quality currently bad?" Having a history of the values via graphs has really helped me to answer the questions, "why is my indoor air currently bad? And, what was the air quality like in the several hours leading up to this air-quality reaction?".


        1. Bureau of Epidemiology. "Air Pollution and Public Health in Utah - Particulate Matter (PM).", Accessed 2019-03-20.
        2. Saratoga Springs City (Utah). "Annual Drinking Water Quality Report.", Accessed 2019-03-20
        3. Jeong-Hyeon, AhnKi-Hyun, KimYong-Hyun, KimBo-WonKim. "Characterization of hazardous and odorous volatiles emitted from scented candles before lighting and when lit." ScienceDirect. Accessed 2019-03-20.
        4. The National Ambient Air Quality Standards for Particle Pollution. "REVISED AIR QUALITY STANDARDS FOR PARTICLE POLLUTION AND UPDATES TO THE AIR QUALITY INDEX (AQI).", Accessed 2019-04-05.

        Sunday, December 18, 2016

        Surprising slower CPU speeds with MacBook Pro 2016

        First off, the new Macbook Pro is impressively designed and I really like the direction they’re going. The Touchbar makes so much sense and it’s immediately obvious that it’s what the function keys always wanted to be.

        However, after comparing between the 2015 model and this latest one, the CPU performance seems surprisingly bad.

        Here are the specs for the two laptops I've tested:

        2015 MacBook:
        • 2.8 ghz i7 (quad core; burst 4.0 ghz; Haswell)
        • 16 GB 1600 MHz DDR3 RAM

        2016 MacBook:
        - 2.7 ghz i7 (quad core; burst 3.6 ghz; Skylake)
        - 16 GB  2133 MHz LPDDR3 RAM

        A good portion of my day includes compiling Scala code, a language which, due to it's extensive feature set, can take a little longer to compile. As such I'm inclined to care about how well the hardware I'm using performs this task. In a somewhat informal test, I compiled the Marathon code 4 times, from scratch, each time completely clearing the compile cache (and not re-downloading dependencies). On average, the compile times were as follows:

        • MacBook Pro 2015 : 2m18s
        • MacBook Pro 2016 : 2m33s

        This represents about a 12% reduction in performance from the 2015 to the 2016 model. The base clock speed is 4% slower, and the burst clock speed is 10% slower. The RAM in the 2016 model is significantly faster. So, to see this bit of a hit on performance was a surprise to me. In related news, clock speeds don't mean anything anymore.

        On the positive side, the touch ID, touch bar, brighter screen, smaller form, new speakers, etc. are fantastic. It’s a very well designed device and it's a joy to use. When I'm browsing the web or performing other non-CPU-intensive tasks, the newer 2016 MacBook Pro is up to the task and has no noticeable performance issues. But, when measuring CPU intensive tasks, alone, it's strange how poorly it performs compares to it’s predecessor. It makes me wonder if maybe there is a software issue, or the MacBook Pro I received is defective, or, perhaps, they made some compromises to how CPU overclocking is handled because the new form factor introduced additional heat distribution constraints.

        At any rate, if you regularly run CPU intensive tasks as a part of your day, then you may wish to pass on the new MacBook Pro, for now.

        --- UPDATE ---

        When I first wrote this post, I was consistently seeing a 27% decrease in speed. However, when I came back to run watch the clock speeds throughout the process, I wasn't able to reproduce those numbers. I've updated the post as such. I'm a little confused as to why it was performing so much worse earlier, but am also relieved.

        I ran the Intel Power Gadget to monitor clock-speeds throughout the compile process. It looks like the newer MacBook Pro 2106 runs cooler and use SIGNIFICANTLY less power, which is what you'd expect from the new Skylake processors. Also, it appears to be bursting much less aggressively than the 2015 model, which explains perhaps the larger performance gap.

        MacBook Pro 2015 CPU activity during compilation process. 

        MacBook Pro 2016 CPU activity during compilation process.

        I turned the MacBook Pro 2016 fans on high (using iStat menus) and it kept the CPU at just under 70°C the whole time during the compile test; no difference in execution time. I wonder why the new MacBook Pro doesn't push the processor harder when temperature is low and electrons are abundant?

        Sunday, July 19, 2015

        The Need for Acknowledgement in Streams

        While working on op-rabbit Akka Streams integration, I had the need to know when a piece of data had completed its flight through a stream, where completion could be defined as the following:

        • The source element was processed by the sink.
        • The element was eliminated by a filter, mapConcat -> 0, or collect.
        • In the case an element was split into multiple sub-elements via mapConcat, all resulting sub-elements were completed.
        • In the case many elements were grouped together via grouped, groupedWithin, etc., the resulting group completed.
        • In the case an element is broadcast over n channels, then similarly, all resulting copies of the element were handled, and their downstream results.

        Error signaling is important, too. If an element fails in any give stage of the stream, then the error should be propagated up through the acknowledgement channel.

        My original implementation of this idea was to simply create a stream whose contents were always a tuple of a Promise and the element. Then, it would be the responsibility of the programmer working with the stream to handle acknowledgements. However, this approach ran counter to the philosophy of "model valid states using the type system"; it was too easy to make a mistake and not fork a promise, collect promises together, or altogether not acknowledge the promise (because it was carelessly filtered from the stream). Also, mixing the concern of acknowledgement with the concern of processing the stream produced some tremendously difficult-to-read code, even with a special helper. An example:

        As you can see, it reads awful. And, worse, it's still incredibly error prone.

        I wrestled with this: is acknowledgment needed? Without acknowledgement, then there is no hope for retrying messages that failed because of simple chaos. If the process crashes, or the stream is not brought down gracefully, then the messages in-flight are lost. In most cases, this is probably acceptable. But, in others, it's not. I wanted to see if I could do better.

        At first, I decided it would be best to create a new class of stream components, based on Akka Streams, called AckedSource, AckedFlow, and AckedSink. The justification is as follows:

        • The requirement of automatic message acknowledgement necessarily implies a reduced set of out-of-the-box operations at your disposal. For example, it is inherently impossible, from the outside, to correlate messages emitted via scan and transform operations with the upstream elements that went into producing that output.
        • It would be too easy to accidentally hook an AckedStream to a normal Akka Streams Sink, or a normal Akka Streams Flow. This would result in leaky acknowledgement. Without a separate class of streams, the compiler would not know to yell at you for making this mistake.
        • Modeling message acknowledgement from head-to-tail provides similar watertight properties as does Future. A Scala Future, when constructed via a thunk, must complete, or must error. The only way to make a Future not complete is to complete it with another Future that doesn't complete (one that was created via a Promise), or actually have it run forever (in which case, never completing is the appropriate course of action). Because you cannot run an AckedSource or AckedFlow until it's connected into an AckedSink, and all stream operations are restricted to those that can guarantee watertight acknowledgement, it is impossible to have an element be processed and not acknowledged (or, "nacked" due to an error). Of course, this is operating on the assumption that AckedSource, AckedFlow and AckedSink do their job correctly.

        Here is the above code modified to use AckedStream

        All of the original functionality was retained, and the code is much simpler, easier to read. Naturally, the complexity is pushed elsewhere (to AckedStream), but that's just good separation of concerns.

        FanOut / FanIn Graphs

        AckedSource, AckedFlow, and AckedSink are a good start; but how do we handle the more complex 1-n or n-1 element routing scenarios provided by Akka Streams FlowGraph? There is a lot of rich functionality here, allowing composable disconnected stream fragments to be assembled into a closed graph.

        Again, I considered, "do we press this pattern onward? Or, do we opt for generic types? Are we pressing it too far and creating too much complexity?" I was divided: proceeding seemed risky, and I could end up with significant lost time and, afterward, a terrible mess of complexity worse the original problem. I convinced myself it was worth it for the following reasons:

        • The provided Akka Stream Broadcast component is incompatible with message acknowledgement; combining an AckedSource with this component would be an error.
        • Combining an AckedSource to an Akka Flow or Sink is also an error.
        • In an Akka FlowGraph, when you combine a Source with a Flow, you get a FlowGraph.Implicits
          . PortOps implements FlowOps and provides stream manipulation operations that are incompatible with an acknowledged Flow; using them is error-prone.

        Continuing to use the delegator pattern, I've created an implementation of AckedFlowGraph that supports a subset of the features. Here's some code using it:


        As of this writing, the AckedStream is contained within the op-rabbit akka-stream integration, and can be found here, with examples found in the tests. The code is general enough that I will likely end up extracting it into its own library, although I presently have no need for it outside of the context of using RabbitMQ.

        Some of this code may make some generalists frown. I consistently fought, along the way, a feeling inside that I was creating too many types. However, because of the aforementioned reasons, I feel the complexity is warranted because, fundamentally, acknowledged streams are different from non-acknowledged streams: their operations have different behavior, their supported operation lists don't overlap completely, and haphazardly intermixing acknowledged streams and non-acknowledged streams will cause issues.

        Wednesday, May 29, 2013

        Ready, Set, Fling ALL THE (BAD ARGUMENTS) !!!!

        Recently, there was an infographic going around on Facebook with the purpose of warning us of the evils of genetically modified organisms. I confess to have been rather amused by the subtle grim reaper in the background. Scientific publications should take a queue from this meme producer.

        Here's my take on how these arguments hold up:
        1. Seems to be an okay argument. If the soil is heavily contaminated with pesticides, only GMO seeds will survive the soil conditions, and farmers will be forced to switch to a new plot of land (or find a way to remove the soil contaminant). Glyphosate, an enzyme inhibitor, breaks down in the environment and has a half-life of 3 to 130 days; 75%-99.99% will be removed in just over a years time, 97%-99.99999999% (or so) in about two years.
        2. Okay argument, but see #1 re: half-life of glyphosate. (if this were entirely true, I don't think I would need to spray for weeds every year).
        3. True... _BUT_: Mono-cropping has been used in traditional agricultural for hundreds of years. That doesn't make it a good practice, it just doesn't make it exclusively true for GMO agriculture.
        4. Weak. As I understand it, Monsanto is not using this technology (currently? or are they?). However, if they did, terminator seed makes a genetic sequence inherently unfit. Nature will select against it. So the effects wouldn't last and it certainly won't wipe a species of crop off the planet (although, admittedly, this position is not specifically taken by Rawforbeaty's grim-reaper-backed info-graphic).
        5. Weak / misguided. Again this is an entirely different problem and is not unique to GMO crops.

        Will I be eating GMO crops? Probably not, I'd prefer the tried and true food source with which our bodies have evolved through the years. Would I eat it if the choice were between cheap GMO food and starvation? You bet. Does cheap food lead to over-population and other problems? It seems to be the case.

        Could bad things come from this? It's possible. But the same could be said for the Internet, the Television, the Radio, the Automobile, the Antibiotic, the Vaccine....

        Does that mean it's an intelligent thing to do, to sling any haphazard argument that can be produced against it?

        Ignorance is Strength?

        Growing impatient with ignorant absolute disdain for scientific medicine, so the following rant is my coping mechanism:

        Human's screw up everything through science?

        In a time, long long ago (long before Monsanto came around and destroyed all the planet), some human cell screwed up while copying DNA and changed a genetic sequence. The so-affected sequence contained the blueprint for producing a rather-necessary digestive enzyme; for short, we'll call it "very long-chain acyl-CoA dehydrogenase". Lucky for the offspring, he happened to receive two copies of this blue-print (redundancy is king!), so his body was still able to make the functioning enzyme. He lived a happy, normal life, and passed the faulty recessive trait on to his offspring. After a while, the recessive trait made its rounds until it started happening that both parents contained the recessive trait. For some reason, said parent's children would die shortly after birth with a 1/4 chance. Clueless as to what caused the death, 3/4 children would turn out fine and continue to propagate the gene.

        Fast-forward to the near present. Meet my nephew: born into this world to WONDERFUL parents that both happen to carry the recessive genetic trait. My nephew is awesome! I love this little guy. He got two copies of the recessive trait and has a genetic disease known as VLCAD.

        Thanks to advances in scientific medicine, medical screening promptly detected the disease and medical intervention saved his life. Thanks to understanding of the condition, he will live a healthy life! This is so awesome, because, like I said, my little nephew is a REALLY COOL KID. Thanks to advances in scientific medicine, we can all be screened for this recessive trait and know the chances of our children being born with a genetic disease. This is REALLY COOL!

        If you think nature is so good at everything, and that we should stop medling with nature, let me introduce you to an idea: NATURE KILLS BABIES [1]. Nature applies natural selection MERCILESSLY AND WITHOUT FEELING. Nature is the means through which future generations are blessed by the fact that the "unfit for survival" did not survive.

        Application of science is not perfect? Sometimes it goes awry and people die? Application of science does harm, from time to time? Profit motives cause people to make compromises that are less than optimal? Sure, it happens occasionally. Can it be better? Yes. Nature still kills babies, and medical science is getting in its way more than ever before.

        My plea is that we set down our weapons of sweeping generalizations, putting forth any argument not because they are sound but because they support a position we've adopted by studying the facts presented by one heavily biased view.

        Since somebody's believer is another's cynic, it seems like a good idea to tread with skepticism any time you see a perfectly constructed, dichotic exaltation/demonization of the sides of a given issue; at least until you have tried to argue the pros and cons of both sides.

        [1] Certain situations only. (that's called honesty!)

        Saturday, October 27, 2012

        Spaced Repetition Learning Systems: Achieving Flow in Rote Memorization

        My 5-year-old daughter and I have been working on her flash cards using the old physical decks for some time. Not having much of a system other than just cycling through the same old deck started to feel wasteful: I didn't have a good way to prioritize the cards with which she'd particularly struggled, so we spent the largest majority of the time going through words she knew really well. Even more, it seemed to discourage her from learning new words as hard words were shown too infrequently between a long stream of easies that the fluctuation between boredom and frustration stressed her out.

        For some time friends of mine have recommended to me the Anki flash card system; Anki is an implementation of the Spaced Repetition Learning System: a system of learning based on common-sense principles such as showing at greater frequency cards with which the learner struggles, and less frequently those that have been mastered.

        Using this system, it seems that my daughter gets a well-balanced mix of words she's already mastered, words she's learning, and brand new words, and in the end that seems to help keep her in the optimal state of flow while learning [1]! This is corroborated by her being excited to do the flash cards each evening.

        After using this system 15 minutes a day, in just ONE WEEK my daughter has mastered all 100 of the sight words her teacher had assigned! I am so proud of her!

        If you'd like to learn more about the Anki system:

        It runs on all the OS's and devices, including iOS, Mac OS, Android, and the web. (the developer seems to fund his efforts through iOS sales, as that is the only platform for which he charges, and notably, a charge I was happy to pay.)

        • [1] - Flow is the"... mental state of operation in which a person performing an activity is fully immersed in a feeling of energized focus, full involvement, and enjoyment in the process of the activity" [Wikipedia]. One of the requirements to reach "flow" is an optimal balance between challenge and ability.

        Saturday, May 26, 2012

        Danger: Dragons

        Promote safety with this effective warning tool:

        Dragon silhouette lifted from Kary (who in turn lifted it from Here).