Sunday, November 27, 2011

Shattered

My lovely wife and I live out of town on an acreage.  Our nearest neighbor is a half mile away.  We have no yard light and don't want one, either.  At night, it gets dark outside.  Very dark.  It looks a lot like this:
Seriously
We also have a dog.  A large dog.  A large dog that refuses to take advantage of modern conveniences like indoor plumbing.  If she has to "go", she has to "go outside".  "Going outside" is her favorite thing in the world, as it is with most dogs.  But our dog is also rarely more than a few feet away from either my lovely wife or myself.  If we try to let her outside to do her dirty, filthy business, she'll just stand on the step and wait for us to come out.  She would probably blow out a kidney waiting for an escort to take her into the tall grass where relief awaits before heading out there to do her business on her own.

Now going outside sometimes means going outside at night.  At night, it gets dark outside (we've been over this - remember?).  And at night, there are various critters that like to come out as well.  We regularly hear the coyote packs howling under the moonlight.  And then there are the porcupines.  Our dog has tangled with these and other critters.  Usually she comes out on top because of her size.  Sometimes, not.  A dog can't win against a porcupine, but they don't seem to be able to figure this out.  These battles can be painful.  And expensive.
Our Dog: Post Surgery after Porcupine Battle #2
So because our dog needs to go out at night, and because it gets dark at night, and because we have to go out with her, and because there are dangerous critters outside at night when it is dark, and because we can't see in the dark, we need a flashlight.  A bright flashlight.  A flashlight that can light up the night for a great distance.  A flashlight that can find the critters before our dog does and hopefully scare them away.  We started off with a four D-cell MagLite flashlight.
Great For Smashing Some Thing's Head In, But Not Great Otherwise
This thing threw a pretty decent amount of light.  We also got a good workout lugging it around.  And it would also go through batteries like Wynonna Judd goes through donuts. Not recommended.
I Am A Mean Person...
Then I discovered the wonders of LED flashlights.  I coincidentally also discovered the wonders of Deal Extreme.  They have Deals there!  And they are Extreme!  After much research a year ago, I picked up this bad boy: an Eastward YJ-XGR2 Waterproof Cree R2-WC 2-Mode 250-Lumen LED flashlight.  It is amongst my most cherished possessions.
Is It Wrong To Love An Inanimate Object?
The first night I tried it out, my lovely wife and I were outside.  I pointed this baby at her eyes and she had to look away.  "Big deal", you say.  "That would happen with any flashlight."  Sure.  But I was 200 yards away at the time.  This thing is light, far brighter than the Mag, and one charge of its single rechargeable batteries last up to 10 hours.  I love this thing.

That is why I was devastated to discover this after the pup and I went for our morning walk-about.
The Lens Was As Shattered As I Was
We don't know how it happened.  But it happened.  This flashlight was in critical condition.  What to do?

There was no way we could go back to the Mag: I am a man of modest means and cannot afford the batteries it chews through.  I had to develop a recovery plan, and fast.  Fortunately, all the pieces of the lens were still in place and nothing of any size had fallen out yet.  If I could only keep the lens together for the short term, I could then work out a longer term plan.  So I dug out a roll of clear packing tape.
You Can See Where I'm Going With This...
 I used a razor to trace around the lens and lightly pressed the tape around the glass.  This held things together enough so I could open up the flashlight and tape the back side together using the same method.  This would hold things together in the short term.

Next, I remembered from my research that Deal Extreme also sold flashlight parts.  I search for flashlight lenses and, lo and behold, they sell replacements.  Yay!  But the replacements come in 20 different sizes.  Time to dig out my calipers make a measurement.  Fortunately I had a banana handy to support one end of the caliper so I could free my hands to take a picture.
Better Eat That Banana - It's Getting A Little Ripe
The measurements vary from 41.6mm to 41.8mm depending on where I measured (smashing a piece of glass tends to affect its symmetry somewhat).  Not shown is my 1.5mm measurement of the lens' thickness.

Back to Deal Extreme.  And wonder of wonders: they sell a 41.5mm Diameter, 1.5mm Thick Replacement Glass Lens.  In ten-packs.  For $2.78.  For all ten.  And free shipping.  And no minimum order.  The well-intentioned recommendation of my cousin to consider somewhere else for a "decent light" falls on deaf ears.
This Makes Me Very Happy
So now that this post is written up, I'm off to put in an order to Deal Extreme.  Unfortunately, the "Add To Cart" button on that site is made of crack cocaine: you can't stop at just one hit.  I have a funny feeling I'll be spending more than $2.78.

Bad News and Good News

First the bad news. In my last post detailing how to build your own Davis console datalogger, I speculated that using a bigger memory chip than what Davis used in their original design might get picked up by the console and allow more archive records to be stored.  Unfortunately that is not the case.  The Davis 1 Mbit version tops out at 2560 records.  So does my 4 Mbit version.  If you are going to build your own (more on this in a second), save the 43 cents and stick with the 1 Mbit chip.
The Bane of the "Off by one" Programming Error
Now the good news.  When I got my DIY logger working a couple weeks back, I made a wxforum post detailing what I had found.  I made a similar post when I figured out how to build your own computer interface to the console, and comments poured in quickly.  This time around, it was a little different...
Kinda Quiet Around Here...

But little did I realize that despite the initial, deafening silence, great things were at work: wxforum member belfryboy put his head down and got busy.  After an iteration or two, behold!
A Schematic!
Yes indeed.  He did what I should have done and put together a schematic in Eagle.  This has a connection for both the Atmel flash chip for datalogging and a six pin connector for hooking up something like the Sparkfun FTDI cable.  These two pieces give you the complete equivalent of the Davis USB logger (except that this version will maintain a reliable connection).  Now, behold again!
A PCB Layout!
This is the corresponding board layout for the schematic above.  The ground pour isn't shown for clarity, and so you can see the shout-out that belfryboy gave me on the bottom layer.  This board is single sided and therefore DIY friendly.  Speaking of DIY friendly, the design can be easily pulled into the freeware version of CADSoft Eagle so that anybody can have a look and play around with this.  All you need to do is grab the design files from here and you are good to go.  And if you don't have the means to build your own circuit boards, belfryboy will build one up for you.

If you do build this, remember from my last post that you need to use a terminal program to actually activate the logging function in the console since Cumulus doesn't do this automatically.  I think I'll ping the author to see if he'd be kind enough to add this function.  belfryboy has a setup utility in the works and I'll add it to the design files once I get it.

So what the heck are you waiting for?  Get out there and build something.  Anything.  Figure something out.  Learn something new.  The reward is the journey.  Remember: You Can't, You Won't, And You Don't Stop.
 
My Earworm Of The Day: Now It Is Yours...

Saturday, November 12, 2011

Build Your Own Davis Console Datalogger!!!

It all started off with my seminal post, Davis Weatherlink Software Not Required, to which I added not one, not two, but three exclamation points to get across the enormity of the discovery.  With a $15 USB to Serial Adapter, you no longer had to pay the exorbitant sum charged by Davis to get a computer connection to your Davis weather station console.  Enthusiastic enthusiasts then charged ahead with several wireless solutions.  How about a wireless XBee link as written up by af4ex on this wxforum post?   Or what about gwynethh's Wi-Fi interface via the almost perpetually out of stock Roving Networks RN-XV (only $35!) as written up on this wxforum post?  If you don't think a weather station with its own TCP/IP stack is cool, then you have no soul.  Or you don't care.  One or the other.

But there were still murmurs of discontent in wx-land, and they centered on two bits of criticism.  The first, of course, was this was no plug and play solution.  Wires would need to be cut and soldering irons plugged in to get anywhere.  That was until SLOweather pitched in and started selling ready to go serial adapters based on the console logger pinout.  It is nice to know that this discovery enabled a cottage industry to spring up, though it must be said that it is an extremely small cottage.

The second criticism was that my hack did not have a data logging feature.  Indeed, this did suck.  I knew the pinout at the back of the console, but this wasn't enough for me to figure out how the logging function worked.
The Pinout That Started It All

It wasn't until I tripped over this post by wxforum member wxtech that things got really interesting.  This is a guy who obviously isn't afraid of letting the magic smoke out of his electronic devices.  He actually unpotted the Davis dongle to see what made it tick.  He wrote "U1 is an Atmel 45DB memory", and it was already well known that the logger had one megabit of memory.  That was all we really needed to figure out the datasheet for the Atmel 45DB011 Dataflash chip.

Technically, it all became clear with the datasheet in hand.  The console talks to the flash chip via I2C, and the remote PC talks to the console.  The remote PC is not directly connected to the flash chip.  That's why my homebrew serial interface works: the datalogger and interface functions are completely separate.  And that's why Davis brought out the I2C lines to the rear connector.

It gets better.  The 1Mbit version of the 45DB011 will set you back less than a dollar.  So you know what that means...
It Means A Cheap Excuse To Embed A Song I Really Like

After a summer spent preparing to feed the rabbits this winter, I got off my butt and ordered up some parts to see if I could make a DIY version of the Datalogger.  I ended up putting my order in to Mouser Electronics,  rather than the usual suspect.  The reason was quite simple really.  They were the only outfit I could find that both stocked 1mm pitch ribbon cable by the foot rather than the roll and also had an SOIC breakout board.  The flash chips can be found just about anywhere.  Here is what I ordered up.
  • A couple Atmel 4Mbit Flash Chips and a couple 1MBit Flash Chips to store my data.  I wanted to try an experiment here to see if the console could actually read chips of the larger capacity.  I bought a couple of each just in case I wrecked one. 
  • A 20 pin, dual row, 2mm pitch IDC header to plug into the back of the console.  I bought four of these to stock up.
  • A 16 pin SOIC to DIP adapter so I could prototype this on a breadboard.  Why 16 pins for 8 pin parts?  My plan was to solder on one chip of each capacity, and try the bigger one first to see if it would work.  This way I wouldn't have to mess with desoldering one to try the other.
  • Some 1mm pitch ribbon cable to connect it all together.  I bought three feet, so I could make a couple cables if need be.  Note that the pitch of the ribbon cable is twice as fine as the header, because it has to pack twice as many conductors in to the same width.
You'll need solder and a fine-tipped soldering iron, of course.  While I intend to buy myself a Hakko FX-888 at some point this winter, I continue to get by against all odds with a pencil-type soldering iron I bought at Radio Shack sometime in the 80's.  You'll also want some kind of solder flux.  I have a flux pen (looks kind of like a highlighter) and it works great.

This first shot at things was going to take a prototype approach in case I had any wiring incorrect.  A 1Mbit and a 4 Mbit flash chip were soldered to the SOIC to DIP adapter.  I followed the Curious Inventor's most excellent instructions for soldering these fine pitch components like I did when I built my GoodFET.

And here is how it came out.  The stuff that looks like water droplets on there is just some flux that didn't burn off.  This is way bigger than actual size, BTW.  4 Mbit on the left, 1 Mbit on the right.
Not Bad, Given My Crappy Eyesight
I would figure out only later that I screwed up the chip on the right: the 1MBit part.  See how this is a 16 DIP breakout and yet there are 24 connections to solder to?  The rightmost pads don't actually connect to anything.  They are apparently there just to mess with people's heads.

Undaunted, I next built up my cable to go from the console to the breadboard.  It is easy to build up your own insulation-displacement flat ribbon cables as you can see here.  All you need is a vise or some kind of flat clamp to apply even pressure across the connector as the insulation is displaced in each contact (hence the name, Insulation Displacement Connector).  It actually takes a decent amount of force to press these on, so don't think you can do it with your fingers or mash it on with a screwdriver.  That will just ruin the connector and make you angry.

Of course, Davis would throw a wrench into things.  The connection on the back of the console doesn't leave room for the top clip on the IDC connector or the key that makes sure you get the orientation right.  I had to pull the top clip off and use an Exacto knife to shave the extraneous plastic bits off.  That got the console side of the cable done.  On the other side, I tinned the wires and soldered them to some 0.1" breakout headers stuck through a piece of perfboard.  This gave me something that was fairly robust with lots of flexibility for future prototyping.
Turned Out Pretty Well, I Must Say

Next I had to wire things up.  Based on the datasheet and the expansion port pinout, I came up with this pinout to connect the console to the flash memory chip.
Console Pin   45DB0x1 Pin
1 - SS        4 - CS*
2 - SCLK      2 - SCK
3 - MOSI      1 - SI
4 - MISO      8 - SO
7 - RST       3 - RESET*
14 - VCC      6 - VCC
14 - VCC      5 - WP*
16 - GND      7 - GND
And now a word of caution:
Before you get all excited and start wiring something yourself, be aware of this: in the pinout diagram above, Pin 1 is shown on the bottom right.  If you use an IDC connector like I did, Pin 1 will be on the top right.  So the blue edge of the cable that normally represents Pin 1 will actually be Pin 2, the second wire over will be Pin 1, etc.  Now don't go blaming me for the confusion.  Way back when I was determining the pinout, I picked Pin 1 as I did because that one has the square pad on the circuit board where the rest are round (proof here).  So I called that pin to be Pin 1 because Davis did.  Now we are stuck with that.  Just remember that the console pin numbers shown above correspond to the diagram at the top of this post and you should be fine.
I wired the cable connecting to the breadboard to the 4Mbit part for my first attempt.  I then powered off the console and plugged in the cable.  Note that the console only looks for the logger on powerup.  If you plug it in after that, it won't be recognized.  Note also that Davis recommends that you enter setup mode before popping the batteries on the console (press and hold Done, then press the '-' down arrow button).

Here is what everything looked like plugged into my breadboard.
L-R: USB to Serial Converter, Cable to Console, SOIC to DIP Breakout with Two Flash Chips

So once I hooked everything up, I powered up the console and none of the magic smoke escaped from the station or my circuit.  A good sign.  I then fired up Cumulus, told it to use the datalogger, and set the archive interval.  I then shut Cumulus down and left it for 10 minutes, hoping to see archived data when I fired it back up.

Nada.

I did't get it.  When I fired up Cumulus, I saw the LED activity lights on my USB to Serial interface flashing like crazy for some length of time, but there was no data?  It didn't make sense.  I thought it might have been because I started with the 4Mbit part, but it did seem to recognize the chip or the converter LED's wouldn't have been flashing like they were.  I tried a bunch of things until I came across this post on the Sandaysoft forums.  Just because you specify to use the datalogger on the GUI and are able to set the archive interval, that doesn't mean that it actually configures the console to use the datalogger or sets the archive interval.

Argh.

That post said you "have to" use Davis' Weatherlink software to actually activate archiving and set the interval, but that isn't exactly true.  You just need the Davis serial protocol document I linked back in the original post.

Here is the magic incantations I entered. 
"SETPER 1": Sets archive period to 1 minute. 1, 5, 10, 15, 30, 60, and 120 are OK too
"START": Enables the creation of archive records.
Remember you need these commands to be entered using a terminal program set to 19,200 8N1.  Set half duplex to see what you are typing and make sure all commands ARE ALL IN CAPS.  Like this:
Activating The Logger... Hopefully

So I gave that a shot and fired up Cumulus again.  This time it spent less time looking for the data but actually showed a progress bar during the download - something it didn't do before.  And then I got this.
Six Records Is More Than Zero
Success!  Now ask Cumulus to show me some graphs:
First Third Party Archived Data Ever
w00t!  We are Free At Last!  And all it took was a few bucks in parts.  You can actually build something up even  cheaper than the route I took here.  You could use this SOIC breakout for less than two bucks, and you might even be able to take an Exacto knife to a 2.5" laptop IDE cable you have lying around that has the correct 2mm pin spacing.  Either way, you are in business for around $5 - $10.  That plus the $15 Sparkfun converter, and you've got a bulletproof console interface with data logging capability.  But of course it goes without saying that if you try to build one of these things and break your console in the process, then you get to keep both pieces and I will disavow all knowledge of your mission.

By the way, this is the capacity of the standard 1MBit data logger.
1 Minute Archive Interval . . . . . . . . .  . . . . . . . . . . . 42 hours
5 Minute Archive Interval . . . . . . . . . . . . . . . . . . . . . 8 days
10 Minute Archive Interval . . . . . . . . . . . . . . . . . . . . 17 days
15 Minute Archive Interval . . . . . . . . . . . . . . . . . . . . 26 days
30 Minute Archive Interval . . . . . . . . . . . . . . . . . . . . 53 days
60 Minute Archive Interval . . . . . . . . . . . . . . . . . . . . 106 days
120 Minute Archive Interval . . . . . . . . . . . . . . . . . . . 213 days
Will I get more capacity because I'm using a 4Mbit part?  I guess I'll find out in a few days.  I'll let you know in a future post.

So like the post that started all this off, the title of this blog also has not one, not two, but three exclamation points.  I thought it rated that.  Good luck to anyone else that gives this a shot.  I'll keep an eye on the comments for anybody who needs a hand.