-->

How to Add an SD Card Data Logger to an Arduino Project

How to Add an SD Card Data Logger to an Arduino Project

2:57 PM
Arduino projects can vary widely from home brew managers to weather stations to advanced home automation. Often you'll want to log data from these projects. In this tutorial I’ll show you how to integrate an SD Card and a real time clock module into a project to allow time stamped data logging.

Gather the Components, Tools, and Libraries

Components

Tools

  • Soldering Iron
  • Solder
  • Third hand tool
  • Computer with Arduino IDE for programming
  • USB cable appropriate for your Arduino—Mini B for the Nano

Libraries

Download the following Arduino libraries from their repositories. For each library, unzip the folder into the Arduino/libraries folder. Rename each of the folders to beSD, RTCLib, and DHT respectively. The Arduino IDE can’t handle the dashes in the file names.

A Word About Choosing a Data Logger

Data loggers are available as discrete elements and can vary in cost from $35 to hundreds. Choosing the right logger is an important decision in the project. It would be an enormous disappointment to retrieve a deep sea observer project to discover that the data over the last year has been corrupt. 
As with all projects you need to consider the power consumption but there are several other features that you want to be aware of when selecting a data logger for a particular project.

Environmental Conditions

Perhaps the project is going to be in harsh conditions. Physical jolts, high temperatures, humidity, and other conditions will affect the choice. There are loggers available that can withstand different conditions. Be sure to know where your project and logger will be living and select the right technology.

Interface Type

There are serial loggers that speak just about any protocol, I2C, RS232, CAN, SPI and more. The interface protocol will affect how fast the log data can be written. Ensure you select the logger that fits with the project technology and write speed requirements.  

Log Retrieval

It’s no good if the logged data can not be retrieved. It’s the proverbial tree falling without anyone around to hear it. Make sure you understand how data is to be retrieved from the logger you are selecting. In some cases of discrete logger IC chips, you will have to build an external interface that you can use to pull the data back from the chip.

Time

Some loggers just dump data while others have incredibly accurate time stamps for the data records. As you would expect, the more accurate the time stamps, generally the higher the price. If your project requires very high speed logging that timestamps to the nanosecond, the logger will run a bit more than a logger that has no clue what time of day it is.  

Persistence

This is what you want to happen when you overrun the end of the storage. Schemes vary from stop logging to start overwriting the oldest data.

Media Type / Durability

Ensure you evaluate how many times can the device memory can be overwritten and what the plan is when it starts breaking down. If you have to replace the whole logger after 10,000 overwrites the value will depend on how many writes you plan on making per second.

SD Card Logger

The upsides to using an SD Card for logging are many. It is relatively inexpensive. You can choose your own logging format and whether or not you want time stamps. The media is easily replaceable when worn out and can be removed quickly and read by any standard computer.  
SD Card logging has some downsides too. 
It is not the best rugged environment where moisture could affect the contacts. It also consumes a lot of precious pins on a micro-controller. Between the real time clock (RTC) and the SD breakout board it consumes six pins of various types.  
I chose the SD Card logger for this tutorial because of the ease and flexibility of working with the SD card for writing and reading the logs. More importantly, the skills of using the SD card and a RTC will come in handy for a variety of other projects.

Build the Project

Follow the steps below to assemble the breadboard. This tutorial uses a simple temperature and humidity sensor to gather data for logging. After following the steps you should have a breadboard that looks similar to the diagram below.
Completed SD Card Datalogger
Completed SD Card Datalogger

Assemble the SD Breakout Board

The SD Card breakout board comes with through hole, breadboard friendly headers, but they are not attached. A quick session of soldering will fix that and allow you to seat the board as shown in the diagram.

Format and Insert the SD Card

Put the SD Card into a computer and verify that it is formatted with a FAT32 filesystem.  You may need a MicroSD to standard SD Card adapter. The SD Card in the components list comes with an adapter. Insert the SD Card carefully into the breakout board.

Assemble the Real Time Clock

The Real Time Clock, or RTC, board comes as a kit and will require a bit more soldering to put together. All of the components are included in the kit. Follow thestep by step instructions to assemble the board including the through hole headers. 
Ensure that you insert the included battery into the clip after assembly. It will not function correctly without the battery in place. The RTC only draws from the battery when there is no main voltage, so the battery should last years.

Attaching the 9V Battery Clip

Many battery cases like the one in the parts list above come with wire leads that are not breadboard friendly. To make the leads breadboard friendly solder a wire or header pin to the end of each wire lead.

A Word About SD Card Readers

SD Card readers can be purchased much cheaper than the one recommended in this tutorial. The catch, however, is that SD Card readers are a 3-volt technology while the Arduino uses a 5-volt micro-controller. 
This means that in order to safely use an SD Card reader with an Arduino a logic level converter is needed to convert voltages between the two. This adds cost and complexity. The reader specified in the parts list might be a little more expensive, but it has several things going for it.
  • It has a built in logic level converter so no additional components or wiring is needed.  It does not use resistors to reduce voltage which can slow down the reads and writes to the card. Hence, it can be wired directly up to the Arduino safely
  • It uses Micro SD Cards for a very compact footprint
  • There is a lot of available code and examples for it
The reader uses the SPI protocol to communicate. Arduino boards come with certain pins that are designed for this protocol. While it is possible to use alternate pins in a pinch, you will get the fastest and most reliable performance by using the hardware dedicated pins.
The pins indicated in the schematic picture are accurate for an Arduino Uno or Arduino Nano.

Program the Arduino

With the project hardware assembled, the next step is to program it. Download the zip packaged with this tutorial or from the github repository for this data logging project.
  • Open datalogger.ino in the Arduino IDE
  • Connect the Arduino Nano to the computer with the USB-to-USB mini cable
  • Ensure the serial port is set to the port the computer detected when you connected the Arduino using the Tools > Serial Port menu
  • Ensure the board is set to Arduino Nano w/ ATmega328 using the Tools > Board menu
  • Verify that the libraries are installed properly by clicking the Verify button and ensure there are no errors
  • Connect the battery to the project. This seems an odd step here, but the Arduino, RTC, sensor, and SD Card reader all pull enough voltage that I found my USB port did not supply consistent enough power and the project continually reset. Make the time to connect up the 9V battery. When I did this I noticed the status indicator led on the Arduino brightened noticeably
  • Use the Upload button to push the code to the Arduino
  • Edit the code to comment out the line: rtc.adjust(DateTime(F(__DATE__), F(__TIME__))); By putting a // at the start of the line. The first time you uploaded the code, it set the date and time of the RTC to the time of the compile. But you don’t want to keep resetting the clock to compile time each time the board resets. Uploading the code without this rtc.adjust in it will leave the clock time alone on the next boot
  • Upload the code again with the time set line commented out
  • Open the Serial Monitor window by pressing Control-Shift-M and ensure it is set to 19200 baud. You should see the Arduino initialize with output like the example below

Notes About the Program

The datalogger program blends together functions from the RTC library, DHT library, and SD library. There are a few options in the top of the program that you can tune such as the name of the data file and what type of DHT you have. 
All defaults, however, should work just fine if you used the components suggested above.

Main Loop

The main loop of the program simple does three things.
  • Read the temperature and humidity from the sensor. I borrowed some code for this from another tutorial I wrote: Building a Wireless Sensor Network in Your Home 
  • Format the retrieved data into a comma separated list to send to the log function
  • Send the data to the logThis() function for writing to the card
  • Sleep for three seconds
It repeats the process as long as there is power and space on the SD Card.

Logging

The logThis() function takes a character string of data as an argument, gets the time from the RTC, formats it with the data and writes it to the log file. The log file name is specified in the #define section of the program. The sketch opens the log file when the setup starts so that file logger does not have to re-open it for each log entry. The cryptic sprintf command shown below formats the data. 
If you want to change how the data is logged make sure you are familiar with thesprintf function and you can move the formatting and data elements around. If you just need to reorder the date fields, you can just change their order between the commas and every thing should work just fine.
Finally, the dataFile.println(message) command writes the formatted string to the file.  The function behaves just like the Serial.println command.
The FAT32 filesystem that is on the SD Card has some limitations worth knowing. The file name must also be in an 8.3 format. The largest file size it can hold is a 2GB file and it can only hold 268,173,300 files. Hopefully you can live with these constraints for a bit.

Time

The functions now.year() and friends perform the work required to read the current time elements (unix epoch, year, month, day, hour, minute, second) from the RTC board and return the value.
The cool thing about real time clocks is that they keep time of course. While the code and project can go off doing other important things like making sure your beer is fermenting at the right temperature, it just goes on keeping time. Even if your project loses power, it keeps on ticking with an independent battery. 
All you need to do is ask it and it will tell you what time it is. This is of course perfect for logging data. The component specified in the parts list is a passable timekeeper, but the specs say it can gain up to two seconds a day. 
If that keeps you up a night, then check out the fancy ChronoDot for higher precision with the exact same protocol. It’s completely interchangeable with the RTC in this project for a few dollars more and it comes fully assembled.

Testing 

You’ve built the project, programmed it (twice), and read all about the theory of operation above. It’s time to try it out.
  1. Set the board up somewhere the temperature and humidity will be interesting and plug in the battery
  2. Let the logger run for a while collecting data
  3. After a bit remove power from the project and remove the SD Card
  4. Put the card into the full sized adapter and insert the SD Card into your computer.
  5. Open the file datalog.txt
You should see something like the following.

Conclusion

In this tutorial I have shown you how to integrate an SD Card and a real time clock module into your project. With the example code you can log data for an extended period of time that can be quickly and easily copied to a computer. 
After completing this project you might try some modifications or upgrades to it. Try upgrading the time chip for more accurate time stamps. The included program simply writes to the same file over and over until it is full. You could try creating a date based file rotation scheme to rotate files once a week. 
I didn’t even mention the card detect wire from the reader that will allow you to modify the schematic and board to test for the presence of a card before trying any read or write operations. Good luck and have fun experimenting.

نبذة عن

أدهم أبهابي مدون محترف مهووس بالتطوير و تغيير ألوان قوالب بلوجر ، أحب ألعاب الفيديو و كرة القدم، ناجحٌ في دراستي إتكلم الفرنسية و الإنجليسية و مالك مدونة نجوم ويب.

Subscribe to: Post Comments ( Atom )