View previous topic :: View next topic |
Author |
Message |
libor
Joined: 14 Dec 2004 Posts: 288 Location: Hungary
|
|
Posted: Tue Jun 13, 2006 3:09 pm |
|
|
You can also compress your GPS log quite effectively if you store only the coordinate's delta from the previous position.
e.g: you store the whole coordinates only at every 100th, or whatever time, at the remaining waypoints you only store the difference in the coordinates from the previous waypoint (this difference can be expressed in much less number of bytes) |
|
|
ckielstra
Joined: 18 Mar 2004 Posts: 3680 Location: The Netherlands
|
|
Posted: Tue Jun 13, 2006 4:26 pm |
|
|
Conclusion:
Storing the whole NMEA as an ASCII data string is waste of memory space. Several methods have been shown for compressing the data, each having their strong and weak points. Data sizes range from 4 to 8 bytes per location, excluding overhead for timestamps, etc.
About the memory type:
For storing the data FRAM is one of my favorites for it can be cleared/written byte wise.
Largest type available: 256kbit = 32kbyte, costs about $6.50
At 4 bytes per location = 8k locations
The original poster wanted to save samples every 2 seconds, resulting in:
8k * 2sec = 16k seconds = 4.4 hours
A technique I've seen in some GPS loggers is to decrease the resolution over time. So at first data is logged every few seconds but later when memory starts to fill up the time between samples is doubled and half of the high resolution samples will be deleted from memory.
Or forget about all the troubles of the small memories and use a flash memory card. The smallest MMC card you can buy now is 32Mbyte for the same $6.50 as the largest FRAM chip. Even when storing the data in the large data format of 2 x int32 (8 bytes) you can save 4 million waypoints, enough for 92 days of data. Or spend $8 for an 128MB card and get 370 days of storage, a whole year of highest resolution data!
Sounds like this would be a fun project to do, and way cheaper than a comparable consumer product. |
|
|
Bart
Joined: 12 Jul 2005 Posts: 49
|
mmc |
Posted: Wed Jun 14, 2006 4:11 pm |
|
|
I agree with the mmc solution.
Writing in RAW on it isn't the problem.
But, as most recent pc's have an MMC reader, there is still no free ccs code available to WRITE on it in FAT12, FAT16 or FAT32 _________________ I like Skype (www.skype.com), my username is BplotM |
|
|
stevenm86
Joined: 11 Jun 2006 Posts: 11
|
|
Posted: Thu Jun 15, 2006 12:08 pm |
|
|
You don't need FAT or FAT32 implementation. Way too much work. Just write the data to the device raw, seqentially, as if it is one giant block of eeprom. Then you can retrieve the data one of several ways. I am sure there are windows programs that will directly show you the raw contents of a block device, in a hex editor form or binary form or whatever. In Linux you can just read the device with hexedit or just dd the contents of the block device to a file on your real FS.
If those options dont sound appealing, consider writing a stub PIC program that reads the contents of the card and spits out the ascii form to rs232, then transfer it to PC that way. |
|
|
ckielstra
Joined: 18 Mar 2004 Posts: 3680 Location: The Netherlands
|
|
Posted: Thu Jun 15, 2006 4:53 pm |
|
|
Quote: | there is still no free ccs code available to WRITE on it in FAT12, FAT16 or FAT32 | True, there is no CCS FAT implementation, but the code section of this forum contains a free FAT16 implementation. http://www.ccsinfo.com/forum/viewtopic.php?t=21721
I agree with Stevenm86, you don't need FAT. Keep your PIC as simple as possible and put all intelligence in the PC. It is far more easier to write and update programs on your PC. Also a FAT system requires frequent updates to the MMC contents. Flash memory is specified for something like 100,000 rewrites per sector but using FAT you will reach this much faster than you think. |
|
|
libor
Joined: 14 Dec 2004 Posts: 288 Location: Hungary
|
|
Posted: Thu Jun 15, 2006 5:22 pm |
|
|
You can write data that can be still read with a PC into the MMC without tha hassle of fully implementing a fully functional FAT system in the PIC:
make a dummy, (i.e. static) directory and FAT structure, and do not alter the FAT table itself, just write into the memory area where the file contents are.
E.g.: You can format a blank MMC in a PC, write a single continuous file on it (of the full size of the media), note the location where the file contents are stored, and use only this (continouos) area in your PIC program. You lose only a small area with tha FAT table, you don't have to implement any FAT system software, but you still have your data in a PC readable format on your MMC. |
|
|
Kenny
Joined: 07 Sep 2003 Posts: 173 Location: Australia
|
|
Posted: Thu Jun 15, 2006 7:31 pm |
|
|
libor wrote: | E.g.: You can format a blank MMC in a PC, write a single continuous file on it (of the full size of the media), note the location where the file contents are stored, and use only this (continouos) area in your PIC program. You lose only a small area with tha FAT table, you don't have to implement any FAT system software, but you still have your data in a PC readable format on your MMC. |
That looks very interesting. Would the search for the location mean that the pic would search through the 512 byte blocks looking for known data bytes? Or would this be done using a windows program as mentioned by stevenm86? Anyone know of such a program?
I am not a PC programmer, so this approach looks appealing. Transferring large amounts of data via RS232 is too slow for my application.
I have formatted a MMC card in a multi-card reader, but didn't know which way to go other than the rather daunting FAT method of storing files. FAT didn't seem appropriate since there would be a large number of records on the MMC. |
|
|
Bart
Joined: 12 Jul 2005 Posts: 49
|
|
Posted: Tue Sep 19, 2006 1:17 pm |
|
|
libor wrote: | You can format a blank MMC in a PC, write a single continuous file on it . |
Hello (Libor),
Is this also an opption if I want to create more than 1 file (with unknown different sizes) on the card ? _________________ I like Skype (www.skype.com), my username is BplotM |
|
|
|