Data Compression

[Gabriel]

Hi all,

I'm looking for suggestions...

I have a datalogger I've built and coded,
16F876A, 25AA256 eeprom, DS1305.

It takes a single 10bit reading at programmable intervals, takes a time stamp.

It saves data as follows:
Day, Month, Year, Hour, Minutes, Data Hi-byte, Data Low-Byte.

Code works, everything is nice and great, BUT ;)

I'm spending more memory on time stamp info than data.

I'm sure alot of you have gone through this before... so I'm looking for maybe some simple way to compress the time stamp info.

My idea so far is as follows, worst case Scenario:

[temtronic]

Some ideas....
Using 1-365 for the day of the year takes 2 bytes...

Don't save the year byte...have the filename encoded with that.
Same could be used for the month...ie:filename 1102data.txt means data collected in Feb of 2011.So only 1 byte for day of month....

Depending on how often the data is recorded, if you only sample every 15 minutes, the time can be represented in one byte(24 * 4=96) or if you sample every 6 minutes you'll get maximum samples/day(24hr*60min=1440 minutes/256=5.625).

[gpsmikey]

Another option (sort of a spin on the way windows/unix etc store time) is to have some arbitrary start time that you could store, then each time stamp is stored as an offset in seconds or minutes or whatever incremental resolution you need. If you used minutes, 16 bits would give you 45 days worth of "time stamps".

mikey
_________________
mikey
-- you can't have too many gadgets or too much disk space !
old engineering saying: 1+1 = 3 for sufficiently large values of 1 or small values of 3

[Ttelmah]

Yes.
Remember the time.h library, has a mktime function, which returns a time converted to a 32bit count of seconds from a start point (based exactly on the Unix version). Use this on a start date/time to give you a 'beginning of time' marker, and subtract this from the values converted the same way from you sample times. Divide by 60, so you are working in minutes, rather than seconds, and store the low 3 bytes of the int32, and you can handle just under 32 years in 3 bytes.....

Best Wishes

[Gabriel]

Hey Guys,

Thanks for your suggestions, i had not thought of storing dates as a day out of 365... nice!

i had however thought of using the latter suggested Win/Unix method... but thought it would be alot of coding... and testing....
but i had no idea there was a time.h library! (never needed it)

ill check that out! Thanks!

i was even thinking of Huffman at some point.... i dont suppose there is a Huffman-Compression.h file laying arround? jejejeje...

Thank you for your suggestions!
G.
_________________
CCS PCM 5.078 & CCS PCH 5.093

[Ttelmah]

I doubt if Huffman coding would help.
It is normally optimal, where you have a stream of symbols with a probability 'tree' on their occurrence, and some having higher probabilities than others. If you use a pure clock counter, probabilities should be equal for all entries. Result, no saving...

Best Wishes