Python EDF Library -- To read cpap machine datalog files

I posted as part of another thread, but I thought I should create a new topic to let more of the possible developer crowd that might be reading this board know that I have started a python library to parse EDF files and pull out my sleep data from my ResMed S9 so I can do more with the data and not have to run windows This is *far* from complete but is a start and I'd like any help I can get with it and really flesh this thing out so its a useful module which can then be used in a variety of different tools.

Here is the github url where I'm storing the source: http://github.com/rays/pyedf the code is 100% free to use, see the LICENSE file that is shipped with the source. And again I ask for help in making sure everything is correct and fully featured. Every little bit helps

PS. I used this website to figure out the EDF spec, the spec is not overly detailed though. http://www.edfplus.info/specs/index.html

.

rays wrote:I posted as part of another thread, but I thought I should create a new topic to let more of the possible developer crowd that might be reading this board know that I have started a python library to parse EDF files and pull out my sleep data from my ResMed S9 so I can do more with the data and not have to run windows This is *far* from complete but is a start and I'd like any help I can get with it and really flesh this thing out so its a useful module which can then be used in a variety of different tools.

Here is the github url where I'm storing the source: http://github.com/rays/pyedf the code is 100% free to use, see the LICENSE file that is shipped with the source. And again I ask for help in making sure everything is correct and fully featured. Every little bit helps

PS. I used this website to figure out the EDF spec, the spec is not overly detailed though. http://www.edfplus.info/specs/index.html

Thanks! I'll have to look at your spec to see if my F/P 244 files are actually EDF (the file extension is .FPH).
Seem to be a a bunch of text followed by a few blocks of what look like IEEE floats. I suppose that it's possible that its actually EDF format but with a different extension but I would have to take a look at your code to find out.

The Windows-only software for this Fisher/Paykel CPAP installs a SQL server, and has to "download" [sic] the data from the USB stick to a database on the PC. The reason for the "[sic]" is that I would think that this would be considered an upload and not a download, but never mind.

It really would be nice not to have to rely on a proprietary Microsoft OS to get at this data, so I applaud your efforts, and if and when time permits, I'll take a look at what you have.

BTW I just recently, in the last week or two, started getting into Python. I have the misfortune to be working with a cross-development system that only works
in Windows. Luckily a Python-based approach will work on my home Linux machine.

I've only started using python in the last week to whip up some command-line uitilties. I find that it's somehow "fun" to use. Nice and simple, and
has a lot of "Lisp-ish" features. A nice break from the tedium of C++. and all it's awkward template sturff. I found that after about eight hours of alternating coding with using the interpreter and browsing the help file, Python it was very easy to get proficient with it, and the dynamic typing is so much easier.

Now implementing a GUI app with Python is probably a bit more involved, and I understand that the normal case is to use the Tcl/Tk module.
Bleachh!


Well getting way OT here so I'll sign off...

And feel free to PM, because software discussions are not appropriate for this forum, I suspect

Larry

larry63 wrote:Well getting way OT here so I'll sign off...

And feel free to PM, because software discussions are not appropriate for this forum, I suspect

Oh no. I think software discussions probably rank just below sex, pets and food as favorite OT topics. Possibly even above food.

First time poster, long time lurker. I have the Resmed S9 Autoset. Wow. Looking at the Signal Labels in the EDF format, the machine is doing everything and the ResScan software is just putting a pretty graph on the screen. (The signals are already all analyzed in the machine I guess. I see labels like Mask Pressure, Leak, Snore Index, etc.) This is going to be awesome. I have a question though. If they are using a 1GB SD card and each night takes about 3MB, why does the machine only store 7 days worth of detailed data instead of months worth? If the machine can see a 2GB card, you could theoretically store a year of data at least...

linagee wrote:First time poster, long time lurker. I have the Resmed S9 Autoset. Wow. Looking at the Signal Labels in the EDF format, the machine is doing everything and the ResScan software is just putting a pretty graph on the screen. (The signals are already all analyzed in the machine I guess. I see labels like Mask Pressure, Leak, Snore Index, etc.) This is going to be awesome. I have a question though. If they are using a 1GB SD card and each night takes about 3MB, why does the machine only store 7 days worth of detailed data instead of months worth? If the machine can see a 2GB card, you could theoretically store a year of data at least...

Probably just an arbitrary decision based on the size of cards available when the firmware was freleased.

larry63 wrote:

If they are using a 1GB SD card and each night takes about 3MB, why does the machine only store 7 days worth of detailed data instead of months worth? If the machine can see a 2GB card, you could theoretically store a year of data at least...

Probably just an arbitrary decision based on the size of cards available when the firmware was freleased.

I'm not sure card size has ever had anything to do with storing only seven days high res data. I haven't got it all figured out yet how it's happening, but the Resscan software appears to only retain 7 days hi res data in the patient folder. I've been downloading every one or two days for the last three weeks all the data off the card and saving it on my PC. At the same time I've been downloading it into Resscan. But when I look for flow data it's there for only the last seven days. Haven't determined yet what exactly happens with the overwrite and discard options during the download.

Maybe originally the seven days was card based but I wonder if Resmed hasn't retained the seven day limit because anything older than that is not considered useful for sleep apnea diagnosis purposes. Just my speculation.

ResScan only deletes high resolution data (HRD) when the discard option is chosen during download, assuming that the files haven't been corrupted. To be honest, I'm not sure exactly what the discard option was intended to do except save disk space on DME computers. I suspect that they meant for it to wipe out all HRD that isn't present on the SD card at the time of the download. However, every time I've hit the discard option something different has happen to my HRD. Usually, I'd only get some of it from the card and everything that was already in the various *.RLK files was deleted. So, I started over with a new patient file and I ALWAYS choose overwrite. After each download, I do a save in ResScan, then exit. Next I go to my patient folder and set the readonly attribute for all *.RLK files. Although I haven't tested it, that will probably stop ResScan from being able to delete any HRD if I accidentally choose the discard option during a download. So far, I have all of my HRD. At this point, that's only 8 days, but it does show that ResScan will keep more than 7 days of HRD as long as you avoid the discard option during downloads and don't experience any file corruptions.

As to the 365/30/7 day card retention scheme, I'll post some charts this evening that will show why a full year's worth of data, including HRD, will not reliably fit on a 1Gb SD card. Choosing 7 days for HRD instead of 14 or 30 may have been arbitrary or it may have been based on what's practical from both an analysis and RRT/MD time availability perspective, but 365 days of HRD would definitely require more than a 1 Gb card, which ResMed doesn't provide. An argument could be made for allowing larger capacity cards, but that's probably a moot point, as that decision was made early on in the S9 design and may not be easily change. From a practical standpoint, given ResMed's target audience of RRTs and MDs, there probably isn't a good reason for a larger card.

Since we're obviously OT now, I'll post the charts in a new thread.

BernieRay wrote:As to the 365/30/7 day card retention scheme, I'll post some charts this evening that will show why a full year's worth of data, including HRD, will not reliably fit on a 1Gb SD card.

I'll be interested in seeing how you come up with that. My calculations, which I've posted a time or two, indicate that there will only be about 30 to 40 mb of data total for a year's worth of data on the card.

I've run out of time today. I have the charts ready to go, but not the post to go with them. I'll try to finish that up tomorrow. Sorry about the delay.

Hi, I have a Fisher and Paykal ICON Premo and would like to be able to read the data without buying software. I tried using EDFBrowser http://www.teuniz.net/edfbrowser/ but I get an error saying 5 char is not ASCII, which is correct when you look at the data. Does anyone have any idea what format the FPH data is in?

First line is ASCII text of product serial number etc, then 3 blank lines followed by non ASCII data.

elec wrote:...I tried using EDFBrowser http://www.teuniz.net/edfbrowser/ but I get an error saying 5 char is not ASCII, which is correct when you look at the data. ...

You may want to try Polyman to import your data, I found that the EDF browser you are using does not cleanly handle EDF+D files which may be what your machine uses. I tried quite a few viewers and found none of them cleanly handle all the variants used, and things are worse because many of the files do not fully conform to the specs fir EDF or EDF+.

I've started to pull together source files and am going through the whole process of handling all the ResMed data and would be happy to look at other machines at the same time. I have a Choice Medical pulse-oximiter MD300W1 (their software is terrible) for which I have also done some work on the file format used in its raw data and can now import it also and am able to convert it to EDF for use in an external viewer. My longer term goal is to have an application to help view the various files all in one common app.

The ResMed S9 appears to use standard EDF format for most of their files, but use EDF+D (discontinuous EDF+) for their annotation information where they store the events the machine found.

ooo, not sure how I missed this... will have to check it out. Rescan is effing slow and EDFBrowser was confusing me... I opened the same data file multiple times and only saw an actual line on the graph the first time. Attempting to export to TXT file resulted in a 3.1 gig file before it hung up and died.

ohammersmith wrote:ooo, not sure how I missed this... will have to check it out. Rescan is effing slow and EDFBrowser was confusing me... I opened the same data file multiple times and only saw an actual line on the graph the first time. Attempting to export to TXT file resulted in a 3.1 gig file before it hung up and died.

EDFBrowser opens up every time with the time and amplitude scales set to the default very high number. Every time I open it I have to change the timescale to something like 1 hr or 20 min and the amplitude to .2L/sec in order to see a line. Could this be the reason you saw the graph only the first time?
What did you use to convert to text? I've never had a hitch using edf2text.exe. If you drag the edf file icon onto the edf2text icon you'll see a command window open momentarily, and then nothing until it's finished and the extracted files show up in the folder. I always put a copy of the edf2text file in the same folder with the edf files.

But then this is on a windows machine. If you're using the Linux versions then maybe things run differently. I don't even know if there is a Linux version of edf2text.

idamtnboy wrote:I'm not sure card size has ever had anything to do with storing only seven days high res data. I haven't got it all figured out yet how it's happening, but the Resscan software appears to only retain 7 days hi res data in the patient folder. I've been downloading every one or two days for the last three weeks all the data off the card and saving it on my PC. At the same time I've been downloading it into Resscan. But when I look for flow data it's there for only the last seven days. Haven't determined yet what exactly happens with the overwrite and discard options during the download.

Maybe originally the seven days was card based but I wonder if Resmed hasn't retained the seven day limit because anything older than that is not considered useful for sleep apnea diagnosis purposes. Just my speculation.

This discussion is WAY above my head - except ...
The Resmed S8 Elite could hold and transfer 4-5 days of Detailed data to the data card and AutoScan and/or ResScan software. I ran a continuous single file w/Detailed data for 17 months.
The Resmed (S8) VPAP Auto can hold and transfer 2 days of Detailed data to the data card and ResScan. I've been running a continuous file of Detailed data for 21 months now.

You are misusing the Overwrite and Discard options. When the software asks if you want to Overwrite or Discard it gives a date. If you had downloaded data for that date before to this patient file use the Discard option. Otherwise that date's data is overwritten.
Tomorrow morning start TWO files. Download data from the card to each file. The next morning try doing one file w/the Overwrite option and the other file w/the Discard option. Assuming you are working w/the S9s instead of the S8s I have no idea how many days of Detailed data the S9s are designed to hold and transfer so you may have to run two files for a couple of days to get the drift. How many days of Detailed data does your Clinicians Manual say will be held and reported for the S9? I would imagine that it may well vary between the different S9 models.