Skip to content

Overview of Python Offline Analysis Tools

Read block files directly from disk or SEV files directly from RS4.

TDT Data Storage

Data collected or used by TDT software is stored in tanks - special directories on your hard drive. Each time you press 'Record' in Synapse, a new block is created within the tank. Within a block different stores can record different types of events at different rates. The blocks are special folders within the tank directories.

read_block

read_block is an all-in-one function for reading TDT data into Python. It needs only one input: the block path.

import tdt
data = tdt.read_block('C:\TDT\TDTExampleData\Algernon-180308-130351')

read_block will return a structure containing all recorded data from that block, organized by type:

  1. epocs are values stored with onset and offset timestamps that can be used to create time-based filters on your data. They can be created by the Epoch Data Storage gizmo, Logic gizmos, Stimulation gizmos, and many more.

    1. If Runtime Notes were enabled in Synapse, they will appear in data.epocs.Note. The notes themselves will be in data.epocs.Note.notes.

      See the Synapse Manual for more information on Runtime Notes.

  2. streams are continuous single channel or multichannel recordings, like those stored by the Stream Data Storage gizmo, the Fiber Photometry gizmo, and many others. The structure includes the data array and sampling rate.

  3. snips are short snippets of data collected on a trigger. For example, action potentials recorded around threshold crossings in the Spike Sorting gizmos, or fixed duration snippets recorded by the Strobe Store gizmo. This structure includes the waveforms, channel numbers, sort codes, trigger timestamps, and sampling rate.

  4. scalars are similar to epocs but can be single or multi-channel values and only store an onset timestamp when triggered. These can be created by the Strobe Store gizmo.

The returned structure also contains an info field with block start/stop times, duration, and information about the Subject, User, and Experiment that it came from (if the block was created in Synapse).

The returned structure also contains an info field with block start/stop times, duration, and information about the Subject, User, and Experiment that it came from (if the block was created in Synapse).

read_block uses input parameters to refine the imported data. To extract specific event types only, use the evtype parameter. For example, to import epocs and snippets only, use this:

data = tdt.read_block('C:\TDT\TDTExampleData\Algernon-180308-130351',
    evtype=['epocs','snips'])

Use the store parameter to extract a particular data store by name, in this example a streaming event called 'Wav1'. Combine this with the channel parameter to extract a single channel, or list of channels, in this case channels 2 and 4:

data = tdt.read_block('C:\TDT\TDTExampleData\Algernon-180308-130351',
    store='Wav1', channel=[2,4])

You can also filter by time, if you are only interested in portions of the recording, or if the entire recording won't fit into available memory (RAM) at one time. Use the t1 and t2 parameters to specify the start and stop time, in seconds, to retrieve from the block. This example reads only from time t1=10s to time t2=20s of the block into Python:

data = tdt.read_block('C:\TDT\TDTExampleData\Algernon-180308-130351',
    t1=10, t2=20)

read_block offers many more useful options that are described in its help documentation.

print(tdt.read_block.__doc__)

epoc_filter

epoc_filter applies advanced epoc filtering to extracted data. For example, if you only want to look at data around a certain epoc event, you will use epoc_filter to do this. See the Raster/PSTH example for a complete demonstration.

To only look at data around the epoc event timestamps, use a t parameter to set the filter time range. In this example, data from 20ms before the Levl epoc to 50ms after the onset is kept.

data = tdt.epoc_filter(data, 'Levl', t=[-0.02, 0.07])

To only look at data when an epoc was a certain value or values, use a values parameter. In this example, only data when the Freq epoc was equal to 9000 or 10000 is retained.

data = tdt.epoc_filter(data, 'Freq', values=[9000, 10000])

If you want to look for a particular behavioral response that occurs sometime during the allowed time range, use the modifiers filter. In this example, only data when the Freq epoc was 10000 AND the Resp epoc had a value of 1 sometime during the Freq epoc is retained.

data = tdt.epoc_filter(data, 'Freq', values=[10000])
data = tdt.epoc_filter(data, 'Resp', modifiers=[1])

As you can see, for complex filtering the output from one call to epoc_filter can become the input to the next call to epoc_filter. If your data sets are large, or if you are iterating through many combinations of epoc variables, it is preferred to extract only the epocs and do all of the epoc filtering first to find the valid time ranges that match the filter, and then use this as the ranges input to read_block to extract all events (including snips, streams) on only those valid time ranges.

# read just the epoc events
data = tdt.read_block(block_path, evtype=['epocs'])

# use the epocs to find time ranges we want
data = tdt.epoc_filter(data, 'Freq', values=[9000, 10000])
data = tdt.epoc_filter(data, 'Levl', values=[70, 80, 90])
data = tdt.epoc_filter(data, 'Resp', modifiers=[1])

# read just the value time ranges from the whole data set
data = tdt.read_block(block_path, ranges=data.time_ranges)

read_sev

read_sev reads SEV files into a Python structure. SEV files are created by the RS4 Data Streamer or by enabling the Discrete Files option when streaming continuous signals in Synapse. SEV files consist of a single channel of data per file, with a short header, so it is very fast to read them. read_block will automatically call read_sev if it finds SEV files in the block directory. Like read_block, read_sev needs only one input: the block path.

data = tdt.read_sev('C:\TDT\Synapse\Tanks\Exp1-160921-120606\Sub1-1')

read_sev will return a structure containing the streams that it found. Each stream field includes the data array and sampling rate.