Tutorial

This tutorial will introduce you to the basics of using pymaid. This is not supposed to be comprehensive but rather to give you a flavor of how things work. For inspiriation, explore the example gallery and for detailed explanations have a look at the API documentation.

Before we get started: pymaid is built on top of navis and you will note that we are using many of its functions throughout this tutorial and the examples. I highly recommend you familiarize yourself with it!

Connecting to your Catmaid Server

At the beginning of each session, you have to initialise a connection to your CATMAID server. These connections are represented by CatmaidInstance which holds the URL and your credentials.

You will find that all functions that fetch data accept such connections as remote_instance parameters. In most situations, however, you won’t use that parameter. That’s because once you have initialized a connection it becomes the “global default” connection and pymaid will use that connection unless told otherwise. Sounds complicated but is actually straight forward - let’s look at some code:

First things first: import pymaid!/

import pymaid

Now there are two ways to initialize a connection. The first one is to simply create an instance of CatmaidInstance with your credentials:

rm = pymaid.CatmaidInstance(server='https://www.your.catmaid-server.org',
                            api_token='YOURTOKEN',
                            http_user='user', # omit if not required
                            http_password='pw') # omit if not required

INFO  : Global CATMAID instance set. Caching is ON. (pymaid)

HTTP_USER and HTTP_PASSWORD are credentials you might use before even getting to the CATMAID landing page, not your CATMAID credentials. If your server does not require them, just leave them blank (''). Instead of your CATMAID credentials, you use an API TOKEN. See here how to generate yours.

The above way of making a connection works perfectly fine but it can get tedious to always type in (or copy-paste) your credentials. It also makes you vulnerable to accidentally reveal your credentials when sharing code with others. Not good!

Hence this second, much more convenient and much safer way of storing credentials: as environment variables. Store your credentials as CATMAID_SERVER_URL, CATMAID_API_TOKEN, CATMAID_HTTP_USER and CATMAID_HTTP_PASSWORD and then you can do this instead:

rm = pymaid.connect_catmaid()

INFO  : Global CATMAID instance set. Caching is ON. (pymaid)

Much easier and cleaner, isn’t it? The way to set up environment variables depends a bit on your operating system and what terminal (e.g. bash or zsh) you are using. I recommend you Google it if you are unsure but here are two articles that do a decent job describing it:

• MacOS

• Linux

rm now holds your credentials and mediates fetching data from your server. This CatmaidInstance has a few additional perks that we won’t cover just yet - check out the API docs and the examples for CatmaidInstance’ to learn, for example, how to access different projects on your server or limit the number of parallel queries.

Fetching data

Now that we have set up the connection, we can query the server for data. The examples below are by no means comprehensive! Have a look at all the functions for fetching data. Most of these functions will follow a pymaid.get_{data} naming convention:

For starters, let’s get a list of skeleton IDs from a single annotation:

skids = pymaid.get_skids_by_annotation('glomerulus DL4')
skids

[6170457, 483289, 23829, 6698696]

Your Catmaid server will provide most data as simple list or dictionary, like the list of skeleton IDs above. Whenever this makes sense, pymaid will convert that data to a pandas.DataFrame.

pandas is the data science library for Python and will help you analyze and visualize your data. I highly recommend familiarizing yourself with pandas! There are plenty of good tutorials out there but pandas’ own 10 Minutes to pandas is a good place to start.

As example, let’s get the treenode table for one of the above neurons:

tn_table = pymaid.get_node_table(skids[0])
tn_table.head()

INFO  : Retrieving 1 node table(s)... (pymaid)
INFO  : 7061 nodes retrieved. Creating table... (pymaid)

	node_id	parent_node_id	confidence	x	y	z	radius	creator	last_edited	skeleton_id	reviewers	tags
0	32048046	63373873	5	504762	216453	45160	-1	vallas	2020-07-15 16:55:44+00:00	6170457	NaN	NaN
1	63373873	32048044	5	504748	216392	45120	-1	helmickl	2020-07-15 16:55:44+00:00	6170457	NaN	NaN
2	32047785	63373858	5	503798	218009	46040	-1	vallas	2020-07-15 16:55:07+00:00	6170457	NaN	NaN
3	63373858	32047777	5	503820	218043	46080	-1	helmickl	2020-07-15 16:55:07+00:00	6170457	NaN	NaN
4	32042427	63373515	5	513298	222217	41240	-1	vallas	2020-07-15 16:36:29+00:00	6170457	NaN	NaN

Neurons

Catmaid neurons are complex structures: they consist of (tree)nodes and connectors, have names, annotations, tags, review status and so on. We could query our Catmaid server for all these data separately and manage it using lists or dictionaries… a pain in the neck, trust me.

pymaid to the rescue! CatmaidNeuron and their collections, CatmaidNeuronList, make managing and accessing neuron data a breeze as you will see below.

As of pymaid version 2.0.0, CatmaidNeuron/Lists are subclasses of navis TreeNeurons and NeuronList, respectively. This means we can plug and play with all navis functions.

Fetching CatmaidNeurons

There are two functions that will return CatmaidNeuron/Lists: pymaid.get_neuron() and pymaid.find_neurons(). The latter is a bit more sophisticated and allows intersections of search criteria - we’ll cover it elsewhere. For starters, let’s get two neurons by their skeleton IDs:

neuron_list = pymaid.get_neuron(['57311', '27295'])

neuron_list is a CatmaidNeuronList containing our two neurons. Calling it will give us a small summary:

neuron_list

	type	name	skeleton_id	n_nodes	n_connectors	n_branches	n_leafs	cable_length	soma	units
0	CatmaidNeuron	Uniglomerular mALT DA1 lPN 57312 LK	57311	5543	463	211	219	1364103.25	[3059181]	1 nanometer
1	CatmaidNeuron	Uniglomerular mALT DA1 lPN 27296 BH	27295	10216	507	304	314	1806000.25	[3005291]	1 nanometer

You can access individual CatmaidNeuron within the CatmaidNeuronList by various means. See example gallery for an in-depth explanation. For now, the easiest is to just index the CatmaidNeuronList like you would a list:

neuron_list[0]

type	CatmaidNeuron
name	Uniglomerular mALT DA1 lPN 57312 LK
id	57311
n_nodes	5543
n_connectors	463
n_branches	211
n_leafs	219
cable_length	1.3641e+06
soma	[3059181]
units	1 nanometer

Accessing neuron data

Treenodes, connectors, etc. are stored as attributes of CatmaidNeuron/Lists:

n = neuron_list[0]

# .nodes is a pandas DataFrame and we can use .head() to show the first couple entries
n.nodes.head()

	node_id	parent_id	creator_id	x	y	z	radius	confidence	type
0	1204977	1204976	4	438669.71875	140775.484375	196840.0	-1.0	5	slab
1	1204978	1204977	4	438669.53125	140795.328125	196880.0	-1.0	5	slab
2	861285	861279	25	488491.90625	183344.390625	139640.0	-1.0	5	slab
3	860950	860949	25	489096.68750	170746.453125	149040.0	-1.0	5	slab
4	862662	862649	25	464031.31250	143550.625000	173320.0	-1.0	5	slab

n.connectors.head()

	node_id	connector_id	type	x	y	z
0	1205514	2163979	0	437151.25000	145751.25000	205040.0
1	1207394	2884058	1	449176.09375	150388.56250	216920.0
2	1207044	2882175	0	446111.00000	152505.00000	215400.0
3	5400609	5400608	0	340840.62500	166648.68750	168240.0
4	5400758	5400757	0	347488.00000	165468.09375	160680.0

n.skeleton_id

'57311'

Almost all attributes exist in both CatmaidNeuron and CatmaidNeuronList. In the latter case, you will simply get that attribute over all neurons in the CatmaidNeuronList.

neuron_list.skeleton_id

array(['57311', '27295'], dtype='<U5')

Note

CatmaidNeuron/Lists support auto-completion. Try typing neuron_list. and hitting TAB.

Deal with missing data

Going back to the neuron’s summary, you will notice that review_status is NA:

neuron_list[0]

type	CatmaidNeuron
name	Uniglomerular mALT DA1 lPN 57312 LK
id	57311
n_nodes	5543
n_connectors	463
n_branches	211
n_leafs	219
cable_length	1.3641e+06
soma	[3059181]
units	1 nanometer

That’s because we are currently lacking that data. It will be retrieved/computed on-demand upon first explicit request:

neuron_list[0].review_status

79

Now review_status is known for the first neuron but not the second:

neuron_list

	type	name	skeleton_id	n_nodes	n_connectors	n_branches	n_leafs	cable_length	soma	units
0	CatmaidNeuron	Uniglomerular mALT DA1 lPN 57312 LK	57311	5543	463	211	219	1364103.25	[3059181]	1 nanometer
1	CatmaidNeuron	Uniglomerular mALT DA1 lPN 27296 BH	27295	10216	507	304	314	1806000.25	[3005291]	1 nanometer

Attributes are not automatically updated once they have been retrieved. You can force an update by calling the according get_... method:

neuron_list[0].get_review()

Most of these methods and attributes work on both, CatmaidNeuron and CatmaidNeuron:

neuron_list.review_status

array([79, 77])

For more advanced stuff (plotting, pruning, cutting, etc.) have a look at the example gallery.