Generating CATMAID URLs
-----------------------

This example will demonstrate how to use pymaid to generate URLs to a set of connectors. You can use this, for example, to perform random sampling of up- or downstream partners.

.. code:: ipython3

    import pandas as pd
    import pymaid
    
    rm = pymaid.connect_catmaid()

First get your neuron of interest:

.. code:: ipython3

    n = pymaid.get_neuron(16)

A simple example: Downstream sampling by connectors
===================================================

.. code:: ipython3

    # Get all connectors
    cn = n.connectors
    
    # Add a column with the URL to coordinates
    cn['URL'] = pymaid.url_to_coordinates(cn, 
                                          stack_id=5,
                                          active_node_id=cn.connector_id.values)
    cn.head()




.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }
    
        .dataframe tbody tr th {
            vertical-align: top;
        }
    
        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>treenode_id</th>
          <th>connector_id</th>
          <th>relation</th>
          <th>x</th>
          <th>y</th>
          <th>z</th>
          <th>URL</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>127234</td>
          <td>999188</td>
          <td>0</td>
          <td>438817</td>
          <td>164242</td>
          <td>217440</td>
          <td>https://neuropil.janelia.org/tracing/fafb/v14/...</td>
        </tr>
        <tr>
          <th>1</th>
          <td>2591</td>
          <td>97954</td>
          <td>0</td>
          <td>437120</td>
          <td>160998</td>
          <td>211920</td>
          <td>https://neuropil.janelia.org/tracing/fafb/v14/...</td>
        </tr>
        <tr>
          <th>2</th>
          <td>2665</td>
          <td>98300</td>
          <td>0</td>
          <td>437184</td>
          <td>162324</td>
          <td>214880</td>
          <td>https://neuropil.janelia.org/tracing/fafb/v14/...</td>
        </tr>
        <tr>
          <th>3</th>
          <td>2646</td>
          <td>98373</td>
          <td>0</td>
          <td>437042</td>
          <td>162452</td>
          <td>214120</td>
          <td>https://neuropil.janelia.org/tracing/fafb/v14/...</td>
        </tr>
        <tr>
          <th>4</th>
          <td>2654</td>
          <td>98415</td>
          <td>0</td>
          <td>436761</td>
          <td>163690</td>
          <td>214440</td>
          <td>https://neuropil.janelia.org/tracing/fafb/v14/...</td>
        </tr>
      </tbody>
    </table>
    </div>



In above example we are using ``stack_id=5``. This is the ID of the image stack for this project and is encoded in the URL as ``sid0=5``.

Now you can export this DataFrame. Two quick examples.

To CSV:

.. code:: ipython3

    cn.to_csv('filename.csv')

To clipboard:

.. code:: ipython3

    cn.to_clipboard()

Randomizing
===========
Randomizing the order e.g. for random sampling of downstream partners is straight forward:

Subset to presynapses:

.. code:: ipython3

    pre = cn[cn.relation == 0]

Randomize order:

.. code:: ipython3

    ran = pre.sample(frac=1)
    ran.head()




.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }
    
        .dataframe tbody tr th {
            vertical-align: top;
        }
    
        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>treenode_id</th>
          <th>connector_id</th>
          <th>relation</th>
          <th>x</th>
          <th>y</th>
          <th>z</th>
          <th>URL</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>868</th>
          <td>10350265</td>
          <td>10350269</td>
          <td>0</td>
          <td>358255</td>
          <td>151135</td>
          <td>167240</td>
          <td>https://neuropil.janelia.org/tracing/fafb/v14/...</td>
        </tr>
        <tr>
          <th>1423</th>
          <td>10366582</td>
          <td>10366581</td>
          <td>0</td>
          <td>336551</td>
          <td>159297</td>
          <td>177360</td>
          <td>https://neuropil.janelia.org/tracing/fafb/v14/...</td>
        </tr>
        <tr>
          <th>785</th>
          <td>10345534</td>
          <td>10345533</td>
          <td>0</td>
          <td>375162</td>
          <td>146617</td>
          <td>170760</td>
          <td>https://neuropil.janelia.org/tracing/fafb/v14/...</td>
        </tr>
        <tr>
          <th>597</th>
          <td>10337834</td>
          <td>10337835</td>
          <td>0</td>
          <td>385421</td>
          <td>134986</td>
          <td>173200</td>
          <td>https://neuropil.janelia.org/tracing/fafb/v14/...</td>
        </tr>
        <tr>
          <th>1591</th>
          <td>10370697</td>
          <td>10370695</td>
          <td>0</td>
          <td>341188</td>
          <td>153709</td>
          <td>169440</td>
          <td>https://neuropil.janelia.org/tracing/fafb/v14/...</td>
        </tr>
      </tbody>
    </table>
    </div>



A more complex example: sampling inputs by treenode
===================================================

First we have to get the presynaptic treenode IDs:

.. code:: ipython3

    post_details = pymaid.get_connector_details(n.postsynapses)
    post_details.head()


.. parsed-literal::

    INFO  : Data for 412 of 412 unique connector IDs retrieved (pymaid)




.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }
    
        .dataframe tbody tr th {
            vertical-align: top;
        }
    
        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>connector_id</th>
          <th>presynaptic_to</th>
          <th>postsynaptic_to</th>
          <th>presynaptic_to_node</th>
          <th>postsynaptic_to_node</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>474311</td>
          <td>203840</td>
          <td>[7911576, 8638, 16, 6483677]</td>
          <td>474312</td>
          <td>[2736161, 474295, 124991, 21109788]</td>
        </tr>
        <tr>
          <th>1</th>
          <td>573905</td>
          <td>54753</td>
          <td>[7634, 279469, 30358, 17238, 16, 597080, 39182...</td>
          <td>573904</td>
          <td>[573902, 1482460, 1670909, 2875768, 127266, 28...</td>
        </tr>
        <tr>
          <th>2</th>
          <td>1335332</td>
          <td>55153</td>
          <td>[8408275, 16]</td>
          <td>1326548</td>
          <td>[27245140, 125650]</td>
        </tr>
        <tr>
          <th>3</th>
          <td>1482374</td>
          <td>203840</td>
          <td>[10379190, 9052688, 279469, 597085, 597100, 59...</td>
          <td>1277055</td>
          <td>[2875773, 30223253, 1482362, 2875771, 2875774,...</td>
        </tr>
        <tr>
          <th>4</th>
          <td>1510194</td>
          <td>288558</td>
          <td>[16, 203840]</td>
          <td>1510193</td>
          <td>[124597, 1510195]</td>
        </tr>
      </tbody>
    </table>
    </div>



If you have connectors without a presynaptic treenode, you will have to fix that and rerun above code before moving on!

.. code:: ipython3

    post_details[post_details.presynaptic_to_node.isnull()]




.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }
    
        .dataframe tbody tr th {
            vertical-align: top;
        }
    
        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>connector_id</th>
          <th>presynaptic_to</th>
          <th>postsynaptic_to</th>
          <th>presynaptic_to_node</th>
          <th>postsynaptic_to_node</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>401</th>
          <td>20041670</td>
          <td>None</td>
          <td>[4954519, 16]</td>
          <td>None</td>
          <td>[20041667, 18719343]</td>
        </tr>
      </tbody>
    </table>
    </div>



Next we need to get the coordinates for those treenodes.

.. code:: ipython3

    # Now get the locations
    locs = pymaid.get_node_location(post_details.presynaptic_to_node.values)
    locs.head()




.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }
    
        .dataframe tbody tr th {
            vertical-align: top;
        }
    
        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>node_id</th>
          <th>x</th>
          <th>y</th>
          <th>z</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>10362880</td>
          <td>367320.0</td>
          <td>152292.0</td>
          <td>177760.0</td>
        </tr>
        <tr>
          <th>1</th>
          <td>18804742</td>
          <td>466093.0</td>
          <td>234873.0</td>
          <td>38560.0</td>
        </tr>
        <tr>
          <th>2</th>
          <td>18802702</td>
          <td>466727.0</td>
          <td>233599.0</td>
          <td>33640.0</td>
        </tr>
        <tr>
          <th>3</th>
          <td>10342419</td>
          <td>381419.0</td>
          <td>140309.0</td>
          <td>180280.0</td>
        </tr>
        <tr>
          <th>4</th>
          <td>10315798</td>
          <td>345150.0</td>
          <td>159327.0</td>
          <td>186640.0</td>
        </tr>
      </tbody>
    </table>
    </div>



Now we have to merge the locations into the connector Dataframe

.. code:: ipython3

    cn_merged = pd.merge(post_details.set_index('presynaptic_to_node'),
                         locs.set_index('node_id'),
                         left_index=True, right_index=True)
    cn_merged.head()




.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }
    
        .dataframe tbody tr th {
            vertical-align: top;
        }
    
        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>connector_id</th>
          <th>presynaptic_to</th>
          <th>postsynaptic_to</th>
          <th>postsynaptic_to_node</th>
          <th>x</th>
          <th>y</th>
          <th>z</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>437124</th>
          <td>7265239</td>
          <td>202541</td>
          <td>[8680461, 16, 13782]</td>
          <td>[28458449, 1786, 22231380]</td>
          <td>438062.0</td>
          <td>140370.0</td>
          <td>203160.0</td>
        </tr>
        <tr>
          <th>437172</th>
          <td>7265176</td>
          <td>202541</td>
          <td>[30571, 40637, 16, 4205169]</td>
          <td>[6428950, 447292, 1819, 7265200]</td>
          <td>437010.0</td>
          <td>141171.0</td>
          <td>204400.0</td>
        </tr>
        <tr>
          <th>474312</th>
          <td>474311</td>
          <td>203840</td>
          <td>[7911576, 8638, 16, 6483677]</td>
          <td>[2736161, 474295, 124991, 21109788]</td>
          <td>421574.0</td>
          <td>127771.0</td>
          <td>194720.0</td>
        </tr>
        <tr>
          <th>573904</th>
          <td>573905</td>
          <td>54753</td>
          <td>[7634, 279469, 30358, 17238, 16, 597080, 39182...</td>
          <td>[573902, 1482460, 1670909, 2875768, 127266, 28...</td>
          <td>434460.0</td>
          <td>165646.0</td>
          <td>216160.0</td>
        </tr>
        <tr>
          <th>1275576</th>
          <td>2873212</td>
          <td>203840</td>
          <td>[595239, 16]</td>
          <td>[2873213, 127687]</td>
          <td>436480.0</td>
          <td>147325.0</td>
          <td>221360.0</td>
        </tr>
      </tbody>
    </table>
    </div>



Let's add the url to the presynaptic node and some additional info: the name of the presynaptic neuron and whether it already has a soma

.. code:: ipython3

    cn_merged['url'] = pymaid.url_to_coordinates(cn_merged, stack_id=5)
    cn_merged['neuron_name'] = cn_merged.presynaptic_to.astype(str).map(pymaid.get_names(cn_merged.presynaptic_to.values))
    cn_merged['has_soma'] = cn_merged.presynaptic_to.map(pymaid.has_soma(cn_merged.presynaptic_to.values))
    
    cn_merged.head()




.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }
    
        .dataframe tbody tr th {
            vertical-align: top;
        }
    
        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>connector_id</th>
          <th>presynaptic_to</th>
          <th>postsynaptic_to</th>
          <th>postsynaptic_to_node</th>
          <th>x</th>
          <th>y</th>
          <th>z</th>
          <th>url</th>
          <th>neuron_name</th>
          <th>has_soma</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>437124</th>
          <td>7265239</td>
          <td>202541</td>
          <td>[8680461, 16, 13782]</td>
          <td>[28458449, 1786, 22231380]</td>
          <td>438062.0</td>
          <td>140370.0</td>
          <td>203160.0</td>
          <td>https://neuropil.janelia.org/tracing/fafb/v14/...</td>
          <td>putative Gad1-F-600202 40613 AJ</td>
          <td>True</td>
        </tr>
        <tr>
          <th>437172</th>
          <td>7265176</td>
          <td>202541</td>
          <td>[30571, 40637, 16, 4205169]</td>
          <td>[6428950, 447292, 1819, 7265200]</td>
          <td>437010.0</td>
          <td>141171.0</td>
          <td>204400.0</td>
          <td>https://neuropil.janelia.org/tracing/fafb/v14/...</td>
          <td>putative Gad1-F-600202 40613 AJ</td>
          <td>True</td>
        </tr>
        <tr>
          <th>474312</th>
          <td>474311</td>
          <td>203840</td>
          <td>[7911576, 8638, 16, 6483677]</td>
          <td>[2736161, 474295, 124991, 21109788]</td>
          <td>421574.0</td>
          <td>127771.0</td>
          <td>194720.0</td>
          <td>https://neuropil.janelia.org/tracing/fafb/v14/...</td>
          <td>APL 203841 MR JSL</td>
          <td>True</td>
        </tr>
        <tr>
          <th>573904</th>
          <td>573905</td>
          <td>54753</td>
          <td>[7634, 279469, 30358, 17238, 16, 597080, 39182...</td>
          <td>[573902, 1482460, 1670909, 2875768, 127266, 28...</td>
          <td>434460.0</td>
          <td>165646.0</td>
          <td>216160.0</td>
          <td>https://neuropil.janelia.org/tracing/fafb/v14/...</td>
          <td>neuron 54754</td>
          <td>False</td>
        </tr>
        <tr>
          <th>1275576</th>
          <td>2873212</td>
          <td>203840</td>
          <td>[595239, 16]</td>
          <td>[2873213, 127687]</td>
          <td>436480.0</td>
          <td>147325.0</td>
          <td>221360.0</td>
          <td>https://neuropil.janelia.org/tracing/fafb/v14/...</td>
          <td>APL 203841 MR JSL</td>
          <td>True</td>
        </tr>
      </tbody>
    </table>
    </div>



OK, nice. Let's rearrange and get rid of superfluous columns:

.. code:: ipython3

    cn_merged = cn_merged[['presynaptic_to', 'neuron_name', 'has_soma', 'connector_id', 'x', 'y', 'z', 'url']]
    cn_merged.head()




.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }
    
        .dataframe tbody tr th {
            vertical-align: top;
        }
    
        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>presynaptic_to</th>
          <th>neuron_name</th>
          <th>has_soma</th>
          <th>connector_id</th>
          <th>x</th>
          <th>y</th>
          <th>z</th>
          <th>url</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>437124</th>
          <td>202541</td>
          <td>putative Gad1-F-600202 40613 AJ</td>
          <td>True</td>
          <td>7265239</td>
          <td>438062.0</td>
          <td>140370.0</td>
          <td>203160.0</td>
          <td>https://neuropil.janelia.org/tracing/fafb/v14/...</td>
        </tr>
        <tr>
          <th>437172</th>
          <td>202541</td>
          <td>putative Gad1-F-600202 40613 AJ</td>
          <td>True</td>
          <td>7265176</td>
          <td>437010.0</td>
          <td>141171.0</td>
          <td>204400.0</td>
          <td>https://neuropil.janelia.org/tracing/fafb/v14/...</td>
        </tr>
        <tr>
          <th>474312</th>
          <td>203840</td>
          <td>APL 203841 MR JSL</td>
          <td>True</td>
          <td>474311</td>
          <td>421574.0</td>
          <td>127771.0</td>
          <td>194720.0</td>
          <td>https://neuropil.janelia.org/tracing/fafb/v14/...</td>
        </tr>
        <tr>
          <th>573904</th>
          <td>54753</td>
          <td>neuron 54754</td>
          <td>False</td>
          <td>573905</td>
          <td>434460.0</td>
          <td>165646.0</td>
          <td>216160.0</td>
          <td>https://neuropil.janelia.org/tracing/fafb/v14/...</td>
        </tr>
        <tr>
          <th>1275576</th>
          <td>203840</td>
          <td>APL 203841 MR JSL</td>
          <td>True</td>
          <td>2873212</td>
          <td>436480.0</td>
          <td>147325.0</td>
          <td>221360.0</td>
          <td>https://neuropil.janelia.org/tracing/fafb/v14/...</td>
        </tr>
      </tbody>
    </table>
    </div>



Now you can randomize this table and start sampling:

.. code:: ipython3

    cn_merged.sample(frac=1).to_clipboard()

These are the relevant functions:

.. autosummary::
    
    pymaid.url_to_coordinates
    pymaid.get_node_location