Morphogénie Logiciels

Offre de stage

2017-11-15T00:00:00+01:00

Offre de stage: Identification des paramètres optimaux d’un modèle d’organisation multicellulaire.

Contexte

Fruit d’une collaboration entre l’Institut Clément Ader (ICA) et l’Institut de Recherche en Santé Digestive (IRSD), le projet MOCCAssIN (Modelisation of epithelio-stromal alterations in colorectal cancer initiation) financé par le plan Cancer 2014-2019, a pour but d’identifier les caractéristiques spécifiques des cellules épithéliales afin d’améliorer les dépistages lors de la phase d’analyse des cultures.

Objectif et travail attendu

Un modèle existant (la bibliothèque tyssue ) permet de décrire l’organisation multicellulaire ainsi que la dynamique d’interaction entre les cellules, de migration et de différenciation des cellules épithéliales. Ce modèle est décrit à l’aide d’un maillage d’éléments finis dont le comportement mécanique dépend de paramètres tels que l’élasticité de la membrane cellulaire ou la contractilité du cytosquelette. Ce jeu de paramètres est identifié via la minimisation d’une fonction d’énergie dépendant de la géométrie du tissu considéré. Un des enjeux du projet MOCCAssIN consiste à :

extraire de nouvelles informations géométriques à l’aide de méthodes d’analyse d’image 2D ou 3D.
enrichir la fonction coût à l’aide de ces nouvelles données afin de prendre en compte le recalage géométrique de l’organoïde sur les échantillons observés.
minimiser la nouvelle fonction coût afin d’obtenir des paramètres optimaux qui permettent à l’organoïde simulé d’évoluer de manière prédictive.

Le stage, situé dans la phase préliminaire du projet, consiste à étudier les différentes informations 2D et 3D qu’il est possible d’extraire (à l’aide de méthode de traitement d’image standard) des données fournies par l’IRSD. Ce projet constitue une pré-étude d’une thèse CIFRE qui poursuivra ce stage et débutera fin 2018.

Profil

Cette offre s’adresse à des étudiants en M2 ou Ecole d’ingénieurs en Mathématiques Appliquées et/ou Mécanique Numérique et/ou Traitement d’Images motivés par la poursuite d’étude en thèse.

Les compétences attendues sont : * maîtrise des bases en traitement d’images * maîtrise des bases d’optimisation numérique. * Notion sur la méthode par Eléments Finis

Le langage de programmation est Python.

Laboratoire d’accueil | ICA CNRS UMR 5312 / Université Paul Sabatier (poste basé à Toulouse) | Durée | 6 mois à compter du premier trimestre 2018 Gratification | 550 € / mois environ

Contact

Merci d’adresser par email 1 CV et 1 LM aux 5 personnes suivantes: - Florian BUGARIN - florian.bugarin@univ-tlse3 Stephane SEGONDS - [email protected] - Audrey Ferrand - [email protected] - Frederick Barreau - [email protected] - Guillaume GAY - [email protected]

Morphogénie Logiciels est née

2016-09-06T00:00:00+02:00

Après 4 ans d’activité en entreprise individuelle concentrée sur le service, une nouvelle étape s’ouvre avec la création de Morphogénie Logiciels SAS.

La nouvelle société s’oriente vers l’édition logicielle avec pour premier projet le développement de la bibliothèque tyssue.

Ce projet s’appuie sur la conviction que la modélisation des objets biologiques est la deuxième phase, après les approches big data, de la révolution numérique en bio-médecine. Les développements expérimentaux permettent d’aborder les objets d’intéret dans leur complexité tri-dimensionnelle. On pense ainsi aux avancées en microscopie permettant de suivre le dévelopement d’un organe au cours du temps, ou les progrès des recherches sur les cellules souches permettant de créer des “mini-organes”.

Mu spim, a ligth-sheet microscopy smartphone

2015-09-09T00:00:00+02:00

La version 0.1 du mu-spim est sortie! mu-spim v0.1 is out!

Go see the brand new website (might not work yet, depending how early you see this ;).

Mu-spim implements light sheet microscopy on a smartphone camera.

Le Mu-spim est un microscope à feuille de lumière pour smartphone

From leg-joint to tyssue, a refactoring story - Ep. 2

2015-08-21T00:00:00+02:00

That’s the second blog post of three reflecting on the modeling.

Performance - read this first !
Future plans - the future is already here
Visualization - even more future !

Previously

I started optimizing, then refactoring the leg-joint code to gain performance. As I was doing this, I realized it would be cleverer to go all the way back to the drawing board (actually a paper notebook), as maybe some parts of the simulation could be re-used with different geometries. After all, many epithelia share common traits with the leg imaginal disk. At the same time, Magali and I started talking with two physicists, François Molino and Cyprien Gay, who are applying bleeding edge soft-matter physics to biological tissues. It turns out Cyprien and his colleges recently published a milestone paper on continuous models; both were keen on trying cell based models, and the leg disk is of course very rich on this regard. So while I’m generalizing geometry, why not physics?

The road ahead

So how do we get there?

Classify existing biophysical tissue models.
Specify the subset of said models we want to address.
Try to craft an API.
Choose the proper libraries.
Code simple test cases (simple geometry, simple physics).
Repeat from 3 until the API and tech are +/- stable…
Implement a real world problem, same physics as in 5.
Try new physics at constant geometry.

The route through those points will start a bit far from code.

A very brief tour of the biophysical models of living tissues

As they imply dramatic changes in a tissue shape and organization, morphogenetic events have long been the focus of mechanical modeling efforts. As early as 1917, D’Arcy Thomson exposed the underlying mathematical and physical relationships underpinning cells and multi-cellular organism shapes and dynamical behaviors (in the seminal work On Growth and Form). In recent years, technical progresses have provided very detailed geometrical, dynamical and biochemical data on morphogenetic processes, allowing for ever finer mathematical modeling and computer simulations. Meanwhile, many models and simulations have been developed. The interested reader will find a very clear, useful and complete discussion of the topic in C.V.T Tamulonis PhD dissertation. Seriously, I can’t stress enough how this work is nice and complete!

For now, I just drafted a tree of the different models, as a basis for the discussion:

The underlying dragonfly wing is from Figure 162 of On Growth and Form (p.476 of the above linked edition). On the tree tips are examples (not even close to exhaustive) of implementation for each models. Bold names refer to softwares, slanted to a researcher and it’s publications.

Continuous models deal with phenomena that span multiple cells in size, such that the changes in shape can be smoothed out, the detailed cell-cell interactions are not necessary to understand the tissue’s shape. Star among this huge family are organ simulations, among which the heart. To grasp the state of the art on the matter, you should rush to see the video of a heart simulation by Shu Takagi’s group at Riken. Cell agregates, such as multi-cellular tumor spheroids, are studied and modeled as continuous tissues. The paper cited above (Sham Tilli et al, 2015) precisely sets down a formalism that includes cell biology specific components (i.e. cell population dynamics and re-arangements, more on that later) in a continuous dynamical (rheological, more precisely) framework. See Cyprien’s site for more details on that. Finally, I must mention the incredible OpenWorm project, which tackles the neurobiology of C. elegans and integrates it to a mechanical (particle based) framework.

For now let’s concentrate on Cell Based models, in which leg-joint fits.

By definition cell based models are described by a multi-agent design pattern. This is where biological tissues radically differ from any other material: cells can act by themselves and exchange information between each others, an individual’s behavior influences the overall shape of the tissue. We’ll see consequences of this for the API design. For now, let’s go further down the tree.

The next branching is between lattice based and lattice free models.

Lattice based were developed first, inheriting directly from early cybernetics research (Norman Wiener, John Von Neumann) on cellular automata. Conway’s game of life was described in 1970, and you can find a nice python implementation by Jake VanderPlas himself here. It’s not a biological tissue model, really, but it captures the essence of lattice based modeling: cells are pixels or collections of such on a fixed grid. The evolution of the system is solved by looking at interactions between each pixel and it’s neighbors. That’s what goes on in a more detailed manner in cellular Potts models and their descendant the Glazier-Graner-Hogweg (GGH) model. These models are well established, and CompuCell3D provides a very handy, optimized and scriptable software for defining and running GGH simulation in 2D and 3D geometries. Lattice based models are well suited to study phenomena such as collective migration (e.g. tumor invasion) or cell population dynamics. As partial differential equations can be solved across the grid, they can also deal with reaction-diffusion mechanisms, and thus signaling. That’s all nice and well, but the grid is also a constrain (if precise shapes are of interest), and the physics governing pixel state transition is very phenomenological, it does not really capture the mechanical aspects of the tissue.

In lattice free models, the system’s space (usually 2 or 3 dimensional) is continuous and the objects are described by their metric in that space. An early split is between models made of descrete spherical elements and the ones relying on a vector based description. In the former class, cells are described as spheres (like in Chaste) or smaller particle clouds, as in the work by P.E van Liedekerke et al. cited by Tamulonis.

The later branch divides into finite elements models (see CVT Tamulonis again) and vertex models, where leg-joint and tyssue fit (ouf! as we say in French). Thanks to the close correspondence between those model architecture and the cell boundaries, they are well adapted to the description of contiguous, one cell thick tissues as the epithelium we’re interested in.

Now that we know where we are in the grand scheme of things, let’s dig on the library’s structure. We’re at step 3 already!

Library architecture and API design.

Prolégomène: libraries vs standalone software.

Software can come in various forms. Traditionally, and as is the case for most of the works presented above, simulations would be developed in a compiled language such as C++ (the uncontested giant in the field) and distributed as standalone executables. I’m a python user, and prefer to use libraries, and develop by importing what I need when I need it. I feel that doing development in the Jupyter notebook gives me a lot of freedom to explore and hack. The user/developer frontier is blurry in the scipy community for a good reason: that’s a very efficient way of developping software. So, contrary to e.g. Chaste, tyssue is designed as a modular, hackable library.

Object architecture and design patterns

Objects to consider

To fix the ideas, let’s work on a minimal 2D example of what we try to model. Generalizing to more complex geometries is deferred to further headaches :-p.

Here is our 2D three cells epithelium:

The epithelium contains cells indexed by Greek letters, junction vertices, indexed by Latin letters, and junction edges. The blue links denote a neighborhoud relation between two cells. In 2D, the edges can be described as pairs of Halfedges (see the documentation on Halfedge Data Structures in CGAL). In 3D, this concept is generalized by the Linear cell complex structure, made of connected Darts. Both Halfedges and Darts hold information on their source, target and the cell they are associated to. So the pair of Halfedges between vertices $i$ and $j$ should be indexed as $i j, \alpha$ and $j i, \beta$. In 3D, there will be a fourth index for a given Dart, giving the associated face of the cell, see the discussion on Darts orbit and $\beta_i$ operators on CGAL’s doc.

The ensemble of those objects and their relations is called the topology of the system. Perhaps abusively, said topology also includes the set of geometrical points associated with the vertices (but nothing more, see below).

Data Structures

To the topology is associated data: geometric characteristics, parameter values and so on. We want to be able to get and set these data by single elements or through fancy indexing. Ideally, without copying it, and in a transparent way to the python user. But the above mentioned concepts are well defined and optimized in CGAL, and it would be a waste not to rely on all this good work. Yet, most of this data is irrelevant to CGAL: we could for example associate a color to a cell for representation purpose, that needs to be dynamically allocated, etc., all that in an interactive python session. graph-tool does a very good job at managing that with PropertyMaps, but we saw it was not fitting exactly our needs. Wrapping C++ and Numpy arrays is not trivial.

So here is how I see this: let the C++ side of things completely ignore the data — to the exception of the positions of the vertices in space, which CGAL need, and will receive special treatment —, and just let it manage topology. This way the only CellAttribute associated with a CGAL object is its index (plus the Point for vertices).

The CGAL/python interface is then just a matter of passing the indices (as std::vectors<int>) in read only mode to python, and updating the points back and forth. Python side, the core data structure is then comprised of 3 DataFrames (4 in 3D actually):

cell_df, indexed by the Index cell_idx
jv_df, indexed by the Index jv_idx
je_df, indexed by the MultiIndex je_idx, itself comprised of a (srce, trgt, cell) triple (e.g. ${i, j, \alpha}$). In 3D, it would be a quadruple (srce, trgt, face, cell).

Here is a sketch summarizing the above, along with the behavior and visualization aspects I’ll discuss next.

You can find a toy implementation (CGAL independent) in this notebook, along with examples of simple computation combining data from cells and junctions.

A note on time

I haven’t spoken of the time dimension yet. The above description gives a static view of the tissue. Of course, the whole goal is to look at evolutions. Time will be a global attribute of the system. For all the DataFrames, we can construct a Panel where the fourth dimension is the time component, stacking up static views of the tissue (this can also be achieved via a supplementary ‘t’ index for each dataframe). Whether this is feasible, or it’s better to record the data at each time step is to be determined. For small systems, the former will be easier, but might not scale, and some kind of buffering might be needed (to be continued, data management is not my strong suit).

External constrains and supra cellular components

Further down the road, it might be necessary to include other elements, for example the extra-cellular matrix, which by definition is not included in this framework. If its shape is simple and static enough, this could be described in terms of a force field in the space surrounding the tissue. One could also envision a mixed continuous - cell based model, where the ECM is described as a finite elements triangulated volume. It is not clear to me how to manage contact points here, I’m sure that will be fun. Apart from the ECM, one can think of trans-cellular actin cables or an egg shell constraining the epithelium.

On that matter, management of contact points and mesh collision is not trivial, but it looks like the great folk at CGAL have this sorted out for us.

Physics

The architecture described above describes the state space of the epithelium, and it’s associated parameters. We can then add physical data: forces or gradients, for example. If the specific columns to consider depend on the physics engine, the resolution of the dynamical equations (wether through gradient descent, ODEs, etc.) should be independent of the topology at one point during the simulation.

Agent-like behavior

As I said earlier, cells are not passive chunks of material, but individuals displaying different behaviors, either individually or collectively. In this sense, cells are agents. This must be reflected in the library architecture as to make it easy to translate in the simulation the biologist’s insight of the modeled biological scenario. Here are some examples:

Cell growth
Cell division
Cell intercalation (aka Type 1 transition)
Apoptosis

For each of those behaviors, one cell or a group of cells will be implicated (the actors), and some specification on the physics involved that might look like a list of actuators (I’m thinking for example at the actin apico-basal cable in the fold formation scenario) specifying the interactions and their application points. Every behavior can trigger a change in topology, requiring a re-indexing from CGAL, and sets the system off-equilibrium, which is resolved by the physics engine. Those concept are still in early development, and the API is still sketchy here.

Events, signals and asynchronicity

Associated with the multi-agent pattern comes the idea that those agents, the cells, could act asynchronously, each behavior sending signals to neighboring cells and the hole epithelium. At each time step, we can imagine to gather all the ongoing behaviors (cell 123 might divide, while cells 234, 235, 236, and 244 undergo a type 1 transition) and solve the physics system only once for the whole tissue. Once again, this is still a bit sketchy, and any comments are welcome.

Next step: data viz!

This vast subject (3D! 3D+time!, vispy!, webGL!) will wait next post.

From leg-joint to tyssue, a refactoring story - Ep. 1

2015-08-17T00:00:00+02:00

Intro

So I’m giving a talk at EuroScipy at the end of the summer (hooray!). It is based on the code I developed to model fold formation in the drosophila leg imaginal disk, which was published earlier this year. A lot as moved since I submitted the abstract. So here are some thoughts on what happened, and what motivated the switch to a new, more generic library.

Here is the submitted abstract (I’m all about code re-use)

Biological tissues, and more particularly epithelia are very particular kinds of material. Not only do they behave like solids and liquids at the same time (think shaving foam), they are also governed by the behavior of their constituent individual cells. Biological processes (a bunch of incredibly complex chemical reactions) and physics are intertwined so that complex forms emerge from initially smooth tissues.

Along advanced imaging techniques and genetic manipulation of model organism, biophysical modeling is key in understanding these shape changes, or morphogenesis. We studied the role of programmed cell death, or apoptosis in the formation of a fold in the fruit fly pupae (an intermediate stage between larva and adult). In a recently published article we demonstrated that apoptotic cells had an active role in shaping this fold (which will later become a joint in the adult fly’s leg). Cells die on a ring around the socket shaped tissue (one cell thick, and about 200 µm in diameter), they contract and pull on their neighbors, initiating changes in the tissue properties.

In this presentation, I will describe how we use python to develop a numerical model of this epithelium. The leg-joint module is based on Tiago Peixoto’s graph-tool library, and uses SciPy optimization routines to perform the gradient descent at the core of the dynamical simulation. The following topics will be discussed:

Visualization: plain matplotlib vs vispy vs Blender.

Performance: can we go from 24 hrs per simulation to less than 1? The pure python vectorization and BoostPython/CGAL routes.

Future plans: towards a biological tissue physics engine.

The code is showcased in a series of Jupyter Notebooks that can be browsed here.

The three points above deserve some developments, so I’ll do 3 posts, not in the original order, though.

Performance - this post
Future plans - not so future anymore - next post
Visualization (where I’m least advanced) - the third one

The pitfalls of research driven developments

The leg_joint code was developed while our understanding of the biology was progressing at a fast pace, as Magali’s team accessed new genetic tools and gradually improved the fluorescence microscopy images of the drosophila’s leg disk. That left little room for API design, or optimization. I went for results straight ahead, tried to document and test, though not enough, but my time was well spent in maths (that bloody gradient), biology and getting correct figures.

For the published version, getting a simulation of the full fold formation process takes about 24 hours on a single core, which is not sustainable …

Optimization

So I started refactoring once the paper was published. The performance bottleneck was quite obvious: the gradient descent code was called locally (only on a group of cells) a lot of times to mimic a global epithelium relaxation, and this code contained explicit loops over each cell of the global patch to update geometry and gradients at each optimization step. This is bad, but was easy to write. It also made whole tissue optimization depressingly slow. As a good SciPythonista (if that’s a thing), I started looking at vectorization strategies.

I started using graph-tool for its efficient management of dynamic graphs and graph drawing capacities. In this library, values attached to vertices and edges can be accessed as Numpy arrays through the get_array attribute of the PropetyMap class. But as the documentation warns:

The returned array does not own the data, which belongs to the property map. Therefore, if the graph changes, the array may become invalid and any operation on it will fail with a ValueError exception. Do not store the array if the graph is to be modified; store a copy instead.

Furthermore, you can only set the ‘true’ values of the property map for the all array at once. Said otherwise, you can’t use fancy indexing to set values of a given variable (e.g. the x coordinate) of a subset of the graph’s vertices directly, you have to modify a copy of the array and feed it back to the PropertyMap.

You can access a subset of the graph through filtering, i.e. defining a binary mask over the network. But filtering is not the same as indexing, for example you can do this with indexing:

import numpy as np
a = np.arange(4) + 2
b = a[0, 1, 3, 3, 3]
print(b)
>>> [2, 3, 5, 5, 5]

But you can’t tell a mask to repeat a value, and that was exactly what I needed to compute my epithelium geometrical properties. With PropertyMaps, you can get values from fancy indexing, but setting them back is more complicated, due to the rather convoluted way graph-tool mirrors the underlying C++ data and the property map .a attribute, that returns a numpy array. Of course graph-tool was not meant for that kind of computation, it’s focus is on graphs’ topology, not geometry or calculus.

The cell’s area is a good example for the type of computation I was trying to run. It is computed as the sums of the cell’s sub-faces areas, which are themselves half the norm of cross product of two sub-face vectors:

The area of the sub-face is $A_{\alpha ij} = || r_{\alpha i} \times r_{\alpha j} || / 2$.

Cross product works just fine with numpy 2D arrays, but to compute it, I need to repeat each vector twice for each adjacent face, sum over the cells, and put this back in the property map holding the cell area, for future use. Hence my indexing issue with graph-tool property maps. This motivated the passage to pandas DataFrames to do the geometrical computing. Fancy indexing is what pandas is made for, isn’t it?

Here is an outline of the strategy I used to gather the data from the graph’s property maps and turn them into DataFrames:

First find all the triangular faces in the graph, using graph-tool’s subgraph_isomorphism, and get the indices of the 3 vertices (two junctions and a cell).
Use this as a MultiIndex to instanciate a DataFrame holding a copy of the relevant data.
Do the maths.
Feed back the data to the graphs’ property maps.

Thanks to graph-tool, the first point is easy to achieve, and quite fast (like some seconds for a full 2000 cells simulation):

def get_faces(graph, as_array=True):
    '''
    Retrieves all the triangular subgraphs of the form
       1 -- > 2
        ^   ^
         \ /
          0
    In our context, vertex 0 always corresponds to a cell
    and vertices 1 and 2 to junction vertices

    Parameters
    ----------
    graph : a :class:`GraphTool` graph instance
    as_array: bool, optional, default `True`
      if `True`, the output of `subraph_isomorphism` is converted
      to a (N, 3) ndarray.

    Returns
    -------
    triangles:  list of gt.PropertyMaps or (N, 3) ndarray
      each line corresponds to a triplet (cell, jv0, jv1)
      where cell, jv0 and jv1 are indices of the input graph.
    '''
    tri_graph = gt.Graph()
    ## the vertices
    verts = tri_graph.add_vertex(3)
    ## edges
    tri_graph.add_edge_list([(0, 1), (0, 2), (1, 2)])
    _triangles = gt.subgraph_isomorphism(tri_graph, graph)
    if not as_array:
        return tri_graph, _triangles
    triangles = np.array([tri.a for tri in _triangles], dtype=np.int)
    return triangles

This works thanks to the definition of the graph edges, with edges from cell center to junction vertices always oriented outwards, such that the triangular pattern uniquely defines the set of faces.

The triangles array then served as a MultiIndex for a pandas DataFrame called faces. Each of the vertex index was repeated as many times as necessary, and it was then easy to pick the correct data to compute the desired crossproduct, and do sums for each cells (something like faces['sub_areas'].sum(level='cell')).

According to git log, it took me about three weeks to vectorize completely the geometry and gradient computation, but the effort was worth it, with a time gain about two orders of magnitudes (unfortunately, I didn’t document properly the successive gains in computing time), close to the 24 to 1 hours goal I bragged about in the abstract, at least on a relatively simple test case. At that point, “all” that was left was the fourth point of the list above.

Refactoring

But then… I spent the next two months (!) trying to integrate back my new faces DataFrame within the general framework. The main hurdle comes when the graph topology changes, which creates indexes mis-alignments and synchronization nightmares. Maybe it’s my fault for not doing this at the proper level, or not specifying things more clearly; alternatively, graph-tool is not that adapted to 3D geometry computations and it’s time for some new API design.

So at the beginning of May, I decided to reboot the project, and started working on tyssue. As this post is already too long, I’ll discuss this on the next one.

Un tour des différentes méthodes de modélisation

2015-08-16T00:00:00+02:00

Ce texte est un travail en cours. N’hésitez pas à commenter ci-dessous pour m’aider à l’améliorer. J’essaierai peu à peu de sortir du style télégraphique.

Comparaison des différentes classes de modèles de tissus

Basé principalement sur la revue de L.A. Davidson et al. / Journal of Biomechanics 43 (2010) 63–70

L.A Davidson et ses collègues distinguent trois grandes classes de modèles:

Modèle de Potts cellulaire
Modèles de type vertex
Modèle en éléments finis

Modèles de type vertex

C’est le modèle utilisé actuellement pour les simulations de formation du pli.

Aspects positifs:

Facile à saisir et à implémenter (peu de paramètres)
Relation très directe image $\leftrightarrow$ simulation
Adapté pour l’organisation appicale de cellules épithéliales jointives
Adapté à des topologies dynamiques

Aspects négatifs:

Limité à des cellules jointives: pas de délamination, par exemple.
Tout se passe au niveau des jonctions.
Peu adapté à la prise en compte des événements baso-latéraux.
Difficile d’inclure des interactions hors du maillage.
Physique peu détaillée.

Modèle de Potts cellulaire

Dans cette classe de modèles, l’espace est divisé en cubes de taille petite devant la taille des cellules. Chaque cube est assigné à une cellule, et on minimise une énergie sur les faces des cubes, en fonction de leur voisinage (intra-cellulaire / interface). On peut l’envisager comme un système multi-agents.

Aspects positifs:

De nombreuses implémentations existent déjà, facile à mettre en œuvre
Adapté à la modélisation de la migration cellulaire dans des approches en termes de populations de cellules.
Grande souplesse dans la modélisation des intéractions

Aspects négatifs:

Mal adapté à des géométries complexes
Mal adapté à la modélisation des interfaces
“Loin” des détails bio-physiques des intéractions cellule-cellule
Grande souplesse dans la modélisation des intéractions

Méthodes par éléments finis

C’est la discrétisation d’un système d’équations continues dans l’espace. Dans le cas de la modélisation d’une couche, apicale par exemple, la discrétisation est effectuée en découpant le tissu en faces triangulaires.

Le maillage est plus fin que dans le modèle vertex. Contrairement au modèle de Potts, il est adaptable, de manière à décrire plus finement le milieu au voisinage des points de forte variations, par example aux interfaces cellule - cellule.

Aspects positifs:

Adapté aux problèmes continus
Fidélité à la géométrie
Proximité avec la réalité biophysique du tissu

Aspects négatifs:

Gourmant en temps de calcul (et en temps d’optimisation)
La segementation n’est pas triviale (maillage régulier ou adaptatif, etc.)

Reste à aborder:

Détailler la physique des méthodes en milieu continu
Inclure des exemples pour chaque méthode
Rédiger une introduction

No comment

2015-04-04T00:00:00+02:00

import pandas as pd

n_times = 3
n_parts = 4


def meth1(n_times, n_parts):
    return pd.MultiIndex.from_tuples([(i,j) for (i, j) in np.ndindex((n_times, n_parts))],
                                      names=['t_stamp', 'label'])



def meth2(n_times, n_parts):
    time_stamps, labels = np.mgrid[:n_times, :n_parts]
    return pd.MultiIndex.from_arrays([time_stamps.flatten(),
                                      labels.flatten()],
                                      names=('t_stamp', 'label'))

def meth3(n_times, n_parts):
    time_stamps, labels = np.mgrid[:n_times, :n_parts]
    return pd.MultiIndex.from_arrays([time_stamps.ravel(),
                                      labels.ravel()],
                                      names=('t_stamp', 'label'))

%timeit meth1(1000, 10)
> 100 loops, best of 3: 10.5 ms per loop

%timeit meth2(1000, 10)
> 1000 loops, best of 3: 779 µs per loop

%timeit meth3(1000, 10)
> 1000 loops, best of 3: 763 µs per loop

Website refurbished

2015-03-03T00:00:00+01:00

Welcome to the brand new DamCB website!

I’m still toying around with themes and general organisation, but I must say it went very smoothly. I won’t commit an installation tutorial, because many have done that quite nicely already, and the doc is nice, once you’ve perused it long and large. Noob as I am on everything web, it took me only two days to get the site up with nice notebooks rendering and an effective deploy work-flow.

Jake Vanderplas migrated from octopress to pelican, and made his choice on a simple metric:

Nikola and Pelican both seem to be well-loved by their users, but I had to choose one. I went with Pelican for one simple reason: it has more GitHub forks. I’m sure this is entirely unfair to Nikola and all the contributors who have poured their energy into the project, but I had to choose one way or another.

Well I don’t know about unfairness of the method, but from someone who tested both, I think I prefer pelican… The first reason of course is that nikola was my first shot at a static, self hosted website (after an ungly-virtual-host-autogenerated-drupal install), so I’m sure I did wrong every thing I could. Now I’ve learned a trick or two, and the logic behind pelican was easier to grasp.

There are some more reason why I prefer pelican though:

It’s very markdown friendly, and though it supports reStructuredText, markdown is the prime format. Nikola is the other way round. So if you prefer markdown as I do, go for it.
The liquid-tags plugin allows nearly out of the box IPython Notebook to be included in your text. Although Damian Oquanta makes terrific themes and ease the publication of notebooks directly from the notebook itself in nikola, I couldn’t get it to work as easily as @jakevdp liquid_tags for pelican. Also, I prefer the former logic and work flow: Notebooks are stored in a specific directory and you include them in your markdown posts (possibly only selected cells), instead of being created for the purpose of publishing them, and having to deal with metadata at notebook creation time, with a pair of (.ipynb, .ipynb-meta) pair of files.

On a final note, I think I did a stupid thing with nikola: I was using nikola directly on the server to serve the html files. I was syncing the sources of the site via github from my local machine to the server, and doing the build server side. Don’t do this!

With pelican (maybe thanks in part to clearer documentation, in part to me growing a brain), I set up a dead simple nginx server and sync only the rendered html. Of course that’s the only way that makes sense, but it’s not explicitly expressed on the docs of nikola…

Well that’s it for now.

The leg joint paper is out!

2015-02-12T00:00:00+01:00

We did it! After nearly three years of work (the initial commit to the code was done on may 24th 2014), the work on the role of apoptosis in the formation of folds in epithelium is out!

I’m very proud for this work. Of course most of the credit goes to Magali Suzanne team. This is top quality biology, and it was really nice being part of the research process. The model itself relies on the marvelous scientific python ecosystem and its supporting community. Special thanks to Tiago for providing the graph-tool library.

The code is described in details in a series of Ipython Notebook that you can read here.

Now for a brief summary on what we did:

[edit 01/23] there’s a nice news and views here where Claudia Vasquez and Adam Martin review the biology and rise interesting questions.

Apoptotic cells last stand - and its consequences for morphogenesis

We showed that, far from being passively eliminated, apoptotic cells do actively influence their environment by increasing the surrounding tissue tension. Indeed, before they die, apopotic cells exert a force that transiently deform the apical surface of the epithelium. This force is then transmitted to the neighbouring cells through an increase in tension which will in turn provoke a change in tissue shape.

Apoptosis is known for its role in morphogenesis, and more specifically in the formation of folds in various developmental contexts; yet the molecular mechanisms implied in those processes remain largely unknown. The formation of folds within an epithelium allows to pass from a bidimentional to a tridimentional tissue and is thus a key step in morphogenesis. Here we showed a new mechanism of fold formation in the Drosophila leg disk. This mechanism not only implies the elimination of apoptotic cells from the tissue, but also their active participation to the tissue remodelling. Indeed, each apoptotic cells within the epithelium generates before it dies a force relying on the establishment of an apico-basal acto-myosin cable. This previously unknown cable drives a transitory deformation of the apical surface of the epithelium, which in turn drives a stabilization of myosin II in the neighbouring cells at the adherent junctions level, as well as an increase in tissue tension. We also showed that the synergistic contribution of several apoptotic cells was necessary to create a myosin II stabilization in the whole neighbouring tissue, a global increase in tension, cells apical constriction and eventually the fold formation.

Finally, in order to test whether those apoptotic forces are indeed the initial signal responsible for the change in tissue shape, we devised a 3D model of the leg disk epithelium, based on the pre-existing vertex model published by Farhadifar et al. in 2007. In this bio-mechanical model, we were able to show that apoptotic forces (the apical-basal force, followed by the apical propagation) are both necessary and sufficient to drive fold formation, suggesting that this new mechanism could happen in any type of epithelium.

This work is an important step in the field of morphogenesis and brings up a new dogma on the active role of apoptosis in apoptotic cells.

Bellow is a movie of the formation of the leg joint in vivo (that’s confocal microscopy, apoptotic cells are marked red:

Apical vue of the fold formation on a drosophila leg disk from glyg on Vimeo.

And a movie of the simulated tissue undergoing fold formation:

Fold formation model from glyg on Vimeo.

Of course my work with Magali’s team continues, and there’s a lot more to investigate: the precise mechanism by which the apical-basal force is translated in an apical myosin accumulation, the role of cell polarity in the process, or the role of the peripodial membrane in shaping the tissue, for example. Exciting times!

Teaser

2014-12-12T00:00:00+01:00

Fold formation model from glyg on Vimeo.

Pattes de mouches

2014-11-29T00:00:00+01:00

In [1]:

import logging
logging.disable(logging.ERROR)

In [2]:

import leg_joint as lj
import matplotlib.pyplot as plt
%matplotlib inline
from IPython import display
import graph_tool.all as gt
import numpy as np

Pattes de mouches: une modélisation de biophysique utilisant python scientifique¶

Guillaume Gay

DamCB - Data analysis and modeling for Cell Biology

e-mail: [email protected]">[email protected]

github / freenode: glyg

twitter: @damcellbio / @elagachado

On travaille sur la mouche du vinaigre, drosophila Melanogaster, un organisme modèle très utilisée en génétique et en biologie du développement. C’est une petite mouche, qui fait quelques milimètres de long.

source: wikipedia

On s’intéresse plus précisément au passage du stade pupal au stade adulte. Des groupes de cellules bien déterminés (les disques) dans la pupe vont se transformer en organes chez l’adulte. On regarde le disque de patte, qui donnera … la patte de la mouche.

On regarde la formation du petit pli entre les segments 4 et 5. Le disque de patte est un épithélium une monocouche de cellules, qui forme une espèce de chaussette.

Vue de très près ça ressemble à ça:

On observe ici les jonctions apicales, un ensemble de protéines qui forment un maillage entourant le haut des cellules.

Le problème biologique: comment la mouche plie ses pattes?¶

L’apoptose, ou les cellules kamikases¶

Mort programmée via une cascade biochimique.
Nombreuses fonctions:
- comme la prévention du cancer
- et la morphogénèse

Un exemple de phénotype quand il y a un défaut d’apoptose

Dans le tissu qui donnera la patte¶

Pour former un pli, ~ 30 apoptoses en anneau autour de l’épithélium

Apical vue of the fold formation on a drosophila leg disk from glyg on Vimeo.

Et lorsqu’on suprime l’apoptose…

Regardons ce qu’il se passe à l’échelle cellulaire (on regarde une coupe des cellules, la région apicale est en haut, la région basale en bas.

En mourant, une cellule exèrce une traction vers le bas, et on voit aparaître une structure verticale de myosine, une protéine capable d’exercer une force (c’est le même type de protéine qui propulse nos muscles).

On est donc en présence d’un problème de forces et d’interactions mécaniques. C’est là que la modélisation peut intervenir, pour tester cet effet mécanique dans un modèle simplifié.

Le problème biophysique¶

Les déformations du tissus sont dues à l’activité d’un ensemble de protéines intra et extra cellulaires.

Ces protéines forment un maillage entre les cellules, que l’on peut modéliser comme un réseau

Minimiser localement l’énergie à chaque modification
Équilibre des forces

Le modèle est donc basé sur la bibliothèque graph_tool écrite et maintenue par Tiago P. Peixoto, permettant de manipuler des graphes.

Voici un exemple rapide de cette bibliothèque

In [3]:

g = gt.price_network(3000)
pos = gt.sfdp_layout(g)

pos = gt.graph_draw(g, pos=pos, output="graph-draw-sfdp.png")

Les avantages de graph-tool:¶

C’est rapide (C++ / meta-programming, openMP)
PropertyMaps interfacées aux ndarray de NumPy.
Bonnes E/S
Mécanisme de filtrage (via des PropertyMaps booléennes appliquées sur le graphe)
Bibliothèque complète d’analyse de graphes.

Le module `leg_joint`¶

Objet Epithelium:
- graphe orienté, masques
- géométrie
- méthodes de base
Fonctions:
- Division cellulaire, apoptose …
- Optimisations
- Représentation graphiques

In [4]:

### Create an epithelium by instanciating the container class

eptm = lj.Epithelium(lj.data.before_apoptosis_xml(),
                     save_dir='.',
                     identifier='slides',
                     copy=True)

### Scale the tissue globaly to approach equilibrium
eptm.isotropic_relax()

Un exemple: la division cellulaire¶

In [5]:

### Select cell a vertex

mother_cell = eptm.graph.vertex(913)
print('The vertex {} is a cell: {}'.format(
      mother_cell, bool(eptm.is_cell_vert[mother_cell])))


### Local mask is used to work only on part of the epithelium
eptm.set_local_mask(None)
eptm.set_local_mask(mother_cell, wider=True)
fig, axes=plt.subplots(1, 2, figsize=(9, 4), sharey=True)
axes = lj.plot_2pannels(eptm,
                        cell_kwargs={'c_text':True},
                        edge_kwargs={'c':'g', 'lw':2, 'alpha':0.4},
                        axes=axes)

The vertex 913 is a cell: True

/home/guillaume/anaconda/envs/python3/lib/python3.4/site-packages/matplotlib-1.4.x-py3.4-linux-x86_64.egg/matplotlib/figure.py:1644: UserWarning: This figure includes Axes that are not compatible with tight_layout, so its results might be incorrect.
  warnings.warn("This figure includes Axes that are not "

In [6]:

septum = lj.cell_division(eptm, mother_cell, verbose=False)

### Gradient descent energy minimization (`fmin_lbfgs_b` is used)
pos0, pos1 = lj.find_energy_min(eptm)
fig, axes=plt.subplots(1, 2, figsize=(9, 4), sharey=True)
axes = lj.plot_2pannels(eptm,
                        cell_kwargs={'c_text':False},
                        edge_kwargs={'c':'g', 'lw':2, 'alpha':0.4},
                        axes=axes)

In [7]:

## Let's define a neat plotting function
def show_apopto_surroundings(eptm, a_cell, axes=None):
    eptm.set_local_mask(None)
    eptm.set_local_mask(a_cell, wider=True)
    is_apoptotic = eptm.ixs.copy()
    is_apoptotic.a[:] = 0.
    is_apoptotic[a_cell] = 1.
    ax_zs, ax_xy = lj.plot_2pannels(eptm, axes=axes,
                                    edge_kwargs={'c':'k', 'lw':0.5, 'alpha':0.5},
                                    cell_kwargs={'cell_colors':is_apoptotic,
                                                 'cmap':'Reds',
                                                 'alpha':0.8})

    eptm.set_local_mask(None)
    return ax_zs, ax_xy

On choisit une cellule apoptotique

In [8]:

fig, axes = plt.subplots(1, 2, figsize=(8, 6))

a_cell = mother_cell
eptm.set_local_mask(None)
eptm.set_local_mask(a_cell, wider=True)

ax_zs, ax_xy = show_apopto_surroundings(eptm, a_cell, axes=axes)

À 10 reprises, on diminue son volume d’équilibre, et on tire dessus avec une force radiale croissante

In [9]:

fig, axes = plt.subplots(1, 2, figsize=(8, 6))

for i in range(10):
    lj.apoptosis_step(eptm, a_cell,
                      vol_reduction=0.7, ## Reduction of equilibrium volume
                      radial_tension=0.1, ## Increase in radial force
                      contractility=1.2, ## Increase in contractility for the apoptotic cell
                      )

ax_zs, ax_xy = show_apopto_surroundings(eptm, a_cell, axes)

Lorsqu’on fait ça 30 fois, voilà ce qu’il se passe:

Fold formation model from glyg on Vimeo.

Pas de force $\Leftrightarrow$ pas de pli:

on confirme l’hypothèse de la nécessité d’un rôle actif de l’apoptose dans la morphogénèse

Reste à faire¶

Tests unitaires & intégration continue
Rendu dynamique 3D avec Blender (en cours)
Gros réusinage du code:
- description basée sur des évennements
- optimiser (y’a du boulot…)

Et ensuite¶

Généraliser la géométrie, surface basale
Plateforme de modélisation de tissus “multi-physique”

Les biologistes du LBCMCP (CNRS/Université de Toulouse)¶

Mélanie Gettings
Bruno Monier
Sonia Shott
Magali Suzanne

L’autre physicien¶

Thomas Mangeat

Merci!¶

PyConFr - un rapport

2014-06-10T00:00:00+02:00

Je reviens tout juste de PyConFr 2014, qui se déroulait ce week end (les sprints sont toujours en court). C’était chouette, et je voudrais fixer un ou deux trucs ici - surtout pour avoir une trace pour plus tard.

Ma propre présentation c’est bien passée, je pense. J’espère avoir été à peu près clair, après j’aurai peut-être dû être plus technique, mais c’est difficile sans expliquer la biologie avant.

D’abord merci beaucoup à Xavier qui a nous a conduit de Marseille à Lyon et retour tout en souplesse, et à son copilote Victor…

C’était la première fois que j’assistais à cette conf, et j’ai trouvé des intervenants de qualité et un audience attentive. C’est aussi la première fois qu’il y avait une programation dédiée à l’enseignement et la recherche, auquel j’ai eu l’honneur de participer et que j’ai suivi toute la journée de samedi.

Voici quelques unes des choses que j’ai retenues:

Sage

Deux conférences parlaient de Sage qui est utilisé par les mathématiciens, et paraît être le standard de facto pour un certain nombre de sous-domaine des mathématiques. La première conférence, de Thierry Dumont dressait un parallèle intéressant entre programmation orientée objet et théorie des ensemble (une structure d’anneau hérite des propriétés d’un groupe, par exemple) et présentait le notebook Sage, prédécesseur de ipython Notebook. La deuxième, par Viviane Pons, présentait l’application de cet outil à l’étude de la combinatoire. J’ai trouvé cette intervention très vivante, et j’aime quand les matheux partagent leur recherche, ça casse un peu cette impression d’ésotérisme complet de leur boulot.

Apparemment il y a la possiblité d’une adoption du notebook Ipython par la communauté Sage, même si le créateur William Stein a plutôt l’air de tendre vers un modèle tout cloud, et un développement spécifique. Pour ma part je trouverai ça dommage (ça me fait un peu penser à Mir/Wayland, avec l’impression que c’est l’égo du créateur qui prime sur l’efficacité). L’avantage du libre, c’est que personne n’est obligé de le suivre dans ses choix…

Marc-André Delsuc et l’enseignement à des biologistes

Bien sûr la problématique m’intéresse tout particulièrement, et en plus M.A.D. a lancé une boîte - casc4de - qui fait du python scientifique pour la biophysique. On a eu quelques brèves discussions, mais j’espère qu’on trouvera d’autres occasions d’interagir. Sinon sur la conf elle même, un point que j’ai trouvé très marrant:

Marc-André explique qu’il utilise le notebook pour ses cours de programmation aux biologistes. Bien sûr, ce cours fait peur (physique+maths+info pour des biologistes ça fait beaucoup). Pour ces étudiant.e.s, un effet inatendu pour moi de l’utilisation d’une interface de programmation web est que ça dédramatise complètement l’usage, parce qu’il est naturel pour eux de rentrer du texte dans des petites cases dans le navigateur, alors qu’ils sont complètement bloqués devant un terminal ou un éditeur de texte en monospace, un effet très innatendu pour moi.

Un survol des autres interventions du samedi

J’ai bien aimé la présentation de Yannick Chopin, qui montre notemment que Python et le stack scipy est le standard en astrophysique, avec astropy comme lieu commun.

Dans Du Python qui ne manque pas d’air Romaric David a présenté l’utilisation de python comme glue pour configurer freefem++ à partir d’une spécification en fichier texte d’une sale de serveur, pour calculer les flux d’air et éviter la surchauffe. Je pense que OpenSCAD pourrait l’aider à dessiner la salle de manière plus simple…

Nicolas Rougier m’a convaincu d’essayer Vispy, surtout pour la visualisation des simulations d’épithélium, qui sont impossible à afficher dynamiquement avec matplotlib. Blender c’est bien, mais le développement est vraiment trop pénible (redémarer Blender à chaque bug par exemple). Je suis aussi impatient de voir ce que ce projet donnera avec WebGL.

Une autre bibliothèque que je veux intégrer dans mes projets prochainement est sympy, présentée par Kamel Ibn Aziz Derouiche, et qui est beaucoup plus riche, notemment pour des aspects de mécanique, que ce que je pensais.

Les autres conférences me concernaient moins, ou parlaient de choses que je connaissais déjà, je vous laisse aller voir le programme…

C’est tout pour l’aspect purement python scientifique, les confs du dimanche portaient sur des aspects plus généraux du langage.

Dimanche

J’ai suivi la présentation des APIs hypermedia par Olivier Hervieu. Ça a l’air d’être compliqué de mettre tout le monde d’accord pour avoir des API web robustes et où tout le monde parle le même sabir. Si j’ai bien compris, l’idée est d’éviter d’avoir à coder des URLs en dur dans l’API, mais plutôt d’avoir une couche de description plus abstraite, qui ne dépend pas des détails de la structure du site… J’ai aussi suivi les deux présentations sur AsyncIO, ça a l’air d’être tout à fait passionant, et big upà Victor Stinner pour sa présentation hyper pédagogique de cet aspect. Je ne suis pas sûr d’avoir besoin de ce type de mécanisme, mais c’est toujours bien de se former un peu. Pour le reste, je ne me suis pas trop senti concerné par Bootstrapping Machine Learning de Louis Dorard… Je crois que scikit-learn est plus adapté à mes problématiques, et j’ai bien aimé la présentation sur Kivy, même si j’ai vraiment le développement d’interface utilisateur en horreur.

Voilà c’est à peu près tout.. J’ai ramené un t-shirt trop petit et une adhésion à l’Afpy dans mes bagages.

Mean square displacement

2014-06-06T12:20:00+02:00

In [1]:

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline

A short tutorial on the mean square displacement¶

The raw mathematical definition¶

The mean square displacement for a time difference $\Delta t$ is computed as the squared distance between the position of the particle at time $t$ and its position at time $t + \Delta t$ averaged over each successive time $t$:

$$ \mbox{MSD}(\Delta t) = \frac{\sum_0^{T - \Delta t} ||\mathbf{r}(t + \Delta t) - \mathbf{r}(t)||^2}{(T - \Delta t) / \delta t} = \frac{d_{t, t+\Delta t}}{(T - \Delta t) / \delta t} $$

Here the vector $\mathbf{r}(t)$ denotes the position of the particle at time $t$.

So for a given trajectory we’ll mesure the mean square displacement for a delay of 10s ($\Delta t = 10s$) by computing the distance between the particle’s position at time $0 s$ and it’s position at time $9 s$, then between $1 s$ and $10 s$, and so on, square all those distances, and take the mean of those values.

The value of the MSD for a 10 seconds delay is the mean of the squares of the distances reprensented by the arrows above

Why bother?¶

The mean square displacement is usefull as it allows to measure the diffusion coefficient of a particle in case the movement is random. When the movement is random, the MSD grows linearly with the delay, and when it’s linear, it grows like the square of the delay. We’ll see why..

Let’s create two trajectories, one random and one linear, in 2D¶

In [2]:

## Number of points in the trajectories
n_points = 1000

## Time step between two points
t_step = 1
## Scale of the random movement (standard diviation)
scale = 1.

## 
xy_random = np.random.normal(scale=scale, size=(n_points, 2)).cumsum(axis=0)
xy_random = pd.DataFrame(data=xy_random, 
                         index=pd.Index(np.arange(n_points), name='t_stamp'), columns=['x', 'y'])
xy_random['t'] = np.arange(n_points) * t_step

xy_linear = np.vstack([np.arange(n_points), np.arange(n_points)]).T
xy_linear = pd.DataFrame(data=xy_linear, 
                         index=pd.Index(np.arange(n_points), name='t_stamp'), columns=['x', 'y'])
xy_linear['t'] = np.arange(n_points) * t_step

$x$ vs $y$ plots of the two trajectories¶

In [3]:

fig, (ax_rnd, ax_lin) = plt.subplots(1, 2, figsize=(12,6))

ax_rnd.plot(xy_random.x, xy_random.y, '-r+', alpha=0.7)
ax_rnd.set_aspect('equal')
ax_rnd.set_title('Random movement')

ax_lin.plot(xy_linear.x, xy_linear.y, '-k', alpha=0.7)
ax_lin.set_aspect('equal')

ax_lin.set_title('Linear motion');

Point to point distance for a given delay¶

In [4]:

delay = 4

#the number of data points corresponding to the delay
shift = np.int(delay / t_step)

### The idea here is just to shif the data and take the difference
pos_diff_rnd = xy_random - xy_random.shift(-shift)
pos_diff_lin = xy_linear - xy_linear.shift(-shift)

### The square displacement is the sum of the squares of each coordinates
sq_dist_rnd = np.square(pos_diff_rnd[['x', 'y']]).sum(axis=1, skipna=False)
sq_dist_lin = np.square(pos_diff_lin[['x', 'y']]).sum(axis=1, skipna=False)


### Print the last points of this squared distance
print('Square distance between points {} s appart (last {} points):'.format(delay, shift+2))
print(sq_dist_rnd.tail(shift+2))

Square distance between points 4 s appart (last 6 points):
t_stamp
994    0.618883
995    2.937484
996         NaN
997         NaN
998         NaN
999         NaN
dtype: float64

Note that the difference is not defined for the last points, because there is no point to measure a distance with. This is an important aspect to keep in mind with MSD computations:

When the delay gets bigger, there is less and less data to compute the MSD and do the average, so the mean square displacement for big delays is meaningless for single trajectories.

In [5]:

## MSD for the given delay:
msd_rnd = sq_dist_rnd.dropna().mean()
msd_lin = sq_dist_lin.dropna().mean()

print('Mean square displacements for delay {}:'.format(delay))
print('\t Random motion: {0:.3f}'.format(msd_rnd))
print('\t Linear motion: {0:.3f}'.format(msd_lin))

Mean square displacements for delay 4:
	 Random motion: 8.133
	 Linear motion: 32.000

In [6]:

### Now let's code a function that will do this for all the possible delays:

def compute_msd(trajectory, t_step, coords=['x', 'y']):
    
    delays = trajectory.t.copy()
    shifts = np.floor(delays/t_step).astype(np.int)
    msds = np.zeros(shifts.size)
    for i, shift in enumerate(shifts):
        diffs = trajectory[coords] - trajectory[coords].shift(-shift)
        sqdist = np.square(diffs).sum(axis=1, skipna=False)
        msds[i] = sqdist.mean()
    return delays, msds

In [7]:

delays, msds_rnd = compute_msd(xy_random, t_step=t_step)
delays, msds_lin = compute_msd(xy_linear, t_step=t_step)

Plot of the mean square displacement as a function of the delay¶

In [8]:

fig, (ax_rnd, ax_lin) = plt.subplots(1, 2, figsize=(12, 4))
ax_rnd.plot(delays, msds_rnd, '-r')
ax_rnd.set_title("Random movement")
ax_rnd.set_xlabel('Delay (s)')
ax_rnd.set_ylabel('MSD (µm²)')

ax_lin.plot(delays, msds_lin, '-k')
ax_lin.set_title("Linear movement")
ax_lin.set_xlabel('Delay (s)')
ax_lin.set_ylabel('MSD (µm²)')
fig.tight_layout()

What we see on the graphs above is that indead - at the begining - the MSD for the random motion varies lineary with the delay, while the linear motion’s MSD behaves like an hyperbole. We could have easily guessed the later behavior:

A linear displacement means the distance from one point to an other is proportional to the delay between those points: $d_{t, t+\Delta t} = a \Delta t$. As this is independant of $t$, the average will be simply $a \Delta t$, and the average of the square, the MDS, $a^2 \Delta t^2$

Relation with the diffusion coefficient¶

TODO…

In [ ]:

Setting up my python working environment

2014-05-15T00:00:00+02:00

Updated !! Hey there… Here is the procedure I used to have a nice scientific python stack, fitting my needs anyways, starting with a fresh linux Mint 17 install. Please note that things got a lot simpler with the latest anaconda releases.

The basics

ssh and git

sudo apt-get install openssh-client openssh-server
sudo apt-get install git git-el git-gui gitk

Those are always useful

I fail to understand why this is not in all decent distribution:

sudo apt-get install imagemagick inkscape

Although I use less and less latex, here it is anyway

sudo apt-get install texlive-common texlive-lang-french texlive-lang-english\
    texlive-lang-spanish texlive-doc-en texlive-fonts-extra texlive-xetex texlive-extra-utils

emacs (yeah sublime text is nifty, but it’s not free)

sudo apt-get install emacs emacs-goodies.el emacsen-common
sudo apt-get install pylint pyflakes python-nose # code checkers

jed is a light text-mode only emacs

sudo apt-get install jed

The emacs for python repo provides all you need to have nice python features in emacs, such as syntax checks, completion etc.

cd .emacs.d
git clone git://github.com/gabrielelanaro/emacs-for-python.git

Also add this to .emacs (changing USERNAME to your user name…):

(load-file "/home/USERNAME/.emacs.d/emacs-for-python/epy-init.el")
(epy-setup-checker "pyflakes %f")
(epy-setup-ipython)
(require 'highlight-indentation)
(add-hook 'python-mode-hook 'highlight-indentation)

;;; Bellow is a header with what's need for python 2.7 compatibility
(defun tv-insert-python-header ()
  "insert python header at point"
  (interactive)
  (insert "# -*- coding: utf-8 -*-\n\n"
      "from __future__ import unicode_literals\n"
      "from __future__ import division\n"
      "from __future__ import absolute_import\n"
      "from __future__ import print_function\n"))

(global-set-key (kbd "C-c e p") 'tv-insert-python-header)

Compilers

sudo apt-get install build-essential gfortran gfortran-multilib

Using Python 3

Since I first wrote this post, I started using anaconda for my basic installation. It really eases the management and creation of virtual environments, updates and package management. In a recent update, you can have a ‘native’ python 3.4 install.

After downloading anaconda from the above link (they ask for your email but that doesn’t translate into flows of spam), just run the installer :

cd directory/where/youdownloaded/thefile
### you might need to chmod the script
chmod +x Anaconda-2.0.1-Linux-x86_64.sh
./Anaconda-2.0.1-Linux-x86_64.sh
### update your bashrc to take into account the new paths
source ~/.bashrc

Creating a Python 3 virtual environment

Easy as pie:

conda create -n python3 python=3 anaconda

Now you can use this new environment by tiping:

source activate python3

Your terminal prompt should now be prepended with a (python3) string

Note that python3 here is just a name, you can use anything you want. Also some advocate the use of one virtual environment for each project… I’m not very fond of this strategy.

If you need recent packages, that might not be included in the conda distribution, you can use pip from whithin the virtual environment, as so:

pip install --upgrade scikit-image

We use Christoph Gohlke’s fabulous tifffile.py to parse tifffiles.

wget http://www.lfd.uci.edu/~gohlke/code/tifffile.py
mv tifffile.py ~/python3/lib/python3.3/site-packages/

The whole procedure is way easier than it use to be in the old days. Most of the time they ship the latest stable of the packages. Furthermore, it is common practice in the exosystem to test ones package through travis continuous integration by installing MiniConda and the required packages… So you end up with a vetted set of libraries.

Using Python 2

Creating a Python 2 virtual environment

You guessed it:

conda create -n python2 python=2.7 anaconda

I think if you just say python=2 it will install version 2.6, which you only want if you have to develop with it for legacy reasons (but even debian stable ships 2.7, so that should be a rare occurence by now).

Graph tool

What is graph-tool?

From graph-tool website:

Graph-tool is an efficient Python module for manipulation and statistical analysis of graphs (a.k.a. networks). Contrary to most other python modules with similar functionality, the core data structures and algorithms are implemented in C++, making extensive use of template metaprogramming, based heavily on the Boost Graph Library. This confers it a level of performance which is comparable (both in memory usage and computation time) to that of a pure C/C++ library.

Add this to /etc/apt/sources.list:

deb http://downloads.skewed.de/apt/raring raring universe
deb-src http://downloads.skewed.de/apt/raring raring universe

Replace raring above with your distribution (or the parent Ubuntu/Debian distro). Then download the public key from the website and add it with apt-key as instructed in graph-tool’s website.

apt-key add <key filename>
sudo apt-get update
sudo apt-get install python3-graph-tool python-graph-tool

link the packages to the respective environments

As graph-tool is not included (yet?) in conda, and the compilation is, well, complicated, we have to link the library to our packages:

ln -s /usr/lib/python3/dist-packages/graph_tool /home/USER/anaconda/envs/python3/lib/python3.4/site-packages/graph_tool
ln -s /usr/lib/python2.7/dist-packages/graph_tool /home/USER/anaconda/envs/python2/lib/python2.7/site-packages/graph_tool

Some more stuffs

I added this in my .bashrc for git clarity (thanks to Hadrien Mary aka @hadim):

## git status in prompt
export PS1='\[\033[01;32m\]\h\[\033[01;34m\] \w\[\033[01;33m\]$(__git_ps1)\[\033[01;34m\] \$\[\033[00m\] '
# export PS1='\[\033[01;32m\]\u@\h\[\033[01;34m\] \w\[\033[01;33m\]$(__git_ps1)\[\033[01;34m\] \$\[\033[00m\] '
export GIT_PS1_SHOWDIRTYSTATE=1

Segmenting nuclei with skimage

2014-05-05T00:00:00+02:00

In this post, we’ll see a way to segment nuclei in a confocal microscopy image stack using skimage

First import all we need¶

As I go on with the analysis, I usually add in the first cell all the import statements, rather than having them spread across the notebook. Once development is done, this will go into a script, and those import will already be sorted out.

I’m using the tifffile plugin (see http://www.lfd.uci.edu/~gohlke/code/tifffile.py.html) for the I/O.

In [3]:

import skimage.io as io
io.use_plugin('tifffile')

from skimage.filter import threshold_otsu, threshold_adaptive, rank
from skimage.morphology import label
from skimage.measure import regionprops
from skimage.feature import peak_local_max
from scipy import ndimage
from skimage.morphology import disk, watershed
import pandas as pd

from mpl_toolkits.axes_grid1 import make_axes_locatable
from scipy.spatial import distance as dist
import scipy.cluster.hierarchy as hier

Load the image¶

Also, we set the pixel sizes for later. In a wonderfull world, this will be a OME XML file and we would read the metadata carrefully collected by the biologist who acquired the image directely from the file. Of course this is rarely the case.

In [5]:

image_stack = io.imread('../files/test_nuclei_stack.tif')
z_size, x_size, y_size = image_stack.shape

z_scale = 1.5 # µm per plane
xy_scale = 0.71 # µm per pixel

Look at the images¶

In [7]:

nrows = np.int(np.ceil(np.sqrt(z_size)))
ncols = np.int(z_size // nrows + 1)

fig, axes = plt.subplots(nrows, ncols, figsize=(3*ncols, 3*nrows))
for n in range(z_size):
    i = n // ncols
    j = n % ncols
    axes[i, j].imshow(image_stack[n, ...],
                      interpolation='nearest', cmap='gray')
    
## Remove empty plots 
for ax in axes.ravel():
    if not(len(ax.images)):
        fig.delaxes(ax)
fig.tight_layout()

Any analysis should start with a look at the histogram¶

In [8]:

fig, ax = plt.subplots(figsize=(8, 4))

ax.hist(image_stack.flatten(), log=True,
        bins=4096, range=(0, 4096))

_ = ax.set_title('Min value: %i \n'
                 'Max value: %i \n'
                 'Image shape: %s \n'
                 % (image_stack.min(),
                    image_stack.max(),
                    image_stack.shape))

Some brief comments on the histogram above. We see that the full 12 bits dynamic range is occupied, which is good. Yet, the much higher peak at the maximum value (4095) tells us some of the signal is saturated, which is not so good, we’ll see why later on. Lastly, the background doesn’t look like the expected background noise for fluorescence imagery, which should follow a Poisson distribution: $P(I) = I / \sigma_I^2 \exp{-I^2/\sigma_I^2} dI$, like the left graph bellow. What we see in the above is the result of an automated background correction, like in the roght graph bellow. So we don’t really deal with raw images…

In [11]:

intensities = np.arange(0, 4096)
background = 1000
dI = 1
proba = (intensities/background**2) * np.exp( - intensities**2 / background**2) * dI
proba /= proba.sum()
n_pixels = 512*512
proba *= n_pixels
proba = proba.astype(np.int)

fig, (ax0, ax1) = plt.subplots(1, 2, figsize=(9, 4))
ax0.step(intensities, proba, 'k')
ax0.set_title('Full histogram')
ax0.set_ylabel('Bin count')
ax0.set_xlabel('Intensity')

ax1.step(intensities[background:], proba[background:], 'k')
ax1.set_title('Background corrected histogram')
ax1.set_xlabel('Intensity')


fig.tight_layout()

Detection parameters¶

We usually need some settings, derived from what we now of the objects we are trying to segment. We’ll also normaly do some kind of smoothing, which will be specified as the pixel size of the filter kernel. So here we go:

In [12]:

smooth_size = 5 # pixels
min_radius = 4
max_radius = 20

Overview of the segmentation strategy:¶

In a nutt shell the idea is to (1) threshold, then (2) label the regions above threshold, then (3) segment the labeled regions. We’ll do this on each image plane. After that, we’ll cluster the resulting objects across the $z$ axis

Per plane segmentation¶

Of course this is always the difficult part. Here we use the threshold_otsu function from skimage. We compute an overall threshold over a maximum intensitiy projection along the $z$ axis, and use this threshold to label each stack individually. To ease the segmentation, we filter the image by first smoothing with a median filter and then a local contrast enhancement.

In [13]:

## Computing threshold on the maximum intensity projection with `threshold_otsu`
max_int_proj = image_stack.max(axis=0)
thresh_global = threshold_otsu(max_int_proj)

smoothed_stack = np.zeros_like(image_stack)
labeled_stack = smoothed_stack.copy()
## Labeling for each z plane:
for z, frame in enumerate(image_stack):
    smoothed = rank.median(frame, disk(smooth_size))
    smoothed = rank.enhance_contrast(smoothed, disk(smooth_size))
    smoothed_stack[z] = smoothed
    im_max = smoothed.max()
    thresh = thresh_global
    # thresh = threshold_otsu(smoothed)
    if im_max < thresh_global:
        labeled_stack[z] = np.zeros(smoothed.shape, dtype=np.int32)
    else:
        binary = smoothed > thresh
        #binary = threshold_adaptive(smoothed, block_size=smooth_size)
        
        distance = ndimage.distance_transform_edt(binary)
        local_maxi = peak_local_max(distance, min_distance=2*min_radius,
                                    indices=False, labels=smoothed)
        markers = ndimage.label(local_maxi)[0]
        labeled_stack[z] = watershed(-distance, markers, mask=binary)

/home/guillaume/python3/lib/python3.3/site-packages/scikit_image-0.10dev-py3.3-linux-x86_64.egg/skimage/filter/rank/generic.py:63: UserWarning: Bitdepth of 11 may result in bad rank filter performance due to large number of bins.
  "performance due to large number of bins." % bitdepth)

Result of the segmentation¶

In [14]:

fig, axes = plt.subplots(nrows, ncols*2, figsize=(3*ncols, 1.5*nrows))

for z in range(z_size):
    i = z // ncols
    j = z % ncols * 2
    axes[i, j].imshow(smoothed_stack[z, ...], interpolation='nearest', cmap='gray')
    axes[i, j+1].imshow(labeled_stack[z, ...], interpolation='nearest', cmap='Dark2')
    
    axes[i, j].set_xticks([])
    axes[i, j].set_yticks([])
    axes[i, j+1].set_xticks([])
    axes[i, j+1].set_yticks([])

## Remove empty plots 
for ax in axes.ravel():
    if not(len(ax.images)):
        fig.delaxes(ax)

        
fig.tight_layout()

a closer look:

In [15]:

fig, axes = plt.subplots(1, 2, figsize=(8, 16))

z = z_size // 2
axes[0].imshow(smoothed_stack[z, ...], interpolation='nearest', cmap='gray')
axes[1].imshow(labeled_stack[z, ...], interpolation='nearest', cmap='Dark2')

Out[15]:

<matplotlib.image.AxesImage at 0x7fd55d9b2110>

Computing the properties of the labeled regions:¶

We use skimage.measure handy function regionprops. For convinience, we store the computed properties in a pandas DataFrame object (this is particullarly usefulll if you have more images or more timepoints, and want to later on manipulate the collected data.

In [20]:

properties = []
columns = ('x', 'y', 'z', 'I', 'w')
indices = []
for z, frame in enumerate(labeled_stack):
    f_prop = regionprops(frame.astype(np.int),
                         intensity_image=image_stack[z])
    for d in f_prop:
        radius = (d.area / np.pi)**0.5
        if (min_radius  < radius < max_radius):
            properties.append([d.weighted_centroid[0],
                               d.weighted_centroid[1],
                               z, d.mean_intensity * d.area,
                               radius])
            indices.append(d.label)
if not len(indices):
    all_props = pd.DataFrame([], index=[])
indices = pd.Index(indices, name='label')
properties = pd.DataFrame(properties, index=indices, columns=columns)
properties['I'] /= properties['I'].max()

Here is what we collected:

In [21]:

properties.head()

Out[21]:

	x	y	z	I	w
label
6	71.249178	40.507531	5	0.394966	5.808687
1	70.186552	39.377994	6	0.854559	8.156394
2	82.987933	50.992160	6	0.171050	4.583498
1	70.022599	38.560650	7	0.990137	8.758578
3	83.256457	50.622169	7	0.316493	5.262410

5 rows × 5 columns

Let’s look at what we collected, plotted over the segmented image

In [22]:

fig, axes = plt.subplots(nrows, ncols, figsize=(3*ncols, 3*nrows))

for z in range(z_size):
    plane_props = properties[properties['z'] == z]
    if not(plane_props.shape[0]) :
        continue
    i = z // ncols
    j = z % ncols
    axes[i, j].imshow(labeled_stack[z, ...],
                      interpolation='nearest', cmap='Dark2')
    axes[i, j].set_xticks([])
    axes[i, j].set_yticks([])
    x_lim = axes[i, j].get_xlim()
    y_lim = axes[i, j].get_ylim()    
    
    
    axes[i, j].scatter(plane_props['y'], plane_props['x'],  
                       s=plane_props['I']*200, alpha=0.4)
    axes[i, j].scatter(plane_props['y'], plane_props['x'],
                       s=40, marker='+', alpha=0.4)
    axes[i, j].set_xlim(x_lim)
    axes[i, j].set_ylim(y_lim)    
 

## Remove empty plots 
for ax in axes.ravel():
    if not(len(ax.images)):
        fig.delaxes(ax)

        
fig.tight_layout()

Bellow is another way to look at the segmented nuclei positions, this time over intensity projections of the original image.

In [23]:

fig = plt.figure(figsize=(12, 12))
colors = plt.cm.jet(properties.index.astype(np.int32))

# xy projection:
ax_xy = fig.add_subplot(111)
ax_xy.imshow(image_stack.max(axis=0), cmap='gray')
ax_xy.scatter(properties['y'],
              properties['x'], c=colors, alpha=1)

divider = make_axes_locatable(ax_xy)
ax_zx = divider.append_axes("top", 2, pad=0.2, sharex=ax_xy)
ax_zx.imshow(image_stack.max(axis=1), aspect=z_scale/xy_scale, cmap='gray')
ax_zx.scatter(properties['y'],
              properties['z'], c=colors, alpha=1)
ax_yz = divider.append_axes("right", 2, pad=0.2, sharey=ax_xy)
ax_yz.imshow(image_stack.max(axis=2).T, aspect=xy_scale/z_scale, cmap='gray')
ax_yz.scatter(properties['z'],
              properties['x'], c=colors, alpha=1)
plt.draw()

Clustering¶

Now that we have detected the nuclei in each plane, we want to regroup them across $z$ so that we have only one 3D position for each of the 4 (or is it 5?) nuclei.

We do so by applying a hierarchical clustering whith scipy.cluster module of the positions in the $(x, y)$ plane. As a parameter for the clustering, we quite naturally use the maximum radius we defined earlier.

In [24]:

positions = properties[['x', 'y']].copy()

dist_mat = dist.squareform(dist.pdist(positions.values))
link_mat = hier.linkage(dist_mat)
cluster_idx = hier.fcluster(link_mat, max_radius,
                            criterion='distance')
properties['new_label'] = cluster_idx
properties.set_index('new_label', drop=True, append=False, inplace=True)
properties.index.name = 'label'
properties = properties.sort_index()

Now the detected elements are regrouped by label:

In [25]:

properties.head()

Out[25]:

	x	y	z	I	w
label
1	57.060095	52.985963	18	0.255932	5.556623
1	57.433341	52.431252	17	0.357211	5.698035
1	55.415167	54.268103	16	0.301234	5.232079
1	55.324279	54.091615	15	0.297468	5.382026
2	44.683366	49.372282	13	0.290805	4.982787

5 rows × 5 columns

Next we want to get the center of each cluster. To do so we run a weighted average of the positions for each cluster, using the measured intensity as weight.

In [26]:

def df_average(df, weights_column):
    '''Computes the average on each columns of a dataframe, weighted
    by the values of the column `weight_columns`.
    
    Parameters:
    -----------
    df: a pandas DataFrame instance
    weights_column: a string, the column name of the weights column 
    
    Returns:
    --------
    
    values: pandas DataFrame instance with the same column names as `df`
        with the weighted average value of the column
    '''
    
    values = df.copy().iloc[0]
    norm = df[weights_column].sum()
    for col in df.columns:
        try:
            v = (df[col] * df[weights_column]).sum() / norm
        except TypeError:
            v = df[col].iloc[0]
        values[col] = v
    return values

In [27]:

cell_positions = properties.groupby(level='label').apply(df_average, 'I')

So here is what we where looking for:

In [28]:

cell_positions

Out[28]:

	x	y	z	I	w
label
1	56.335143	53.412561	16.471683	0.307242	5.474775
2	44.899683	49.921097	11.522186	0.419746	5.945169
3	82.425622	50.970181	11.551490	0.364361	5.626174
4	70.343025	37.816205	8.129519	0.794513	7.884338

4 rows × 5 columns

The final result¶

Let’s plot all that

In [29]:

fig = plt.figure(figsize=(12, 12))
colors = plt.cm.jet(properties.index.astype(np.int32))

# xy projection:
ax_xy = fig.add_subplot(111)
ax_xy.imshow(image_stack.max(axis=0), cmap='gray')
ax_xy.scatter(properties['y'],
              properties['x'],
              c=colors, alpha=0.2)


ax_xy.scatter(cell_positions['y'],
              cell_positions['x'],
              c='r', s=50, alpha=1.)


divider = make_axes_locatable(ax_xy)
ax_yz = divider.append_axes("top", 2, pad=0.2, sharex=ax_xy)
ax_yz.imshow(image_stack.max(axis=1), aspect=z_scale/xy_scale, cmap='gray')
ax_yz.scatter(properties['y'],
              properties['z'],
              c=colors, alpha=0.2)

ax_yz.scatter(cell_positions['y'],
              cell_positions['z'],
              c='r', s=50, alpha=1.)


ax_zx = divider.append_axes("right", 2, pad=0.2, sharey=ax_xy)
ax_zx.imshow(image_stack.max(axis=2).T, aspect=xy_scale/z_scale, cmap='gray')
ax_zx.scatter(properties['z'],
              properties['x'],
              c=colors, alpha=0.2)

ax_zx.scatter(cell_positions['z'],
              cell_positions['x'],
              c='r', s=50, alpha=1.)

plt.draw()

A brief final comment: The detection looks fine althou we only have 4 nuclei, and it well looks like (especially in the right pannel) two nuclei where merged in the process. I don’t really see how to avoid that with this method, maybe by searching for two local intensity maxima along the $z$ axis in each cluster (I’ve tried it), but this is not very noise resitant. Maybe a random walker segmentation in 3D could avoir this, but I must say I don’t really see how to implement that strategy. One of the difficulty with a random walker here is that as the data is saturated you don’t have well defined local maxima, but instead big regions at maximum value.

Backporting python 3 code to python 2.7

2014-04-23T00:00:00+02:00

We decided to develop scikit_tracker in python 3.x, and test only against this brand of python, because all the Scipy stack is ported now, so why not.

Unfortunately, our first user (whom we don’t want to scare off) uses canopy, which is python 2.7.

So I thought I would document the porting of the code, so here we go.

A script to put a correct header on top of each file.

Adding __from__ future import ... statements on top of your file goes a long way in porting the code, making most of the new feature from python 3 available to 2.x code.

I’m sure there’s a sed one liner to do that, but I’m just more efficient with python:

Recursively find all the python file in the project’s directory

pyfiles = []
for root, subFolders, files in os.walk(base_dir):
    for f in files:
        if f.endswith('.py'):
            pyfiles.append(os.path.join(base_dir, root, f))

Check we really got files:

import numpy as np
print('''All files ok: {}'''.format(np.all([os.path.isfile(fname)
                                    for fname in pyfiles])))

Adding imports and coding on top of the python files. TODO: avoid adding this if it’s allready there…

compat_string = '''# -*- coding: utf-8 -*-

from __future__ import unicode_literals
from __future__ import division
from __future__ import absolute_import
from __future__ import print_function

'''

for fname in pyfiles:
    with open(fname, 'r+') as pyfile:
        new_f = ''.join([compat_string]+
                          [line for line in pyfile.readlines()])
    with open(fname, 'w+') as pyfile:
        pyfile.write(new_f)

We use UserDictto subclass dictionnaries, we need to modify import so I replaced this:

from collections import UserDict

by that:

if sys.version_info > (3, 0):
    from collections import UserDict
else:
    from UserDict import UserDict

Using 3to2

We don’t really want to do all the fixes provided by 3to2, as we only seek python 2.7 compatibility

Here is the list of possible fixes, and a comment on what it does when passed :

/home/user $ 3to2 -l
Checking Python version info... 2.7.5
Available transformations for the -f/--fix option:
annotations # says no file need modification
bitlength # says no file need modification
bytes # Only wants to modify tifffile.py and I know it works fine in both versions
classdecorator # says no file need modification
collections # messes with the correction above, which is version agnostic
dctsetcomp # that's dict comprehension, which was backported to python 2.7
division # says no file need modification
except # Only wants to modify tifffile.py and I know it works fine in both versions
features # says no file need modification
fullargspec # says no file need modification
funcattrs # says no file need modification
getcwd # says no file need modification
imports # says no file need modification
imports2 # says no file need modification
input # says no file need modification
int # Appends an L to all the ints (I guess because all ints are long ints in py3k)
intern # says no file need modification
itertools # changes zip to izip, but that's not interesting for us (py 2.7 ok I think)
kwargs  # says no file need modification
memoryview  # says no file need modification
metaclass  # says no file need modification
methodattrs  # says no file need modification
newstyle ### This one is usefull, adds (object) to the class definition
next # Only wants to modify tifffile.py and I know it works fine in both versions
numliterals  # says no file need modification
open ## Backported
print ## Fixed by from __future__
printfunction   # says no file need modification
raise   # says no file need modification
range ## Keep range for py3 compatibility
reduce # says no file need modification
setliteral # says no file need modification
str ## Fixed by from __future__
super #### Usefull
throw # says no file need modification
unittest # says no file need modification
unpacking # says no file need modification
with # New in 2.5

Let’s resume all that:

Usefull fixes:

newstyle, super

Fixes that don’t do anything:

annotations, bitlength, classdecorator, division, features,
fullargspec, funcattrs, getcwd, imports, imports2, input, intern, kwargs,
memoryview, metaclass, methodattrs, numliterals, printfunction, raise,
reduce, setliteral, throw, unittest, unpacking

Fixes that would modify tifffile.py only (we don’t want those):

bytes, except, next

Fixes that are corrected by from __future__ import ...:

print, str

Fixes that are unnecessary with python 2.7:

dctsetcomp, itertools (?), open, with

So now we just issue this command to port the code:

/home/user$ 3to2 -f super -f newstyle -w sktracker/ ## This is our project's directory

Cleaning

Once this is done, there’s quite a lot of work to fix, and have tests passing (by the way, tests are really great, seeing how many quirks I had to fix).

Now let’s go through the diff to see what we had to change.

This of course is here everywhere:

+# -*- coding: utf-8 -*-
+
+
+from __future__ import unicode_literals
+from __future__ import division
+from __future__ import absolute_import
+from __future__ import print_function

Pytables HDFStore files can’t be exchanged between python 2 and python 3, so we have to have two samples files for the tests

 import tempfile
 import shutil
 import pandas as pd
+import sys

 from ..io.utils import load_img_list


 def sample_h5():
     """
     """
+    if sys.version_info[0] < 3:
+        return os.path.join(data_path, "sample_py2.h5")
     return os.path.join(data_path, "sample.h5")

There I guess subprocess is inconsistent. I think this is relatively harmless

         # module such as numpy (only needed on linux)
         if os.name == 'posix':
             subprocess.call("taskset -p 0xff %d" % os.getpid(),
-                            shell=True, stdout=subprocess.DEVNULL)
+                            shell=True)#, stdout=subprocess.DEVNULL) ## Py2.7 compat

This is rather self explanatory

-from collections import UserDict
+if sys.version_info[0] > 2:
+    from collections import UserDict
+else:
+    from UserDict import UserDict
+

Old style / new style classes (there are other like that)

-class ObjectsIO():
+class ObjectsIO(object):

This one had to be called like that, not through super, maybe because UserDict is old style

     def __init__(self, metadata_dict, objectsio):
         self.objectsio = objectsio
-        super().__init__(metadata_dict)
+        UserDict.__init__(self, metadata_dict)

Now that one involved really obscure utf-8 / unicode shenaningans, plus io.StringIO not working with 2.7, while creating a temp file was not with 3…

So here is what the solution looks like:

         et = ElementTree.ElementTree(self.root)

-        f = io.StringIO()
-        et.write(f, encoding='unicode', xml_declaration=True,
-                 default_namespace=None)
-        output = f.getvalue()
+        if sys.version_info[0] < 3:
+            f = tempfile.NamedTemporaryFile()
+            et.write(f, encoding='utf-8', xml_declaration=True,
+                     default_namespace=None)
+            f.seek(0)
+            output = ''.join(f.readlines())
+        else:
+            f = io.StringIO()
+            et.write(f, encoding='unicode', xml_declaration=True,
+                     default_namespace=None)
+            output = f.getvalue()
         f.close()

I don’t realy know why, but I had to explicitely use BrownianLinkCostFunction here, and not the automated self.__class__

-        super().__init__(context={}, parameters=_parameters)
+        super(BrownianLinkCostFunction, self).__init__(context={}, parameters=_parameters)

Unicode / string mess, I didn’t really fix it up yet…

-    cost_func.check_context('test_string', str)
+    ### This fails in py2.7
+    if sys.version_info[0] > 2:
+        cost_func.check_context('test_string', str)

More encoding quirks (makes you love python 3):

     if message:
-        bar += " " + str(message)
+        bar = " ".join([bar, message])

And now all the tests are passing, which is great, have to try to get into production now. There are still some quirks to fix (essentially anytime we call str).

Morphogénie Logiciels

Offre de stage

Offre de stage: Identification des paramètres optimaux d’un modèle d’organisation multicellulaire.

Contexte

Objectif et travail attendu

Profil

Contact

Morphogénie Logiciels est née

Mu spim, a ligth-sheet microscopy smartphone

La version 0.1 du mu-spim est sortie! mu-spim v0.1 is out!

From leg-joint to tyssue, a refactoring story - Ep. 2

Previously

The road ahead

A very brief tour of the biophysical models of living tissues

Library architecture and API design.

Prolégomène: libraries vs standalone software.

Object architecture and design patterns

Objects to consider

Data Structures

A note on time

External constrains and supra cellular components

Physics

Agent-like behavior

Events, signals and asynchronicity

Next step: data viz!

From leg-joint to tyssue, a refactoring story - Ep. 1

Intro

The pitfalls of research driven developments

Optimization

Refactoring

Un tour des différentes méthodes de modélisation

Comparaison des différentes classes de modèles de tissus

Modèles de type vertex

Modèle de Potts cellulaire

Méthodes par éléments finis

No comment

Website refurbished

Welcome to the brand new DamCB website!

The leg joint paper is out!

Apoptotic cells last stand - and its consequences for morphogenesis

Teaser

Pattes de mouches

Pattes de mouches: une modélisation de biophysique utilisant python scientifique¶

Le problème biologique: comment la mouche plie ses pattes?¶

L’apoptose, ou les cellules kamikases¶

Dans le tissu qui donnera la patte¶

Le problème biophysique¶

Les avantages de graph-tool:¶

Le module leg_joint¶

Un exemple: la division cellulaire¶

Reste à faire¶

Et ensuite¶

Les biologistes du LBCMCP (CNRS/Université de Toulouse)¶

L’autre physicien¶

Merci!¶

PyConFr - un rapport

Sage

Marc-André Delsuc et l’enseignement à des biologistes

Un survol des autres interventions du samedi

Dimanche

Mean square displacement

A short tutorial on the mean square displacement¶

The raw mathematical definition¶

Why bother?¶

Let’s create two trajectories, one random and one linear, in 2D¶

$x$ vs $y$ plots of the two trajectories¶

Point to point distance for a given delay¶

Plot of the mean square displacement as a function of the delay¶

Relation with the diffusion coefficient¶

Setting up my python working environment

The basics

ssh and git

Those are always useful

emacs (yeah sublime text is nifty, but it’s not free)

Compilers

Using Python 3

Creating a Python 3 virtual environment

Using Python 2

Creating a Python 2 virtual environment

Graph tool

Le module `leg_joint`¶