Existing Tools For Generating Web Based Network Visualisation

Question

[Djie]

Hi all,

I'd like to make a web application which is able to automatically generate graphs / networks. For instance I'd like to visualize gene expression data stored in a relational database. Nodes would consist for instance of chemicals (ie chemical(s) used in the experimental condition), gene groups and genes. Edges between gene groups or between chemicals and gene groups would be weighted by correlation.

Some demands for the graph would be: - Interactivity with the graph: dragging of nodes, clicking on nodes for more information, collapsing neighbour nodes, adjusting visual style. - Support for potentially large networks (5000+ elements (nodes + edges) ) - must be run inside a browser (java applets are allowed)

Which existing tools/frameworks are suitable for above situation? And what are the pros and cons of each tool?

I'm already aware of Cytoscape Web, Seadragon plugin for Gephi (which isnt entirely web based), and jSquid. Cytoscape Web seems the best solution at the moment, although it is said performance can get quite slow on large networks.

Does anyone know any other tools I can use to built upon or implement for creating webbased graph visualisation?

[Qdjm]

Can you please clarify your requirements a bit more. Do you mean 5,000 genes or 5,000 objects (genes + interactions)? Are you willing to ask users to download a Java plugin or do you want it to run in the browser? Is this for an internal network or do you need this to run over the Internet?

[Chris Evelo]

Since Cytoscape is open source you would probably be better of looking at the existing code and trying to improve performance for your needs than creating a whole new tool.

Answer 1

[Max]

Hi,

All of the libraries that I've seen to visualise graphs, i.e. nodes and edges, slow down as more elements are added. Unfortunately, this is a limitation based on the efficiency of current browsers.

I haven't seen a library that will offer the features you're looking for out-of-the-box apart from Cytoscape Web, especially with regards to visual styles, event handling, and supported file formats. GeneMANIA (http://genemania.org) uses Cytoscape Web for its visualisation, and you can see how those features of Cytoscape Web are used.

I can give you some tips on how to use Cytoscape Web to try to squeeze as much efficiency as you can out of your app. Here are a few:

• Cache node and edge objects: When getting data from Cytoscape Web, the data is sent from Flash to Javascript. This can get expensive. Caching node and edge data saves you from having the Flash-to-JS overhead.
• Limit the speed of filters: When binding filtering to UI elements, like sliders, limit the frequency of calls to filtering.
• Don't use Cytoscape Web as a database: Avoid storing many data attributes for nodes and edges, as this will slow down loading time and data-driven methods, like filtering.
• You can use a preset layout and calculate layout on the server side---using for example, Cytoscape's force-directed layout implementation in Java.
• Avoid some expensive visual styles, such as selection glows and node background images.

If you have further questions or need help, there is an active discussion group for developers who use Cytoscape Web: https://groups.google.com/forum/#!forum/cytoscapeweb-discuss

Cheers,
Max

[Djie]

So far Cytoscape Web also seems to me the library with the most features out of the box. Thank you for the tips on using Cytoscape Web!

Answer 2

[Drio]

Use a combination of javascript, json, jquery and specially d3. Use Json to store your data and javascript to process it. Visualize the results using the d3 framework (Abstracts svg browser capabilities in a nice javascript library). You can offer interactivity via jquery or within d3 since it offers you different types of hooks for events. Browse the examples to get exited.

[Djie]

D3 also looks like a great library. Thanks for the suggestion. Do you think using SVG + javascript will have a better (faster) user experience when using large networks than cytoscape web (flash + javascript)?

Answer 3

[Iain]

Cytoscape Web might be a good option, it is written in Javascript http://cytoscapeweb.cytoscape.org/

And there is a tutorial available too, http://cytoscapeweb.cytoscape.org/tutorial

[Boboppie]

It's actually based on Flash with a Javascript API.

Answer 4

[Rich Apodaca]

It sounds like you're looking for a tool that will generate graph reprensentations client-side from source data provided from a server.

For what it's worth, one toolkit on my list to evaluate is InfoVis.

Advantages over Cytoscape appear to include many animation effects and several graph representations.

The biggest advantage over jSquid is that there's no Java dependency - meaning a smoother experience for users and access to mobile devices, if desired.

Answer 5

[Boboppie]

As @max said, web browsers limit the performance of network visualization, it has been already very slow even for a network smaller than 5k elements to be visualized in CytoWeb, see example here, just wondering what would be the bottleneck of a pure javascript based network displayer such as the example @drio gives.

Also wondering how could it bring good experience to users to play with a big network with over 5k elements, it's basically a spaghetti monster out there...

[Djie]

Good question. I'm also interested what the performance would be of a javascript based network displayer like d3 or InfoVis.

Answer 6

[Jay]

I have had a look at InfoVis and Cytoscape Web. Both have nice features, and advantages, but in the end I went for Cytoscape Web due to the rich library inherited from the Cytoscape project (I've used the Cytoscape client in anger and it is good). Shame about the Flash, hopefully someone will fix that one day :).

The slow performance with large graphs is never going to go away if you store the whole graph in RAM as node and edge objects. That method just doesn't scale.

My approach to the problem of large graphs is to cache the graph in a database and use PHP to feed Cytoscape Web local subgraphs that it retrieves from the database by queries around nodes of interest. Double-click on a node or edge recenters the graph by retrieving a new neighbourhood subgraph from the database to a given depth. This seems to be the only sensible way to be scaleable. Holding a graph as objects in memory is never going to scale well.

At the moment I have the graph stored in a lashed together triple store in MySQL but plan on moving to Neo4J when I get some time to write the parsers to get all my data in.

Answer 7

[Pablo Pareja]

Hi!

You can check my answer to the question:

A: 53893 Genes In Ensembl?

All code used here is open-source. Regarding the visualization core, it uses SigMa project, (flash).

Don't hesitate to ask any question you may have ;)

Pablo

Answer 8

[Djie]

@Fengyuan Hu

I just stumbled upon a new java applet which is also capable of displaying networks. It's called Cobweb http://bioinformatics.charite.de/cobweb/. I tested it with your cytoscape web example of the regulatory network of the fly (downloaded the xgmml file) and used it for cobweb. On first view the performance is not much better than Cytoscape Web, but Cobweb has an option to improve performance under the Display Panel. The visuals will be less pretty, but the performance increases drastically.

Answer 9

[miro.marchi]

Hi all. I see this is quite an old question, but always fresh. I'm writing because few days ago Cytoscape.js 2.0.0 was released. Cytoscape.js is the successor of Cytoscape web. I've never used it, anyone does it? I also have a question, is it possible to read from database with this library? Doc says it only loads JSON... Well, I should reformulate my question clearer another time. Also I am not much into database... I'm an anthropologyst phd student in Verona, Italy, and I want to use this sort of software to draw cooperation network... Do you know also gephi toolkit? Anyway, I wanted to say cytoscape.js is out there.