Software for metadata management, alternative to Excel

Question

[DriesB]

My group is trying to keep track of metadata in a shared research project. Information such as age, sex, disease status, etc. Before, this was collaboratively done via spreadsheets managed through Microsoft Excel. It is generally known that Excel's autoconversions cause problems however. Moreover, the fact that anyone wanting to insert a new datapoint could, by mistake, edit an earlier entry is also problematic.

The problems of using Excel in biology have been mentioned before, but alternatives are hard to find. Using OpenOffice or LibreOffice is not likely to solve the issue, and statistical software like R is meant for data analysis, not data entry.

I have started to look for alternatives for managing metadata. I'm looking for software that matches the following requirements:

• Importantly, inputting data should be easy for users not used to programming. It should be as intuitive as spreadsheet software or a web interface. In this way, it is accessible to both wet lab and dry lab staff.
• Allows to input new entries, without being able to modify earlier entries. Only admin can modify earlier entries.
• Checks for input, for example only specific naming for cancer types. For example, not 'LC', but 'lung cancer'.
• No autoconversions
• Exports to open data formats
• (Easy joins/merges with similar tables. However this can also be done via external software like R.)

Does anyone have suggestions for software for managing metadata?

[DriesB]

This question has some similarity to Management software for my samples?, but quite some time has passed since that was asked and I think my question is more narrowly asked by revolving around Excel and my requirements.

[GenoMax]

At the end of the day you will find that you will need to design something yourself since practices at each facility differ a lot and users are not keen to change their protocols. Sounds like you can look at various open-source LIMS options and then plan to use just the components/parts you need.

[DriesB]

Thank you for your first point about requirements differing too much between groups to allow for a simple standard answer. However, as I'm sure that my requirements regarding accessibility for lab biologists, modification tracking, autoconversions and export are very common, I still hoped to find some suggestions that make the search for a suitable system easier for the next person trying to get away from Excel.

Regarding a LIMS, I thought it might be more than we need in our situation, as all information can for now be contained in three simple tables. Also, the question Is there a Lims that doesn't suck? made me a bit pessimistic, although that discussion is very old. Indeed, I think this falls in the category Lab Information Management, but I want to get closer to what the properties of such a LIMS should be. And perhaps which LIMS to start from.

[Arup Ghosh]

For basic usage a CRUD application with dataTables in the front end and PHP+MySQL back-end will do the job.

[DriesB]

Yes, this description does seem to meet my requirement. But could you make it more concrete in the form of a tool, such as genomax in his comment?

Answer 1

[Jeremy Leipzig]

Here is a Flask/Vue example https://testdriven.io/blog/developing-a-single-page-app-with-flask-and-vuejs/ or consider Dash https://dash.plotly.com/datatable/editable

[DriesB]

Thank you, this makes more clear how to build the application that Arup Ghosh's described! Combined with his description, I think it serves as an answer.

Answer 2

[GenoMax]

It does sound like your requirement is fairly simple and that in itself may be a problem to find something readily available. This may be a good project for an intern/CS student (if you want to hand it to someone else) to over summer.

sierra is a system for managing sequencing runs from authors of FastQC. You could look at it and see if it can be useful. As is or after modifications.

[DriesB]

Thank you for both your suggestions, sierra looks simple and robust.

Answer 3

[DriesB]

During an in-house discussion about this topic, a colleague suggested using a version control system (VCS) to keep track of which changes are made to the metadata table.

By saving flat files or by modifying a diff function this could be combined with spreadsheet software.

However, regarding the requirements defined in my question:

• Getting non-programming staff to use VCS is probably not realistic.
• A system administrator might have a large task managing merging the input of all involved if the VCS is not used properly.
• The risk of autoconversion errors occurring will still be big.

[igor]

What about Google Docs? It has a web interface, has version control, can be exported and be programmatically accessed.

[GenoMax]

Extending this thought one could come with with something in Office 365 (if your institute has access) using VB coding.

[DriesB]

Do you mean "access to Office 365" or "access to Access"? Because I've actually looked into the latter: Microsoft's comparison between Access and Excel. It looks like Access should work well. Also, because it's part of Office, colleagues should quickly get used to it.

[DriesB]

Although Google Docs is very user-friendly and has many features, the metadata includes personal information and I therefore need to be sure about where and how it is stored.

Answer 4

[DriesB]

Another solution proposed by a colleague was to use the clinical data platform Castor. After some orientation, we decided to stick with this system. Major benefits for us were that Castor handles data GDPR-compliantly (which is necessary for us); that a license was freely available because of the research organization with which we are associated; and that some staff members already had experience with the system.

These benefits are specific for our group, but apart from those, Castor matches all requirements above. It's quite accessible, protects entries from modification, checks for input, doesn't do autoconversions and easily exports and imports data.

A disadvantage is that the system's architecture, with Forms, Records and Reports is unnecessarily complicated when only handling a table. Another disadvantage is the licensing. OpenClinica seems to be similar to Castor and might function as an alternative.

[GenoMax]

Consider accepting this answer to provide closure to this thread since in this case it seems to fit your specific needs.

[DriesB]

I was more looking to have an open discussion; I don't think this solution is necessarily better than Jeremy Leipzig's or yours. So I'm not sure if accepting the answer improves the threads, unless there is an advantage for Biostars?

[DriesB]

Another name for a clinical data platform such as Castor or OpenClinica is 'Electronic Data Capture (EDC) system' (see Wikipedia). Many other names exist.