Perl6 and the Future of Bioinformatics - Compatibility and Integration

Question

[Austin]

As someone who has yet to work in a professional and/or communal environment I have been curious as to the current and expected paradigm of Bioinformatics and the use of Perl as programming language for data manipulation and analysis. With the announcement of Perl6 and its (tentative) emergence in late 2015 what are the major considerations for its future relationship? Perl(5) has a number of historical connotations and a result is widely used along with its assets like CPAN and BioPerl. Moreover, (correct me if I am wrong) with Unix-Like systems being dominate in the scientific community and that Perl is often pre-installed, it is therefore is a good alternative or compliment to the “power” of the command line. However, Perl6 is “officially” not compatible with Perl5.

1. What is the status of the pre-existing resources (BioPerl etc.) in terms of conversion and/or compatibility?
2. Will Perl6 also be pre-installed on future Mac/Linux operating systems?
3. What has been the impact or reaction (now/future) in your institution/employer and professional community; are alternatives being considered?

As a current Perl user will you begin using Perl6, what of your colleagues? Please feel free to include any other considerations you are aware of. Thanks for your time.

[nine]

Unlike the mentioned Python 3, Perl 5 and Perl 6 _are_ compatible and were always meant to be. One can use Perl 5 and Perl 6 modules and classes in the same program, including XS modules. While there are still ways this can be made more comfortable, it is already working right now. Also Perl 5 is being actively developed and will continue to be for the forseeable future. So the question of a migration may not even come up. It's more a question of utilizing the strenghts of both languages.

Answer 1

[Dan D]

What is the status of the pre-existing resources (BioPerl etc.) in terms of conversion and/or compatibility?

It will have to be on a package-by-package basis, but I imagine that for anything except the major, highly-active projects, migration to 6.x will happen slowly if at all. BioPerl6 is active but I haven't seen much from the other major packages.

Will Perl6 also be pre-installed on future Mac/Linux operating systems?

I think that's highly unlikely, as there's little need for it. Some of the specialty distros might include it for grins, but there's no practical reason to include it out-of-the-box.

What has been the impact or reaction (now/future) in your institution/employer and professional community; are alternatives being considered?

Perl and I had some lovely times together, but I've moved on to the asynchronous hotness of JavaScript and the sexy ease and practicality of Python. Most people I know who've used Perl in the past have moved on also. I don't personally know anyone in my circle of ~40 programmers who uses it for new projects or problems. The programming community at large has moved on as well. I can't see Perl 6 bringing them back.

At best, I think Perl 6 will have the same impact as Python 3. Some people might pick it up out of curiosity, but the larger Perl community won't have much of an incentive to use it, as most of the calls from recruiters I get about Perl jobs (and they are few) are for positions maintaining legacy systems.

At worst, the broad response will be Perl who?

Fun fact: The Perl 6 project is well over half the age of Perl.

[SES]

I can't predict the success of the project, but I know that Perl 6 development has taken from and given back to other communities. An earlier Perl 6 project had an experimental compiler written in Haskell (the Pugs project), and the developers spent a lot of time on the Haskell mailing list and ended up contributing to the development of Haskell itself. Another experimental runtime was written in Scala. There were workshops at the Perl conference this year that were for learning Go and Rust. The meta-object system is based on the success/failings of lisp, Java/Ruby (traits/mixins) for sure, and other languages. I think the Perl 6 developers are looking outwards, and that is why I think it has a chance to be successful. So, I wouldn't say it is only relevant to people who use Perl 5. If you are a programmer, I think you should pay attention because this might be what the future of languages looks like (or is), in terms of OOP, concurrency, use of grammars, etc.

At the least, I would say it gives a deeper understanding of modern programming (and other languages) to see the what/why of Perl 6 parts, and at best, it could make our lives better. That is optimistic, but the developers use other languages in their day jobs and they seem to be very driven and excited about this project, so why else would they do it? It could be for fun, but still something we can learn from.

[Dan D]

Very interesting perspective. I can see how my post might come across as dissing the language itself. I don't know enough about it to make a value judgement at this point. My thinking is that the association of "Perl" with "old" will hamper its uptake, and Perl 6 has the stigma of being vaporware as it's been in development since 2000. It's trendy right now to scoff at Perl.

However, fickleness works both ways and the things you said about Perl 6 have convinced me to check it out once it's released. The diversity of the Perl 6 team is pretty interesting. Folding in ideas from all those modern languages could end up being a major boon. Bioinformatics is all about fresh ideas, so if Perl 6 can make our jobs easier or better I just might have to renew my Camelid club membership. :)

Answer 2

[SES]

What is the status of the pre-existing resources (BioPerl etc.) in terms of conversion and/or compatibility?

I don't know if Perl 6 will compile your BioPerl code, but I know that Perl 6 will compile (at least some) Perl 5 code. Yes, there are a couple of projects underway to have a set of Bio* classes in Perl6. The main project is called bioperl6.

Will Perl6 also be pre-installed on future Mac/Linux operating systems?

Who knows what the future holds! When a stable release is made later in the year I imagine it will become available via your package managers but I wouldn't expect it to become as ubiquitous as Perl 5, at least not for a long time.

What has been the impact or reaction (now/future) in your institution/employer and professional community; are alternatives being considered?

I don't think most people know what Perl 6 is, so there hasn't been a lot of discussions among my colleagues or in any Bio communities that I follow online. Part of the problem is the name, which makes it sound like a standard version change like Python 2 -> 3 or PHP 4 -> 5. Most of those version changes don't involve revolutionary changes that alter the way you do things, so what is there to be excited about Perl 5 -> 6? Of course, Perl 6 is a completely different language from Perl 5 and revolutionary in a number of ways (support for different runtime environments, an integrated meta-object protocol, concurrency, improved performance, unicode support, etc.). The project has taken a long time to develop, but Larry Wall has said he never set out to simply create another scripting language. The goal was to create something more powerful than anything else and that takes time (the interview is online, I'll look for it).

As a current Perl user will you begin using Perl6, what of your colleagues?

I can't speak for my colleagues, but I have been experimenting with Perl 6 and I plan to spend more time with it. In the past, the performance of Perl 6 was too far below Perl 5 for me to seriously consider it, but now Perl 6 is faster than Perl 5 at a number of tasks (still slower or the same at others). What I would like to have in a programming language would be for it to run my code/pipeline in parallel without me doing back flips to create threads or forks, and Perl 6 seems to do that for you very simply (here is a link from a recent talk about concurrency in Perl 6 by the lead developer). Okay, parallelism isn't that hard in Perl 5, but I have to think about it, and it would be great to not think about it. What I have a hard time getting my head around is how different Perl 6 is. I will have to learn quite a lot to be efficient at Perl 6, but that is also a good reason to try something new (and it is a bonus if it potentially makes you more productive).

[Dan D]

What I would like to have in a programming language would be for it to run my code/pipeline in parallel without me doing back flips to create threads or forks

Have you looked at NodeJS or Go? Both are very good at those things.

[SES]

I am familiar with them, most modern languages are built around concurrency. You can do non-blocking web development and event-based programming in perl quite easily but it isn't a core feature (of course, given it's origin). I like JS, and use it more and more. Go is not what I was referring to exactly (compiling code) and Node.js is a library (not a scripting language or command line tool). My limited understanding based on the talks I have seen is that concurrency is not a solved problem in any language, but it is getting better and Perl 6 seems to be incorporating/implementing some of the best features. We shall see, I wish I could say more about it.

[Istvan Albert]

Interestingly both Python and Perl core devs seem to be focusing on asynchronous type of programming - in a way "betting the future" of the language on that.

Recently Python 3.0 has shown a more pronounced uptake though I must say every time I use Python 3.0 I feel like I am using a somewhat quirkier Python 2.7 - I personally I still don't really think it was worth it.

[SES]

Indeed, asynchronous programming seems to be at the core of every new framework or library, especially in web applications. The implementations are slightly different but most seem to be based on Futures (Scala) or Promises (JS) that I have seen (Promises are a core feature of Perl 6).

I don't think splitting the community into non-compatible projects is good for the language itself, but it can bring in new ideas. I find it frustrating when projects require Python 3, yet other Python projects aren't compatible with it. That makes it less practical to use everyday. Perl 5/6 will probably suffer the same, though I can't predict how compatible the languages will be.

[lh3]

Most of my scripts operate on one or several sync'd I/O streams. I call a function like readline() and then get a line from a stream. Async I/O is a bad fit for such text processing routines. I liked node.js/dart mostly due to their high performance, but forcing me to use async I/O for basic line reading (their sync I/O is very limited) has pushed me away.

Perl 6 will probably be far worse than Python 3. I see perl is still relevant due to the existing code base, fast scripting and stability (many perl scripts written today are likely to work on 10-year-old machines and vice versa). Perl 6 has none of these. If I want to pick up a new language, there are many interesting ones that are as competent as and more mature than Perl 6.

[Austin]

If you woke up tomorrow without any of the code you had previously written and could only remember C++ or Java, which language (or two) would you start?

[lh3]

It will be python and/or julia.

[William]

Python and Scala because of their use as scripting languages and as supported interface languages for big data systems like Apache Spark.

[Austin]

Thank you both for your responses, I really appreciate it. As far as I understand Perl is valued for regular expressions. If that is the case and with what has been mentioned here as working with/in concurrent and parallel while (mostly) not compiling. What of Julia and Clojure? Are they discussed or used? It would seem like they could alleviate some of the issues while still interacting well with Python (which seems to be the norm for tasks previously done in Perl).

[Istvan Albert]

I think functional languages such as Clojure rely on (very elegant) constructs that require a much higher ability for abstraction - that becomes a high barrier of entry. In contrast explaining the way a 'for loop' works is a whole lot easier. Julia seems like an interesting language, but in the end the size of the ecosystem is just as important. People want to learn languages that are suited for a wide array of applications.

[Endre Bakken Stovner]

I've been playing around with Clojure, but being tied to the JVM and its startup times is less than ideal.

My language of the future is Nim; as fast as C, more powerful than Python/Ruby and its metaprogramming capacity is only surpassed by Lisps. (It is not in 1.0 yet and the docs are still meh). See http://nim-lang.org/

[SES]

I was wrong about the packaging, Perl6 (rakudo-star) and MoarVM are available from the package managers already and have been for a long time (since ~2011 best I can tell). I think rakudobrew is the easiest way to install though, as it doesn't require any special privileges to get started.

Answer 3

[mattoates]

I've been having fun with bioinformatics use case in Perl 6 since 2010. It's really great, absolutely the perfect language syntactically and semantically for getting stuff done naturally. For some pointers I wrote a blog article in the last Perl 6 Advent Calendar you might want to check out if you haven't already.

Along with the proper more official BioPerl6 project there is the BioInfo repo available in the ecosystem which is just some of my early code playing around, but still might give you some ideas on why Perl 6 is nice.

The only downside is performance. So for anything really computationally intense Perl 6 isn't quite there yet, but the ease of use in parallel processing goes some of the way to mitigate that. You can just tell performance will get there though I have seen more than a 10x improvement in the speed of my tests in under two years of development on the Rakudo compiler. The new native shaped arrays and more recent concurrency stuff should go a long way to speeding things up further and reducing memory cost.

Another huge win that cannot be sold enough and makes up for the performance problems is the Native Call interface. It is almost too trivial to bind C libraries (and soon to be C++ libraries) only using Perl 6. You for the most part dont have to write or compile a single line of C to call functions in a C library from a .so file.

Despite the FUD out there you can use all the BioPerl modules via the Inline::Perl5 module which is really advanced and has great support including for XS modules. You can even mix and match Moose classes/roles with Perl 6 ones. Inline::Python from the same author also exists but isn't quite as mature but I used it to pop up an ete2 window for displaying a phylogenetic tree! So not too shabby.

Compatibility wise I've used Rakudo on Windows, Linux and OSX my same code runs everywhere. You can even compile to a .jar with the JVM implementation and just hand that to someone (though its quite slow for short lived scripts). You can also use any Java class from a Perl 6 script on JVM, so by extension BioJava is available when targeting the JVM with your Perl6 code.

I think for prototyping or orchestrating a mixed bag of libraries Perl 6 is great.

For running and installing Rakudo Perl 6

Windows just grab the most recent JIT .msi from http://rakudo.org/downloads/star/

OSX you can brew install rakudo-star

For Linux I would recommend using rakduobrew to build a personal copy of Perl 6 in your home, this works on OSX just as well:

#Setup the rakudobrew software to manage Perl 6 installs
git clone https://github.com/tadzik/rakudobrew $HOME/.rakudobrew
echo 'export PATH=~/.rakudobrew/bin:$PATH' >> $HOME/.bashrc
export PATH=~/.rakudobrew/bin:$PATH

#First time you want to build the chosen VM back end
#and the panda package manager to get all the basic modules
rakudobrew build moar
rakudobrew build-panda
panda install Task::Star

#Then get a REPL and profit
perl6

If you want the JVM version just replace build moar with build jvm. You will need a recent JRE installed though.

Answer 4

[Jean-Karim Heriche]

A lot of perl5 modules are actually wrappers around C libraries. This is made possible using XS, a perl5 extension language that interfaces perl5 and C. Since there is no XS in perl6, all the perl5 modules relying on XS will not be compatible. However, perl6 should make it easy to rewrite those modules with its NativeCall module.

[nine]

Perl 5 modules relying on XS can be used in Perl 6. This works already:

panda install Inline::Perl5

Then in your Perl 6 code:

use JSON::XS:from<Perl5>;
say encode_json($[1, 2, {a => 3}]);