Best Way For A Beginner To Get Up To Speed On Unix Quickly?

Question

[Brett Thomas]

Does anybody have recommendations for the best way to for him/her to learn unix for bioinformatics quickly?

Our group works with genomics datasets. We work exclusively in unix, but we'll sometimes have a researcher visit temporarily that has never used unix. If the researcher is using one particular tool - R or Perl, for example - we'll guide them to a beginners book for that tool that covers unix in passing. But sometimes they just want to get started quickly and run a few command line utilities. A short book or long tutorial would be best for this case, but I can't find a good one.

Answer 1

[Frédéric Mahé]

You should start with a list of very basic assignments. Install the beginner in front of a terminal and made him/her perform simple tasks using Unix tools:

• where am I? (pwd)
• what's in there? (ls)
• what can I do? (TAB TAB)
• create a folder (mkdir newfolder)
• enter that folder (cd newfolder/)
• go back one level (cd ..)
• erase the folder and introduce man (rm newfolder ; man rm ; rm -r newfolder/)
• read a file (less file.fasta)
• count lines (wc -l file.fasta)
• count sequences (grep -c "^>")
• and so on.

It's time consuming for the guider but it's a very nice way to learn (I wish someone did that for me). The good news is that the beginner will rapidly gain confidence and autonomy. After that it's mainly a matter of motivation and self-learning.

[Neilfws]

This is exactly what I used to do with students. Show them just enough to achieve a few basic tasks, then build on that as required.

[Frédéric Mahé]

For students familiar with Apple computers, it's easier because they are working on a Unix system with a shell (they just don't know it). But for students only familiar with Microsoft Windows, the cultural gap is much wider. They just can't open a Terminal and experiment on their own computer, they have to go through a more complex process (install putty and connect to a Unix box).

Students familiar with Unix/Linux are becoming more frequent but, with modern distributions, the need for command line interface in a day to day use is reduced to almost zero. So, they also need guiding.

[Jelena Aleksic]

I've been getting my Windows-using students to install Cygwin and work in that. I find that bridges some of the gap, and has been working well so far. It's what I use on my PC at home too (I'm a Mac user at work...).

Answer 2

[Pierre Lindenbaum]

In my bookmarks: @kbradnam 's

"Unix and Perl Primer for Biologists"

http://korflab.ucdavis.edu/Unix_and_Perl/

[Steve Moss]

With this one - http://www.amazon.co.uk/UNIX-Perl-Rescue-Sciences-Data-rich/dp/0521169828 - coming soon ;)

Answer 3

[Stephen]

Don't underestimate the power of find and xargs for operating on multiple files at once. The Bioinformatician's UNIX toolbox has several examples of xargs, awk, and other useful tools.

Also, you'll want to get a good text editor. I use notepad++ on windows, and either vim or textmate on mac. If you're on a linux or mac, type vimtutor at the command line to get started.

Answer 4

[Sptaylor]

I highly recommend Software Carpentry: http://software-carpentry.org/4_0/shell/.
They also have great tutorials for beginning Python.

[Brett Thomas]

Great link. Just to clarify: this is a set of videos, not text tutorials. Wouldn't be my choice, but I imagine some beginners will love these.

Answer 5

[Gjain]

Hi Brett,

I gave a presentation once and there was a slide with few practice questions that helped my colleagues. Some of those are:

• First list the files in the directory.
• Copy one file to another file
• Look at the content of one file from the top
• Look at the content of one file from the bottom
• Check if two files are identical.
• Delete a file
• Make a test directory
• Move couple of files to it.
• Remove test directory.
• Make five new directories one inside the other like level1/level2/level3/level4/level5 in one command.
• Move a file to the innermost directory.
• Copy a file to your home directory

You can find the commands to do these tasks here Basic Unix/Linux Commands

You should also work on learning few important unix utilities like:

• grep (link1, link2)
• sed (link)

I hope this helps.

[Brett Thomas]

Great commands reference -- I'll definitely recommend people print that out.

[Gjain]

thanks, I felt that this is an easier way to introduce command and ease people with non-unix/linux background into the environment.

Answer 6

[Steve Moss]

Install Ubuntu (Debian based) - it makes life so much easier when you're starting out with UNIX based systems, partly due to the power that aptitude (it's pacakage management system) provides, but also because of it's excellent driver support! Fedora's (RedHat based) yum and Gentoo's emerge are close runners-up in this regard (but Gentoo is something to aim at installing when you have been using Linux for a good while - or, if you want to throw yourself in at the deep end).

Incidentally, Gentoo is my personal favourite and really hones your skills. It also compiles everything from source, so really allows you to optimise your system through compile flags etc. It has an excellent handbook you can follow to install, but is no easy task. If in doubt, go with Ubuntu - or BioLinux is cool too (similar Fedora one here too), as it has a lot of the software and libraries you may need for bioinformatics pre-installed. It is based on Ubuntu 10.04, so is a little outdated, although 10.04 is a long-term support release.

There is a lot to be said for trial and error. If you have an old machine you don't mind messing around with and testing out commands on, then install it on that, or if you don't then install it in a virtual machine using the likes of VirtualBox and you don't have to worry about breaking anything and can hack away to your heart's content. One useful thing with Virtualbox is you can find images already setup to run straight away - e.g. http://qiime.org/install/virtual_box.html.

Join this Google user group http://groups.google.com/group/unix-and-perl-for-biologists?pli=1, check out http://www.omgubuntu.co.uk/ and feel free to ask questions here http://askubuntu.com/.

Check out http://www.linuxjournal.com/, http://www.linuxformat.com/, http://www.linux-mag.com/ and http://www.linuxpromagazine.com/.

Read the UNIX and Perl Primer http://korflab.ucdavis.edu/Unix_and_Perl/ and the new book coming soon http://www.amazon.com/Unix-Perl-Rescue-Keith-Bradnam/dp/1107000688, which also has a site http://unixandperl.com/.

Documentation is rife for most UNIX and Linux tools and you can either use man, or simply Google most things and come across a plethora of answers (e.g. Linux man pages). Also, browse Linux.com, look at their tutorials, documentation and this link.

Any of the O'Reilly books are great.

IRC seems to be used less these days, but is an excellent resource - I hang out in #freeside and #hulllug at irc.freeside.co.uk (run by CS department at Hull University), but there should be UNIX/Linux channels on most IRC servers - e.g. http://justlinux.com/forum/showthread.php?t=107414 - freenode is arguably the best?

[Leszek]

+1 for Ubuntu. changed my world definitely :)

[mcc]

Ubuntu is the right place to start to learn Linux. There is Soooo much software out there for free!

[Steve Moss]

It is pretty awesome! I'm not a great unity fan in 11.04+ though! Been installing gnome-shell instead :-)

[Neilfws]

I moved to Linux Mint. All the goodness of Ubuntu without the horror of Unity.

[Leszek]

As many others, I don't like Unity. But you can easily get old and beloved gnome-panel: http://bit.ly/V1pPzC so no reason to abandon Ubuntu ;)

Answer 7

[Alex Paciorkowski]

This is not an easy task. Unix/Linux is one of the most byzantine, labyrinthine environments for a newcomer without a background in comp sci to try to work in -- yet remains the most effective and versatile environment in which to get our work done. To get around this very issue, we are going to be moving our workflows as much as possible to Galaxy to take advantage of that user interface, or you can write some quick in-house user interfaces for beginners to use to access your Perl scripts or whatever.

What it sounds like you are asking is -- how to get a beginner familiar with and comfortable with the command-line environment?

What I assume you are not talking about is this: Teach yourself programming in 10 years.

Instead, you want to help a newcomer get started -- this means learning how to manipulate files and directories, and how to get started using bash. There are a couple of reasonable on-line tutorials out there:

Teaching Unix, Linux Shell Scripting Tutorial (which is kinda old, but a lot of this just doesn't change).

[Alex Paciorkowski]

Of course, Frédéric, we're in complete agreement. I was speaking from the perspective of the beginner. Although the OS may be logic itself once it is familiar, it is nevertheless a common experience that users coming from a MS windows or Mac OSX environment find it very hard to believe that anything other than a point-and-click environment is progress. For them, Unix appears esoteric and its traditions very unusual indeed. But it's so much more fun, isn't it?

[Frédéric Mahé]

Instead of "byzantine", I prefer to say that "traditions" exist in the Unix/Linux world. Regarding the folder hierarchy and environment organization, I find it logic (but I may say so because I'm used to it). I am 100% with you when you say that it is one of the best environment in which to get work done.

[Frédéric Mahé]

My first experience with computers was really bad. I had to struggle with floppies and an erratic word processor to write down an essay, and I ended up considering computers as hostile entities. Then one day, someone gave me an old Sun computer running on GNU/Linux and learned me how to use LaTeX. I then learned that computers are not supposed to be erratic, that you can actually control them, and that they are wonderful tools with endless possibilities. You're right, users coming from Windows or Mac OS see the command line interface as something outdated, where we see raw power.

Answer 8

[Steve Lianoglou]

This tutorial covers lots of tools to manipulate and summarize text: Unix for Poets.

The domain within which unix commands are explained is obviously different, but the reader will have an exercise-based-text that covers lots of things you do from the command line, eg grep, awk, shell scripts, counting lines/words, using | and >, etc.

Answer 9

[Thaman]

It seems most of things has been already covered. But, if someone wants to learn everything from a single portal then http://www.bashoneliners.com/ is the best choice.

[Josh Herr]

This is great! Thanks for the link. I also like alias.sh for some ideas for bash oneliners that I can use for FASTA processing.

Answer 10

[Josh Herr]

In addition to all the great recommendations, resources, and tools presented above, I have found that the book "Practical Computing for Biologists" was really great for beginning UNIX literacy for people who may not have ever used the command line. I knew a lot in the book already, but I also use it for reference all the time. I wrote a review of the book on my blog here.