MCB3895-004 Lecture #3 Sept 2/14 Intro to UNIX terminal.

Slides:



Advertisements
Similar presentations
Introduction to Linux command line for bioinformatics Wenjun Kang, MS Jorge Andrade, PhD 6/28/2013 Bioinformatics Core, Center.
Advertisements

Linux, it's not Windows A short introduction to the sub-department's computer systems Gareth Thomas.
Learning Unix/Linux Bioinformatics Orientation 2008 Eric Bishop.
การใช้ระบบปฏิบัติการ UNIX พื้นฐาน บทที่ 4 File Manipulation วิบูลย์ วราสิทธิชัย นักวิชาการคอมพิวเตอร์ ศูนย์คอมพิวเตอร์ ม. สงขลานครินทร์ เวอร์ชั่น 1 วันที่
Introduction to Linux Alan Orth April 17, 2010 ILRI, Nairobi.
Introducing the Command Line CMSC 121 Introduction to UNIX Much of the material in these slides was taken from Dan Hood’s CMSC 121 Lecture Notes.
Exploring the Internet Creating and setting up your website Instructor: Michael Krolak Instructor: Patrick Krolak See also
Lesson 22 – Introduction to Linux Systems Administration.
Linux+ Guide to Linux Certification, Second Edition
Exploring the Internet Creating and setting up your website Instructor: Michael Krolak Instructor: Patrick Krolak See also
Lane Medical Library & Knowledge Management Center Essential UNIX Skills for Biologists Yannick Pouliot, PhD Bioresearch Informationist.
T UTORIAL OF U NIX C OMMAND & SHELL SCRIPT S 5027 Professor: Dr. Shu-Ching Chen TA: Samira Pouyanfar Spring 2015.
Introduction to Unix – CS 21 Lecture 5. Lecture Overview Lab Review Useful commands that will illustrate today’s lecture Streams of input and output File.
Linux & Shell Scripting Small Group Lecture 4 How to Learn to Code Workshop group/ Erin.
A crash course in njit’s Afs
Introduction to UNIX/Linux Exercises Dan Stanzione.
Help session: Unix basics Keith 9/9/2011. Login in Unix lab  User name: ug0xx Password: ece321 (initial)  The password will not be displayed on the.
Unix Primer. Unix Shell The shell is a command programming language that provides an interface to the UNIX operating system. The shell is a “regular”
CSE 390a Editing and Moving Files
HCC Workshop August 29, Introduction to LINUX ●Operating system like Windows or OS X (but different) ●OS used by HCC ●Means of communicating with.
Welcome to Linux & Shell Scripting Small Group How to learn how to Code Workshop small-group/
1 Intro to Linux - getting around HPC systems Himanshu Chhetri.
CIS 90 - Lesson 15 Lesson Module Status Slides – draft Properties - done Flash cards – 1 st Minute quiz – NA Web calendar summary – done Web book pages.
Accessing Barney Off- Campus How can I get my H: files when I am not on the GU network? Business 111 Edward Mitchell Fall 2006.
Agenda User Profile File (.profile) –Keyword Shell Variables Linux (Unix) filters –Purpose –Commands: grep, sort, awk cut, tr, wc, spell.
Basic unix commands that everyone should know (Even if you have a mac) Slightly more advanced:
Dedan Githae, BecA-ILRI Hub Introduction to Linux / UNIX OS MARI eBioKit Workshop; Nov , 2014.
Cluster Computing Applications for Bioinformatics Thurs., Aug. 9, 2007 Introduction to cluster computing Working with Linux operating systems Overview.
AN INTRO TO UNIX/LINUX COMMANDS BY: JIAYANG WANG.
Linux+ Guide to Linux Certification, Third Edition
Carnegie Mellon Linux Boot Camp Jenna MacCarley, Peter Pearson, Shashank Goyal 9/19/2015.
PROGRAMMING PROJECT POLICIES AND UNIX INTRO Sal LaMarca CSCI 1302, Fall 2009.
CCPR Workshop Introduction to the Cluster July 13, 2006.
Week 3 Exploring Linux Filesystems. Objectives  Understand and navigate the Linux directory structure using relative and absolute pathnames  Describe.
Chapter Five Advanced File Processing Guide To UNIX Using Linux Fourth Edition Chapter 5 Unix (34 slides)1 CTEC 110.
Agenda Link of the week Use of Virtual Machine Review week one lab assignment This week’s expected outcomes Review next lab assignments Break Out Problems.
1 Operating Systems and Using Linux Topics What is an Operating System? Linux Overview Frequently Used Linux Commands Some content in this lecture added.
Operating Systems and Using Linux CMSC 104, Lecture 3 John Y. Park 1.
Intro to Programming Environment 1. Today You Will Learn how to connect to a machine remotely with “nomachine NX client” Learn how to create a new “source.
Chapter Five Advanced File Processing. 2 Lesson A Selecting, Manipulating, and Formatting Information.
Getting the most out of the workshop Ask questions!!! Don’t sit next to someone you already know Work with someone with a different skillset and different.
The Genome Analysis Centre Building Excellence in Genomics and Computational Bioscience.
Week 9 - Nov 7, Week 9 Agenda I/O redirection I/O redirection pipe pipe tee tee.
Linux Commands C151 Multi-User Operating Systems.
CS 245 – Part 1 Using Operating Systems and Networks for Programmers Jiang Guo Dept. of Computer Science California State University Los Angeles.
AN INTRO TO UNIX/LINUX COMMANDS BY: JIAYANG WANG.
Linux+ Guide to Linux Certification, Second Edition Chapter 4 Exploring Linux Filesystems.
File Transfer Protocol (FTP) CIS 130. File Transfer Protocol (FTP) Copy files from one internet host (server) to your account on another host –Need domain.
 Last lesson, the Windows Operating System was discussed along with the Windows command shell  Unix is a computer operating system, that similarly manages.
1 CS3695 – Network Vulnerability Assessment & Risk Mitigation – Introduction to Unix & Linux.
Operating Systems and Using Linux Courtesy of John Y. Park 1.
CS 120 Extra: The CS1 Server Tarik Booker CS 120.
Learning Unix/Linux Based on slides from: Eric Bishop.
Intro to GNU/Linux See, Stallman? I said GNU. Are you happy now?
Review Why do we use protection levels? Why do we use constructors?
Tutorial of Unix Command & shell scriptS 5027
BIF713 Managing Disk Space.
C151 Multi-User Operating Systems
Introduction to Linux Dr Karina Kubiak - Ossowska
Tutorial of Unix Command & shell scriptS 5027
Tutorial of Unix Commands
Tutorial of Unix Command & shell scriptS 5027
Operating Systems and Using Linux
College of Engineering
Operating Systems and Using Linux
Tutorial of Unix Command & shell scriptS 5027
Tutorial Unix Command & Makefile CIS 5027
Downloading workshop files to your computer
CSE 303 Concepts and Tools for Software Development
Introduction to Linux and Code Editing
Presentation transcript:

MCB Lecture #3 Sept 2/14 Intro to UNIX terminal

Introduction to UNIX Nearly all bioinformatics software runs on UNIX and its derivatives (e.g., LINUX and Mac OS) Very little bioinformatics software runs on Windows Bioinformatics is very strongly tied to the open- source software movement Lots of help available on-line Most programs are free Windows is not very open-source friendly

Windows users: Option 1: Do all of your work connected to the Biotechnology Cluster server. Download sshclient (ftp://ftp.uconn.edu/restricted/ssh/)ftp://ftp.uconn.edu/restricted/ssh/ Option 2: Install LINUX to run in parallel with Windows (e.g., Biolinux

Terminal The terminal is the primary way to do computational biology Mac: Utilities/Applications/ Terminal Linux: Applications/Accessories/ Terminal Windows: sshclient

Assignment A handy resource to learn the basics of UNIX is the “Unix and Perl Primer for Biologists”, which can be found here: _perl_v3.1.1.pdf _perl_v3.1.1.pdf The commands they demonstrate mainly involve creating, removing and moving around files and directories Once you learn them, these commands will take you far beyond what you can do with a more familiar GUI like Mac Finder or Windows Explorer

Worthy of special comment 1.Directory trees 2.Using tab to autocomplete 3.Wildcard characters like * to perform the same operation to multiple files (this is insanely useful once you get the hang of it!) 4.Using nano as a very basic text editor Never, ever, ever use Word for this! 5.Use underscores “ _ ” not spaces in your filenames

Directory trees All computer files are organized hierarchically Each folder has an address /Users/Jonathan/ Laptop_backup/Destop/ e-Books

A quick reference to where you are in UNIX “ / ” - root “ ~ ” - your user home directory “. ” - “here”, the directory you are in now “../ ” - one level up in the directory tree

More UNIX tricks “ > ” (greater than) redirects the output of a command into a new file e.g., ls * > list a list of the files in this directory is now stored in the file “ list ”

More UNIX tricks cat joins multiple files together e.g., cat file1 file2 > file3 file3 contains file1 and file2 joined together file1 and file2 still exist as they were

More UNIX tricks grep extracts all lines containing a particular pattern from a file e.g., grep “NP_” file1 Prints every line that contains the pattern “ NP_ ” to the screen

More UNIX tricks wc counts the newlines, words and bytes in a file e.g., wc file1 Prints an output like this: file1 newlineswordsbytesfilename

More UNIX tricks “ | ” (pipe) directs the output of one command into another e.g., grep “NP_” file1 | wc Sounds the output of the grep command into wc, because grep extracts lines from a file, can be used to count the number of lines matching the grep expression e.g., grep “NP_” file1 | less Displays grep result as a list you can scroll through

More UNIX tricks gzip / gunzip : single file compression e.g., gunzip file.txt.gz Decompresses file.txt e.g., gzip file.txt Creates compressed file file.txt.gz, removes file.txt

More UNIX tricks tar : file archive management e.g., tar -cf all.tar * Creates tar archive all.tar containing all files in that directory, individual files unchanged e.g., tar -xf all.tar Extracts all files from tar archive all.tar to the current directory, all.tar not deleted tar is very commonly used before gzip - “tarballs”

Connecting to the Bioinformatics facility server UNIX command ssh e.g., ssh -l jlklassen bbcsrv3.biotech.uconn.edu Will ask for a password If the first time connecting, will want you to authenticate an RSA key (security feature) Your terminal now controls the bioinformatics facility server, not your own machine You can have multiple terminals open at the same time

Transferring files to the Bioinformatics facility server Method 1: Filezilla ( project.org/) project.org/ Nice GUI Works on all platforms Install the client, not the server

Transferring files to the Bioinformatics facility server Method 2: UNIX command scp e.g., scp all.tar Copy all.tar from my computer to the biotech server e.g., scp -r Copy the directory “ dir ” from the biotech server to the current working directory “ -r ” flag indicates “recursive”, needed for directories

Text editors Using nano works, but can be cumbersome for complex tasks Word is always bad! Adds layers you don’t see. Mac and LINUX have TextEdit and Gedit as default text editors, both work well Windows: Notepad and Wordpad are insufficient. I suggest downloading Gedit for Windows ( Other options exist for all platforms

Assignment See instructions posted on the website at Part 1: work through Korf manual sections U1- U27 (some commands require external files, ignore these but understand what they do) Part 2: log on to the Biotech server, download a genome from NCBI and answer the questions given The assignment is due at the start of class 1 week from today

Command line power! The simplest way to download these data is to use the terminal command wget $ wget –r --no-directories --retr-symlinks -P Acaricomes_phytoseiuli/ ftp://ftp.ncbi.nlm.gov/genomes/refseq/bacteria/Aca ricomes_phytoseiuli/latest_assembly_versions/GCF_ _ASM37624v1/ Deconstructed: -r – “recursive”, i.e., download everything in this directory --no-directories – does not create the entire ftp directory structure --retr-symlinks – NCBI uses a fancy file structure using something called “symbolic links”, where a file points to another file somewhere else. “--retr-symlinks” gets the actual files, not just the links -P Acaricomes_phytoseuili/ – where to put the output