Setting Up and Managing a Bioinformatics Project Andrii Rozumnyi and Gagandeep Singh Spring 2016.

Slides:



Advertisements
Similar presentations
Web Development & Design Foundations with XHTML
Advertisements

CSS Cascading Style Sheets. Objectives Using Inline Styles Working with Selectors Using Embedded Styles Using an External Style Sheet Applying a Style.
HTML: HyperText Markup Language Hello World Welcome to the world!
WeB application development
Chapter 3 Working with Text and Cascading Style Sheets.
HTML Hypertext Markup Language –First proposed by CERN in 1989 –It is non-linear so it allows you to jump from place to place –Markup refers to the structure.
Shell Basics CS465 - Unix. Shell Basics Shells provide: –Command interpretation –Multiple commands on a single line –Expansion of wildcard filenames –Redirection.
Web Page Development Identify elements of a Web Page Start Notepad
Guide To UNIX Using Linux Third Edition
Guide To UNIX Using Linux Third Edition
There is a certain way that an HTML file should be set up. The HTML section declares a beginning and an ending. Within the HTML, there should be a HEAD.
HTML: PART ONE. Creating an HTML Document  It is a good idea to plan out a web page before you start coding  Draw a planning sketch or create a sample.
Introducing HTML & XHTML:. Goals  Understand hyperlinking  Understand how tags are formed and used.  Understand HTML as a markup language  Understand.
Creating and publishing accessible course materials Practical advise you can replicate.
CSS (Cascading Style Sheets): How the web is styled Create Rules that specify how the content of an HTML Element should appear. CSS controls how your web.
 Missing (or duplicate) semicolons can make the browser completely ignore the style rule.  You may add extra spaces between the properties/values to.
Understanding HTML Style Sheets. What is a style?  A style is a rule that defines the appearance and position of text and graphics. It may define the.
Creating a Simple Page: HTML Overview
DAY 6: EXCEL CHAPTER 2 Tazin Afrin September 05,
Chapter 2: The Client Side: HTML CIS 275—Web Application Development for Business I.
Internet Skills An Introduction to HTML Alan Noble Room 504 Tel: (44562 internal)
Online Chapter 1 4 th Edition.  Review elements  Whitespace handling  Rule structure  Linking to an external style sheet  Alternate Style Sheets.
Unit 1 – Developing a Web Page. Objectives:  Learn the history of the Web and HTML  Describe HTML standards and specifications  Understand HTML elements.
Lesson 7-Creating and Changing Directories. Overview Using directories to create order. Managing files in directories. Using pathnames to manage files.
ULI101 – XHTML Basics (Part II) What is Markup Language? XHTML vs. HTML General XHTML Rules Block Level XHTML Tags XHTML Validation.
HTML II. Factors to consider in designing a website. Organizing your files. HTML Tables. Unordered Lists. Ordered Lists. HTML Forms. Learning Objectives.
HTML (HyperText Markup Language)
Programming in HTML.  Programming Language  Used to design/create web pages  Hyper Text Markup Language  Markup Language  Series of Markup tags 
Chapter Four UNIX File Processing. 2 Lesson A Extracting Information from Files.
The UNIX Shell. The Shell Program that constantly runs at terminal after a user has logged in. Prompts the user and waits for user input. Interprets command.
Dedan Githae, BecA-ILRI Hub Introduction to Linux / UNIX OS MARI eBioKit Workshop; Nov , 2014.
Nael Alian Introduction to PHP
Using Html Basics, Text and Links. Objectives  Develop a web page using HTML codes according to specifications and verify that it works prior to submitting.
Learning Web Design: Chapter 4. HTML  Hypertext Markup Language (HTML)  Uses tags to tell the browser the start and end of a certain kind of formatting.
1.  Describe the anatomy of a web page  Format the body of a web page with block-level elements including headings, paragraphs, lists, and blockquotes.
Chapter 2 HTML Basics Key Concepts Copyright © 2013 Terry Ann Morris, Ed.D 1.
1 Web Developer Foundations: Using XHTML Chapter 2 Key Concepts.
Python CGI programming
Programming in HTML.  Programming Language  Used to design/create web pages  Hyper Text Markup Language  Markup Language  Series of Markup tags 
 2008 Pearson Education, Inc. All rights reserved Introduction to XHTML.
CSS Basic (cascading style sheets)
INTRODUCTION. What is HTML? HTML is a language for describing web pages. HTML stands for Hyper Text Markup Language HTML is not a programming language,
Intro and Review Welcome to Java. Introduction Java application programming Use tools from the JDK to compile and run programs. Videos at
HTML: Hyptertext Markup Language Doman’s Sections.
Liang, Introduction to Java Programming, Eighth Edition, (c) 2011 Pearson Education, Inc. All rights reserved Introduction to Android (Part.
Ali Alshowaish. What is HTML? HTML stands for Hyper Text Markup Language Specifically created to make World Wide Web pages Web authoring software language.
© 2008 The McGraw-Hill Companies, Inc. All rights reserved. WORD 2007 M I C R O S O F T ® THE PROFESSIONAL APPROACH S E R I E S Lesson 15 Advanced Tables.
Web Development & Design Foundations with XHTML Chapter 2 HTML/XHTML Basics.
Linux+ Guide to Linux Certification, Third Edition
XP 1 Developing a Basic Web Page Tutorial 1. XP 2 Introducing the World Wide Web Network - linking computers, sharing resources Network - linking computers,
Java Doc Guideline R.SANTHANA GOPALAN. Java Doc Guideline Audience Internal Developers PQA - who write test plans PPT – who write the documentation Customers.
HTML Basics. HTML Coding HTML Hypertext markup language The code used to create web pages.
PTA Linux Series Copyright Professional Training Academy, CSIS, University of Limerick, 2006 © Workshop III - Part A Shell Commands Professional Training.
What is HTML? HTML is a language for describing web pages. HTML stands for Hyper Text Markup Language HTML is not a programming language, it is a markup.
Linux+ Guide to Linux Certification, Second Edition
Department of Computer Science, Florida State University CGS 3066: Web Programming and Design Spring
`. Lecture Overview HTML Body Elements Linking techniques HyperText references Linking images Linking to locations on a page Linking to a fragment on.
Department of Computer Science, Florida State University CGS 3066: Web Programming and Design Spring
1999, COMPUTER SCIENCE, BUU Introduction to HTML Seree Chinodom
Tutorial #1 Using HTML to Create Web Pages. HTML, XHTML, and CSS HTML – HyperText Markup Language The tags the browser uses to define the content of the.
Linux Tutorial Lesson Two *Getting Help in Linux *Data movement and manipulation *Relative and Absolute path *Processes Note: see chapter 1,2,3 from Linux.
HTML Tutorial. What is HTML HTML is a markup language for describing web documents (web pages) HTML documents are described by HTML tags Each HTML tag.
NOTEPAD++ Lab 1 1 Riham ALSmari. Why Notepad++ ?  Syntax highlighting  Tabbed document interface  Zooming  Indentation code  Find and replace over.
Commands Basic syntax of shell commands UNIX or shell commands have a basic structure command -options target command comes first (such as cd or ls) any.
Computing Fundamentals
Bare bones notes.
Chapter Four UNIX File Processing.
Microsoft PowerPoint 2007 – Unit 2
Lab 2: Terminal Basics.
Presentation transcript:

Setting Up and Managing a Bioinformatics Project Andrii Rozumnyi and Gagandeep Singh Spring 2016

Content Why do we need care about it? Basic Unix commands Project Directories and Directory Structures Project Documentation Automate File Processing Tasks Markdown for Project Notebooks

Why do we need care about it? Save our time on maintains Convenience & Consistency Collaboration Save time on reproducibility (by someone or by you over time) Easier to automate the file procesing So, it makes our scientific life a bit easier

Basic Unix commands man, help or info cd /home/BI/projects/zmays-snps/data/stats/qual.txt cd../data/stats/qual.txt mkdir dir-name ls touch shell expansion wildcards

Project Directories and Directory Structures All files and directories of your project should be in a single project directory with a clear name. Inside the main directory should be next folders: data (contains all raw and intermediate data) scripts (project-wide scripts) analysis (high-level analysis) $ mkdir project-name $ cd project-name $ mkdir data scripts analysis

Project Directories and Directory Structures Names of files and folders should be without spaces (use dashes or underscores instead) Better to include extensions in filenames Always use relative paths rather than absolute Divide Your Project into Subprojects

Project Documentation All the next information is the best stored in plain-text README files can easily be read, searched, and edited directly from command line Keep README files in each of your project’s at least in main directories Better to avoid formats like Microsoft Word (less portable) Easy copy and paste commands from document to command line Document the versions of the software that you ran

Project Documentation Document your methods and workflows Any command that produces results needs to be documented Include full command lines Describe any possible option to run the script Write default values in explicit manner (as they may change) Document the origin of all data in your project directory Where it was downloaded or who gave it to you Document when you downloaded data Record data version information Describe how you downloaded the data (which tools have you used)

Automate File Processing Tasks Automating file processing is an integral part of bioinformatics Using clear and consistent file naming schemes allow us to programmatically refer to files Shell Expansion Tips save our typing time in terminal ~, * brace expansion $ mkdir -p project-name/{data/seqs,scripts,analysis} project-name/data/seqs project-name/scripts project-name/analysis

Automate File Processing Tasks $ touch seqs/zmays{A,B,C}_R{1,2}.fastq $ ls seqs/ zmaysA_R1.fastq zmaysB_R1.fastq zmaysC_R1.fastq zmaysA_R2.fastq zmaysB_R2.fastq zmaysC_R2.fastq Wildcards are expanded to all matching file or directory names $ ls seqs/zmaysB* zmaysB_R1.fastq zmaysB_R2.fastq

Automate File Processing Tasks WildcardWhat it matches * Zero or more characters. ? One character. [A-Z] Any character between the supplied alphanumeric range (in this case, any character between A and Z); this works for any alphanumeric character range (e.g., [0-9] matches any character between 0 and 9) * Table 1. Common Unix filename wildcards * - for numerical range you should use curly braces ({10..15})

Automate File Processing Tasks While wildcards are powerful, they’re useless if files are inconsistently named! NB! Better to be as restrictive as possible with wildcards. If you want to process all zmaysB FASTQ files: 1.zmaysB* (worse as capture zmaysB-interesting-SNPs-found.xls etc.) 2.zmaysB*fastq (better)

Automate File Processing Tasks Leading Zeros and Sorting $ ls -l -rw-r--r-- 1 vinceb staff 0 Feb 21 21:24 genes-1.txt -rw-r--r-- 1 vinceb staff 0 Feb 21 21:24 genes-11.txt -rw-r--r-- 1 vinceb staff 0 Feb 21 21:24 genes-12.txt -rw-r--r-- 1 vinceb staff 0 Feb 21 21:24 genes-13.txt -rw-r--r-- 1 vinceb staff 0 Feb 21 21:24 genes-14.txt [...] $ ls -l -rw-r--r-- 1 vinceb staff 0 Feb 21 21:23 genes-001.txt -rw-r--r-- 1 vinceb staff 0 Feb 21 21:23 genes-002.txt [...] -rw-r--r-- 1 vinceb staff 0 Feb 21 21:23 genes-013.txt -rw-r--r-- 1 vinceb staff 0 Feb 21 21:23 genes-014.txt

Markdown Files

 TOOLS: Pandoc( Remarkable( Markdownpad(Windows)( dillinger.io (online)

 Used for keeping chronological information about your project. For example: Steps you have taken Information about why you have made decisions Relevant information to reproduce your work  Modern day equivalent to notebooks for keeping notes of the task we are performing.  Simple plain text format, which can be easily rendered to HTML or PDF Usage

Syntax  Special Symbols in Markdown: SymbolName \backslash `backtick *asterisk _underscore {}curly braces []square brackets ()parentheses #hash mark +plus sign -minus sign (hyphen).dot !exclamation mark

 Types of Elements  Block Level: * Block-level html elements-- e.g.,,,, etc.-- must be separated from surrounding content by blank lines. Example: This is a regular paragraph. Foo This is another regular paragraph.

o HEADERS :- Use #’s up to 6 to mark the headers or use = or – sign for marking headers up to level 2. Example: # This is an H1 ## This is an H2 ###### This is an H6 or This is an H1 ============= This is an H Optionally, you may close the # style header(s) o BLOCKQUOTES: Use > symbol to mark piece of text as a block quote.

o LISTS: Unordered: Use */+/single – for creating Unordered lists. Ordered lists use numbers followed by periods: Example(s):  Unordered List - Red - Green - Blue  Ordered List: 1. Bird 2. McHale 3. Parish

Questions: What will be the output?: Red 1. Green 1. Yellow was the warmest year.

 Span level Span-level HTML tags — e.g.,, or — can be used anywhere in a Markdown paragraph, list item, or header. Unlike block-level HTML tags, Markdown syntax is processed within span-level tags. o LINKS:  Inline This is [an example]( "Title") inline link. [This link]( has no title attribute.  Reference-style links use a second set of square brackets, inside which you place a label of your choosing to identify the link: This is [an example][id] reference-style link. Then, anywhere in the document, you define your link label like this, on a line by itself: [id]: "Optional Title Here"

o EMPHASIS Markdown treats asterisks (*) and underscores (_) as indicators of emphasis. Text wrapped with one * or _ will be wrapped with an HTML tag; double *’s or _’s will be wrapped with an HTML tag. E.g., this input: *single asterisks* _single underscores_ **double asterisks** __double underscores__ To produce a literal asterisk or underscore at a position where it would otherwise be used as an emphasis delimiter, you can backslash escape it: \*this text is surrounded by literal asterisks\*

o IMAGES: Markdown uses an image syntax that is intended to resemble the syntax for links, allowing for two styles: inline and reference.  Inline: ![Alt text](/path/to/img.jpg) ![Alt text](/path/to/img.jpg "Optional title")  Reference-style image syntax looks like this: ![Alt text][id] Where “id” is the name of a defined image reference. Image references are defined using syntax identical to link references: [id]: url/to/image "Optional title attribute"

Trimmed down Tutorial in pdf format for markdown files(Double click the blue box and pdf file will open)