1 Applied CyberInfrastructure Concepts ISTA 420/520 Fall 2013 1 Nirav Merchant Bio Computing & iPlant Collaborative Eric Lyons.

Slides:



Advertisements
Similar presentations
Introduction to Unix Shell & Scripting with csh/tcsh  Brief Unix History  Unix Shell & Flavor  CSH/TCSH Scripts.
Advertisements

1 Introduction to UNIX Ke Liu
NETW-240 Shells Last Update Copyright Kenneth M. Chipps Ph.D. 1.
Chapter Seven Unix Shell Environments1 System Programming UNIX Shell Environments.
More about Shells1-1 More about Shell  Shells (sh, csh, ksh) are m Command interpreters Process the commands you enter m High-level programming languages.
Linux+ Guide to Linux Certification, Second Edition
CS 497C – Introduction to UNIX Lecture 36: - Customizing the Environment Chin-Chih Chang
Guide To UNIX Using Linux Third Edition
Introduction to Unix (CA263) Introduction to Shell Script Programming By Tariq Ibn Aziz.
1 Applied CyberInfrastructure Concepts ISTA 420/520 Fall Nirav Merchant Bio Computing & iPlant Collaborative Eric Lyons.
1 Applied CyberInfrastructure Concepts ISTA 420/520 Fall Nirav Merchant Bio Computing & iPlant Collaborative Eric Lyons.
Linux Commands LINUX COMMANDS.
Nic Shulver, Scripts and Batch files Scripting in Windows and Linux What is scripting? What is it for? DOS/Windows batch files.
Using the Bash Shell. Linux Shell Options Linux provides a range of options for shells. bash –The GNU Bourne Again Shell (bash) bsh – The Bourne Shell.
Welcome to CSE  Name: Di Cao   Classroom: DL357  Class Time: T 8:30am - 9:18am  Office.
UNIX command line. In this module you will learn: What is the computer shell What is the command line interface (or Terminal) What is the filesystem tree.
Agenda User Profile File (.profile) –Keyword Shell Variables Linux (Unix) filters –Purpose –Commands: grep, sort, awk cut, tr, wc, spell.
CENT 305 Information Systems Security Linux Introduction.
Shell Features CSCI N321 – System and Network Administration Copyright © 2000, 2005 by Scott Orr and the Trustees of Indiana University.
CS 2061 Shells Also known as: Unix Command Interpreter.
UNIX command line. In this module you will learn: What is the computer shell What is the command line interface What is the directory tree Some UNIX commands.
Introduction to Unix – CS 21 Lecture 9. Lecture Overview Shell description Shell choices History Aliases Topic review.
1 UNIX essentials (hands-on) the directory tree running programs the shell → command line processing → special characters → command types → shell variables.
UNIX/LINUX Shells Shell is an UNIX/LINUX command interpreter. Shell command can be internal or external. The code to execute an internal command is part.
Essential Shell Programming by Prof. Shylaja S S Head of the Dept. Dept. of Information Science & Engineering, P.E.S Institute of Technology, Bangalore
LINUX System : Lecture 6 Shell Programming
Workbook 6 – Part 2 The Bash Shell
Users Greg Porter V1.0, 26 Jan 09. What is a user? Users “own” files and directories Permission based on “ownership” Every user has a User ID (UID) 
UNIX/LINUX SHELLS.  “A Unix shell is a command-line interpreter or shell that provides a traditional user interface for the Unix operating system and.
Introduction to Unix Shell & Scripting with csh/tcsh  Brief Unix History  Unix Shell & Flavor  CSH/TCSH Scripts.
1 Applied CyberInfrastructure Concepts ISTA 420/520 Fall Nirav Merchant Bio Computing & iPlant Collaborative Eric Lyons.
Unix/Linux cs3353. The Shell The shell is a program that acts as the interface between the user and the kernel. –The shell is fully programmable and will.
Unix Shell Environments February 23rd, 2004 Class Meeting 6.
1 Applied CyberInfrastructure Concepts ISTA 420/520 Fall Nirav Merchant Bio Computing & iPlant Collaborative Eric Lyons.
UNIX shell environments CS 2204 Class meeting 6 *Notes by Doug Bowman and other members of the CS faculty at Virginia Tech. Copyright
User Environments Objectives –to provide appropriate environments for different types of users Contents –different login programs –user profiles –restricted.
UNIX shell environments CS 2204 Class meeting 4 Created by Doug Bowman, 2001 Modified by Mir Farooq Ali, 2002.
1 Applied CyberInfrastructure Concepts ISTA 420/520 Fall Nirav Merchant Bio Computing & iPlant Collaborative Eric Lyons.
Introduction to Programming Using C An Introduction to Operating Systems.
1 Applied CyberInfrastructure Concepts ISTA 420/520 Fall Nirav Merchant Bio Computing & iPlant Collaborative Eric Lyons.
Unix – Linux CS3353 Ssystem Administration. OS The Operating System – Acts as the interface to all software, hardware, and users of a computer. – Requires.
UNIX Shell Dr. Tran, Van Hoai
Advanced Programming in the UNIX Environment Hop Lee.
Lesson 3-Touring Utilities and System Features. Overview Employing fundamental utilities. Linux terminal sessions. Managing input and output. Using special.
Module 1 - Introduction to Linux. Users must log-in Linux is case sensitive File and Directories naming conventions (No spaces!) Files and Directories.
Chapter 8: The Bourne Again Shell It’s a command interpreter, it’s a programming language, and it makes a mean martini.
Jozef Goetz, expanded by Jozef Goetz, 2006 Credits: Parts of the slides are based on slides created by textbook authors, Syed M. Sarwar, Robert.
1 CS3695 – Network Vulnerability Assessment & Risk Mitigation – Introduction to Unix & Linux.
Configuration your environment Many user-configurable Unix programs (such as your shell) read configuration files when they start up. These configuration.
Agenda Customizing a Unix/Linux account Environment Introduction to Start-up Files (.bash_profile,.bashrc,.profile,.kshrc) Safe Methods for Changing Start-up.
Introduction to LINUX command line Allegra Via IBBE, National Research Council, Italy Sapienza Università di Roma, Italy EMBO Practical Course on Computational.
MaSH – Miniature Shell ● Brief introduction to shell. – What is a shell? ● A program that reads inputs from the keyboard and executes the commands. ● Basic.
1 Applied CyberInfrastructure Concepts 1 Nirav Merchant Bio Computing & iPlant Collaborative Eric Lyons
Department of Computer Engineering
SUSE Linux Enterprise Desktop Administration
Introduction to Shells
Shell Features CSCI N321 – System and Network Administration
Andy Wang Object Oriented Programming in C++ COP 3330
CAP652 Lecture - Shell Programming
LINUX System : Lecture 5 (English-Only Lecture)
Applied Cyber Infrastructure Concepts Fall 2017
John Carelli, Instructor Kutztown University
Unix : Introduction and Commands
CSCI The UNIX System Shell Startup and Variables
Andy Wang Object Oriented Programming in C++ COP 3330
Linux Shell Script Programming
Unix Shell Environments
Introduction to Unix Shell & Scripting with csh/tcsh
Linux Commands LINUX COMMANDS.
Bash Scripting CS 580U - Fall 2018.
Presentation transcript:

1 Applied CyberInfrastructure Concepts ISTA 420/520 Fall Nirav Merchant Bio Computing & iPlant Collaborative Eric Lyons Plant Sciences & iPlant Collaborative University of Arizona or Will Computers Crash Genomics? Science Vol 331 Feb 2011

Tasks for today  Managing your VM  Add user, permission, security considerations etc.  Understanding where the files are  Terminal, editors etc  Shell and scripting  Start building your “Data Science ToolBox”

Step #1 for Big Data Toolkit  Command line competency

Permissions  Why do you need them ?  What is a ACL (Access Control List) ?  The UNIX model of permissions (next slides are from Greg Wilson at carpentry.org) carpentry.org  Path statement and finding things

Has unique user name and user ID user

Has unique user name and user ID User name is text: "imhotep", "larry", "vlad", … user

Has unique user name and user ID User name is text: "imhotep", "larry", "vlad", … User ID is numeric (easier for computer to store)

usergroup

usergroup Has unique group name and group ID

usergroup Has unique group name and group ID User can belongs to zero or more groups

usergroup Has unique group name and group ID User can belongs to zero or more groups List is usually stored in /etc/group

usergroupall

usergroupall Everyone else

usergroupall Has user and group IDs

usergroupall read

usergroupall read write

usergroupall read write execute

usergroupall read ✔✔✗ write ✔✗✗ execute ✗✗✗

usergroupall read ✔✔✗ write ✔✗✗ execute ✗✗✗ File's owner can read and write it

usergroupall read ✔✔✗ write ✔✗✗ execute ✗✗✗ File's owner can read and write it Others in group can read

usergroupall read ✔✔✗ write ✔✗✗ execute ✗✗✗ File's can read and write it Others in group can read That's all

Where are my files ? Understanding layout of data –Home –Root –Tmp Permissions Storage space and planning for it Managing runaway items (more in next class)

Security considerations Update your OS (how can you do that ?) Why you should NEVER run as root (how do I add a user ?) Password and keys (and dual factor) Ssh foo

What is Shell? Shell is –Command Interpreter that turns text that you type (at the command line) in to actions: –User Interface: take the command from user Programming Shell can do –Customization of a Unix session –Scripting –Many Many automation steps

What is Shell? Shell is –Command Interpreter that turns text that you type (at the command line) in to actions: –User Interface: take the command from user Programming Shell can do –Customization of a Unix session –Scripting –Many Many automation steps

Customization of a Session Each shell supports some customization. –User prompt –Where to find mail –Shortcuts (alias) The customization takes place in startup files –Startup files are read by the shell when it starts up –The Startup files can differ for different shell

Popular Shells  sh Bourne Shell  ksh Korn Shell  csh,tcsh C Shell (for this course)  bash Bourne-Again Shell

Flavors of Unix Shells Two main flavors of Unix Shells –Bourne (or Standard Shell): sh, ksh, bash, zsh Fast $ for command prompt –C shell : csh, tcsh better for user customization and scripting %, > for command prompt To check shell: –% echo $SHELL (shell is a pre-defined variable) To switch shell: –% exec shellname (e.g., % exec bash)

Startup files and why you should care  bash: /etc/profile (out-of-the-box login shell settings) /etc/bash.bashrc (out-of-box non-login settings) /etc/bash.bashrc.local (global non-login settings) ~/.bash_profile (login shell user customization) ~/.bashrc(non-login shell user customization) ~/.bash_logout (user exits from interactive login shell)

Some Special Keys How do you invoke tcsh ? Ctrl-U = Delete everything on the command- line Ctrl-A = Move cursor to the front Ctrl-E = Move cursor to the end Ctrl-P = Set the current command-line to the previous command Ctrl-N = Set the current command-line to the next command TAB = Filename completion

Preview pieces of toolbox We will work though Step 5 and go straight to commands

Next class  Preparing to play with your data set – Can you download a piece of it ?  Learn about space and process management  Introduction to shell scripting and automation  Start building your Big Data command line tool kit