2. Part 3 – Remote Connection, File Transfer, Remote Environments
Unix Shell Workshop
Part 3 – Remote Connection, File Transfer, Remote Environments

3. Remote Connection
Remote connections in the terminal are made over Secure Shell (ssh)
A connection is formed from two parts, Username and Host Name
Username identifies you on the host, there must be a user for you on the host in order to login to it
Host Name is the remote machine or server that your trying to connect to

4. Remote Connection Let’s connect to LSUS’s Sun server
First and foremost is the ssh command, since we’re connecting via secure shell
Following that is our username
We separate the username and host name with symbol
Lastly we have the host name, note that the host name is not necessarily a name, but may be an IP address instead.

5. Remote Connection
If you haven’t connected to this host before ssh will first prompt you to continue or not and will add the host to known hosts if you do
This is a security feature that will recheck the host’s identification each time to make sure the host is who it says it is.
After successfully logging in you’ll need to change your password with the passwd command.

6. Remote Connection
Working in a remote host works the same as a local terminal. Remember the commands for working with files in general.
ls – list all files and directories in the location
mkdir – creates a directory
rmdir – removes a directory, must be an empty directory
rm – removes files, -r removes directories and all files inside them
mv – moves files and can optionally rename files
nano – file editor for opening and editing a file

7. Aliasing
Aliasing helps make our life easier by setting up commands that do a lot with little typing.
The most frequent use of aliasing is to attach option flags to commands we regularly use.
To make aliases that will persist across log ins we set them up in the bashrc file or bash_alias file. Both of these run automatically each time you log in.

8. Aliasing We’ll be setting the aliases in the ~/.bash_rc file.
Here’s a few we’ll likely want to setup:
alias ls = ‘ls -al’ – using just ls will now display all files with extended info
alias rm = ‘rm -i’ – make rm prompt before deleting something
alias nano = ‘nano -c’ – give us a line counter to know where issues occur

9. File Transfer
Working in a remote host is separate from your local machine, including your files. We need to transfer files we intend to use over to the remote host.
Transferring files is done with Secure Copy (SCP).
The structure of an scp command is as follows:
Alternatively, you and pull a file from the remote host by switching the order

10. File Transfer
Similar to the rm command, in order to transfer a directory and all its contents you up use the -r flag like so:
scp -r dataFolder
If needed it is also possible to transfer files between two remote hosts like so:
scp -r dataFolder

11. File Transfer
File transfer can also be handled in separate software such as FileZilla or WinSCP.
Both provide simple clean interfaces for connecting to and transferring files to and from remote hosts.

12. File Transfer
For connecting to Sun with FileZilla we need to use a saved Profile.

13. File Transfer For Sun in particular, The protocol should be SFTP
with a normal login type.

14. File Transfer With the profile saved you can now connect using it.
On a successful connection, the remote host’s files will appear on the right side and the local files on the left.

15. File Transfer
Transfering files with FileZilla is as simple as drag and drop or you can use right click to select multiple options.
You also have access to the compete file structure, allowing you to quickly look through your directories and select out particular files you want.
Files can also be edited with local software on a temporary copy of the file. When done editing and saved FileZilla will prompt you to upload the file back to the remote host (Recall local files are separate from remote host files, even in FileZilla).

16. File Considerations
There are a few things to keep in mind when moving files onto a remote linux host.
Windows users will frequently encounter a recurring problem in linux, end of file and end of line markings.
In windows these are invisible codes that denote the next line or end of the file.
In linux these show up as literal text that interfere with all sorts of things.
To make sure these get removed the command dos2unix can be run on the file to remove these markings.

17. File Considerations
Different files types are not necessarily human readable or can be edited.
Binary files in particular are a special type of file that are not human readable and as such get handled differently.
Trying to run dos2unix accidently on a binary file will return an error that binary symbols are being seen and will ignore that file.
Files that are frequently stored as binary include images, PDFs, and compiled code.
In order to view the contents of a binary file, transfer it to a different machine, such as your local machine that can view the file properly. This doesn’t work for everything, such as compiled code.

18. File Permissions
Files permissions are important for understanding what you can do with a file and what others have access to as well with that file.
Permissions are divided for 3 groups, User, Group, and Everyone.
The allowable permissions are Read, Write, and Execute.

19. File Permissions
Setting up good file permissions is important for keeping work private or setting up collaborative work.
To change the permissions on a file you use chmod like so:
chmod 700 filename
Each digit represents the group, and the value of that digit assigns the permission.
Permissions are assigned by the digits 1 – Execute, 2 – Write, 4 – Read.
In order to assign multiple permissions you add the numbers you need together: = read and write, 7 = read, write, execute … etc.

20. Background Processes
We frequently need to run a lot of jobs, and some of these take a lot of time to complete. To spare us from having to open multiple terminal connections we can put these jobs in the ‘background’.
After a job has been started use ctrl+z to pause the process then bg to place the process in the background.
You can check the status of jobs with the ‘jobs’ command.
A job can be terminated with the ‘kill’ command on it’s job number like so: kill %1
To start a job in the background from the beginning, add an & sign at the end of the command.

21. Error Checking
Things regularly go wrong, and knowing how to find the problem and fix it will be important.
Earlier we turned on nano’s line counter which tells us which line our cursor is currently on.
With the help of the line counter we can quickly identify where errors are in our code.

22. Error Checking Example:
Here we have two errors in my moleculeSearch.sh script, one says it couldn’t find a matching set of quotes and the other says it reached the end of the file unexpectedly.

23. Error Checking Here’s the script from our example:
Just as the error said, we’re missing matching quotes behind our H on line 8
The error referring to line 10 is the confusion caused by the command not being complete

24. Final Questions
Any Questions?

25. Practical Exercise
To practice working with a remote environment we’re going to write a text file which will hold the names of folders we need to create, and then write a script to create new directories for the names held in the text file.
First on your own computer use notepad to create a text file with the names of our directories:

26. Practical Exercise
Transfer this text file either with scp or FileZilla to your sun account.
Next we’ll write a simple script on sun that creates all our directories:
While read line; do
mkdir –p ElementsFolders/$line
done < Elements.txt

27. Practical Exercise
This script will take a text file names Elements.txt , read each entry in that file line by line, and create a directory under the parent directory ElementsFolders named by whatever was on that line.
It is good practice to give your scripts executable permissions.
Any Errors? Check the line number the error gives for something wrong.
Any strangely named directories? Make sure to run dos2unix before running our script, as your text file may still have end of file and end of line markings.

28. Practical Exercise
When your done try running an expanded version of the script that writes a text file in each folder:
i = 1
While read line; do
mkdir –p ElementsFolders/$line
echo “$i” > ElementsFolders/$line/$line.txt
$i = $i + 1
done < Elements.txt

29. Practical Exercise
When you’ve successfully run the previous script, try transferring the parent directory that contains everything back to your local machine by scp or FileZilla.
You’ve successfully worked in a remote environment, completed a task, and got back files that you want, creating the beginnings of a small work pipeline.