Curl Commands

 

1) Curl Command to download the data from a internet site.

curl  -O https://testserver.com.au/file.zip 

sometimes if secure option is enabled it will not allow to download the file. We may get error

curl failed the legitimacy of the server and therefore could not establish a secure connection to it.  so to fix this we use -k option.

curl  -k -O https://testserver.com.au/file.zip 

 2) to check curl version, below command will be used

curl --version

3) To download the files from Mainframe we can use curl command. First we need to configure netrc file which contains user name and password to connect mainframe file.

curl -netrc --ftp-ssl --use-ascii ftp://mainframeserver.com/"'AB.G.C'" -o /var/opt/sw/Files/output.dat

4) To upload the files to Mainframe we can use curl command. First we need to configure netrc file which contains user name and password to connect mainframe file.

curl -netrc --ftp-ssl --use-ascii -quote "site RDW LRECL=200 RECFM=FB CYLINDERS PRIMARY=4000 SECONDARY=400" --upload-file /var/opt/sw/Files/output.dat ftp://mainframeserver.com/"'AB.G.C'"  

5)  Curl Command to use verbose mode a internet site.

curl  -verbose -O https://testserver.com.au/file.zip 

6)Curl Command to use ntlm for download file and place in specific directory

curl  -k -ntlm -netrc  https://testserver.com.au/file.zip -o /var/opt/sw/Files/output.dat

UNIX File Zipping and unzipping Commands

There are various file compression and un compression commands in UNIX.

1) Unzip the gz file we wil use the gunzip command

 

if we need to see the contents of the file without unzipping the file we can use -c option. This will enable to view file without unzipping

gunzip -c filename.gz

need  The below command use to uunzip the file completly

gunzip filename.gz

2) to Zip the txt,csv,dat etc file into gz format we wil use the gzip command

gzip  file.txt 

The above command will convert the file.txt to file.txt.gz format.

Also we can zip the file 

3) To compress the folder and its contents we use the tar command

tar -cvf filename.tar /opt/sw/SrcFiles/backup

The above command compress the folder and its contents that include its subfolders into a single tar file.

tar -xvf filename.tar

The above command un-compress the folder and its contents that include its subfolders into a single tar file.

4) We can use the unzip command to unzip the .zip file for that we need to install the 7zip software in unix.

unzip file.zip

Unix Server health check commands

 

1) To check the top process used in Unix server 

topas 

Above command will provide details CPU Usage , order top processed based on their CPU usage in descending order.

2) To check the unix OS version used in Unix server 

oslevel -s

3) To check the unix ssl version 

lslpp -l | grep -i opessh

4) Memory related commands

vmstat

provides the system configuration. No of logical cpus, current ram allocated, entitle ment of CPU etc.

svmon -G

Provides the details of usage of memory usage and free space in server memory.

lparstat -i |egrep 'Memory|CPU'

Provides the details of no of virtual CPU , Max CPU,Min CPU, Max and Min memory , Active Cpus etc.

4) To check process in the unix

ps -ef | grep processname

to check the idle process

ps -aux

to check the Zombie process below command can be ued

ps -el | grep 'Z'

to kill the process

kill -9 <processid>

to get user user process

ps -u <userid>

5) To check the logical partition we can use the command

laparstat

6) To check the host name we can use the hostname command

and once we get hostname we can idetify ip address using the below command

nslookup servername

Unix Commands : File Size commands

 There are various command that can be used to check the file size in UNIX.

1) Finding the Largest directory in a particular mount point

du -sk . [!.] *  * | sort -n

2) to check the size of partiular mount pount

df -g | grep mountpoint

mount point is the storage path like /opt/sw/Unix/dev

3) To check space for all mount points

df -g 

g - stands for size in giga bytes.

Unix Commands : Listing the files

 

For listing the files in Unix, we can use the command

ls  -  ls will allow you to list the files and folder in a particualr permissions

For see the files full details like last modfied date , file owner , file groups -lrt can be used

ls -lrt or ls -ltr can used used tol files with file owner ,file group and last modified stamp

For listing the hidden files in Unix the below command -a can used be along with -lrt

ls -lart  - This will all files inlcude hidden files in a directory

To see what are the folders that has extened permissions aprart from file owner and file groups below command can be used.

ls -le  - This will display the directory that has extened permission.

After identifying the extened permission, For updating the group or add extra group

acledit an be used.

acledit directory

head :  used to for get top rows in a file

usage :  head -10 file.txt

The above command will fetch first 10 rows from a file file.txt

tail :  used to for get last rows in a file

usage :  tail -10 file.txt

The above command will fetch last 10 rows from a file file.txt

To check the number of lines wc command will be used

wc -l filename.txt  (filename.txt is the filename we need to check the number of lines)

To list the files with specific text in a folder 

grep -Ril "search keyword"   "filepath"

with two arguments like search keyword and filepath we can find the keyword that is used in all files.

UNIX File Operations and User permission Commands

 
1) Vi editor

ESC + ! + wq   -- This for saving the file
ESC + ! + q  --This will abort current changes
CNTRL +Z  also works for aborting the changes but it is not proper way to abort.
shift + G  -- Will come to the end of the file
dd -- typing two times in vi editor will delete the enitre line.
d -- only one time d will delete the one letter
i or insert button will be used for insert option. By default vi editory will be in edit mode. 
vi -R file.txt -- Opens the file.txt in read only mode

2) Change file group of a folder 

chgrp usergroup foldername
chgrp  -- Command name
usergroup -- user group that will be changed to
foldername -- folder details that need to be changed

3) to check the permission of a user

id userid
id -- Command
userid- any user login that need to check for their permission

4) to check the default group of user

id -Gn userid
id -- Command
Gn -- Group name
userid- any user login that need to check for their permissions

5) to check the current user groups

groups  -- command will be used to check current user groups

6) to check the full user details from passwd file.

cat /etc/passwd | cut -d: -f5
The above command gives all user details in Unix server.

6) to check the user full information finger command can be used

finger -- Command
finger username  -- This gives infromation about current user login name and home path and from which servers its connected.

7) to change the permission of file

chmod 775 filename
There are three 3 modes of permissions 

8) find command will be used to find the file in a folder or directory

find /var -xdev -type f -size +300000c -exec ls -al {} \; | sort -k5nr | head -50
The above command fetches the files from the mount point /var with files greater that 300000kb that included hidden files and then sort the files to retreive the 50 large files. 
-type :  There can be f (file) or d (directory)
ls :  for listing files 
sort :  for sorting the order
head:  for limiting the record after sorting
find command will be used to find the file in a folder or directory
find /var  -type f -name bkp* -exec ls -al {} \; | sort -k5nre | head -50
The above command fetches first 50 the files that starts with bkp name
Similary find can used with most of the file operation commands 
find /var -type f -name bkp* -exec chmod 775 {} \;
The above command changes permission of the all the files starts with bkp to rwxrwxr-x (775) .
find /var -type f -name bkp* -exec chmod 770 {} \;
The above command changes permission of the all the directories starts with bkp to rwxrwx--- (770) .
If the current user who executing the above command does not have permission then it will not change and will get permission denied error.

9) Removing the files 

For Removing the files we can use rm command
rm file.txt
Above command will remove the file.txt. For removing directory we can 
use rmdir command
rmdir testdir
Above command will will remove directory testdir. This will work only when directory is empty.
if we need to remove recursivly with subfolders and files we can use
rm -fR testdir
Above command will will remove directory testdir. This will remove the full directory. if there is any files inside directoy in which user does not have permission then it will skip that files and direcotry will not be removed.
find /var -type f -name bkp* -exec rm -fR {} \;
Above command will remove the files that starts with bkp in all directories. 
find /var -type f -name *.txt  -mtime +30  -exec rm -fR {} \;
Above command will remove the files that has txt extension which is older than 30 days from all the directories. 

10) Replace the keyword in a file  or replace the line

sed : sed command will be used for replacing the text
sed -e 's/orginal keyword/replacing keyword/g' inputfile.txt

example
-------- 
sed -e 's/abc/cda/g' inputfile.txt > newfile.txt
above command replace the text abc in the file inputfile.txt with cda and put into newfile.txt file.

to replace old line with new line and copy the updated file to new file.

sed -e 's:opt/sw/abc.txt:opt/sa/tcb.txt:g' inputfile.txt > newfile.txt

11) moving the file from one place instead of copying

For moving the file we use the mv command. When you use mv command the actual file will be moved. The permissions and file creation timestamp will not be changed it will be as original file. 
mv  filename1.txt filename2.txt
In the above command it will rename filename1.txt to filename2.txt
mv /opt/sw/ss/filename1.txt /opt/sw/filename1.txt
The above command will move file from /opt/sw/ss/ drirectory to /opt/sw/ directory.

To move the entire directory we can use mvdir command

mvdir /opt/sw/ss/dir1 /opt/sw/dir1

Sending mail using Unix commands

 

1) uuencode filename.txt | mailx -s "filename details"  -c abc@test.com -r noreply@test.com sender@test.com

2) mail -s "Test mail" abc@test.com < TestFile.txt

3)  (echo "Hi All, \n Please find the attached details in the email \n Thanks \n Application Team"; uuencode File.txt attahedFile.txt) | mailx -s "File details" -c abc@test.com -r noreply@test.com sender@test.com

UNIX Shell Commands : File copy using various commands

 

1) SFTP Commands in Unix

The basic command used for connecting remote server using sftp in Unix

sftp username@servername

username:  Username for the remote server. This user should be created in remote server should have Public/private key present in the remote server.

servername :  server is the remote servername we need to connect.

After connecting to the sftp server, basically we will perfrom two operations 

a) Pulling the file from remote server.  For this we will use the get command

 get :  get will fecth the file from remote directory and place it to the current directory into the local server.

get will have two parameters . First parameter is mandatory which remote filepath and second parameter local directory path. if we do not provide the second variable it will copy int present directory

i ) get /opt/sw/unix/Remote/Test.txt  

ii) get /opt/sw/unix/Remote/Test.txt  /opt/sw/unix/local/Test.txt

The above commands will copy only the file. What if you need to copy the entire directory from remote server to local server using sftp command, We have abiltiy to copy enitre directory using -r option.

i ) get -r /opt/sw/unix/Remote

ii) get  -r /opt/sw/unix/Remote  /opt/sw/unix/local

 put :  put will place the file in remote directory from the current directory into the local server.

put will have two parameters . First parameter is mandatory which filepath and second parameter remote directory path. if we do not provide the second variable it will copy into present directory in remote server

i ) put /opt/sw/unix/Remote/Test.txt  

ii) put /opt/sw/unix/Remote/Test.txt  /opt/sw/unix/local.Test.txt

The above commands will place only the file. What if you need to place the entire directory from local server to remote server using sftp command, We have abiltiy to place enitre directory using -r option.

i ) put -r /opt/sw/unix/local

ii) put -r /opt/sw/unix/local/  /opt/sw/unix/remote/

Above will copy the local directory

2) Copy Commands in Unix

The basic command used for Copying file from one server to other server. It accepts two parameters.

First parameter is madatory which source file. Second parmeter is target path.

cp /opt/sw/sw/SourcePath/Testfile.txt  .

Here . represents local path which is current direcotry. Above command will copy file to current directory. 

cp /opt/sw/sw/SourcePath/Testfile.txt /opt/sw/sw/TargetPath/Test.txt

This comand will copy file from /opt/sw/sw/SourcePath/ to  /opt/sw/sw/TargetPath/ Path

We can copy the entire directory using -R option. This will copy all directory files and folders inside the current directory

cp  -R /opt/sw/sw/SourcePath/   /opt/sw/sw/TargetPath

The above command will copy the SourcePath directory to TargetPath directory

3) Move Commands in Unix

The basic command used for moving file from one server to other server. It accepts two parameters.

First parameter is madatory which source file. Second parmeter is target path.

mv /opt/sw/sw/SourcePath/Testfile.txt  .

Here . represents local path which is current direcotry. Above command will move file to current directory. 

mv /opt/sw/sw/SourcePath/Testfile.txt /opt/sw/sw/TargetPath/Test.txt

This comand will move file from /opt/sw/sw/SourcePath/ to  /opt/sw/sw/TargetPath/ Path

We can move the entire directory using -R option. This will move all directory files and folders inside the current directory

mv -R /opt/sw/sw/SourcePath/   /opt/sw/sw/TargetPath

The above command will move the SourcePath directory to TargetPath directory

4) File Copy using scp in Unix

 The scp command can be used to copy securly in Unix. This is same as Copy command but copies the files securly to remote server. This will be used purely for copying the files.

scp -rp /opt/sw/sw/SourcePath/file.txt  username@remoteserver:/opt/sw/sw/TargetPath

The above command copy file from current seever to remote user and retain same permission as source server. (-rp for replicating same permission of source server)

to copy the files into current server from remote server we can use the below command

scp -rp  username@remoteserver:/opt/sw/sw/SourcePath/file.txt     /opt/sw/sw/TargetPath/