The A – Z of Python Scp

In this era of technological advancement, programming plays an important role. And python has always proved as one of the most versatile programming languages. It covers a wide variety of domains with utmost versatility, it is always the programmer’s favorite. One of the primary reasons for such tremendous success of the python programming language is that it has many modules that help programmers cover such a wide range of domains. One of such open-source libraries is SCP. In this article, we are going to discuss the python SCP library.

What is Scp.py?

Scp.py is an open-source python library used to send and receive files between client and server using the paramiko transport. It provides us the programming interface to use the SCP1 protocol, which lets a server and client have multiple conversations over a single TCP connection.

Scp module has several methods that provide different functionality of sharing file resources between servers and clients. We will discuss each of them and then see demonstrations to understand it more clearly. Before that, let’s see its installation.

Installing Scp Module

To install the SCP module, open CLI and use the following command.

pip install scp

Once the installation is completed, we are ready to use it. Now, it’s time to see some examples to understand it.

Example 1: Establishing connection and sending and downloading text file

from paramiko import SSHClient
from scp import SCPClient

# instantiating client object and connecting it to server
ssh = SSHClient()
ssh.load_system_host_keys()
ssh.connect('example.com')

# passing SSHClient transport as the argument to SCPClient
scp = SCPClient(ssh.get_transport())

# Sending files to the server
scp.put('test.txt', 'test2.txt')
# Downloading files to the server
scp.get('test2.txt')

# Uploading the 'test' directory with its content in the
# '/home/user/dump' remote directory
scp.put('test', recursive=True, remote_path='/home/user/dump')

Explanation

So, in the above example, we imported SSHClient from paramiko module to establish the connection between client and server. After that, we imported the SCPClient from the SCP module to define the protocol for file sharing. Once we import it, we instantiate the client object and connect it through the server. After that, we instantiate the SCP client object, which takes SSHClient transport as the argument. Once we define the things mentioned above, we send our file using the put() method, which takes the file name as the argument.

Similarly, we used the get() method to download the file from the server. In the other part of the code, we sent the whole directory named ”test” from our local computer to the server. We can also specify the path as the argument for the server computer using the keyword argument named remote_path.

Key Points:

put(): This method sends/uploads files from client computer to server computer.
get(): This method is used to get/download files from server computer to client computer.

Trending

[Fixed] typeerror can’t compare datetime.datetime to datetime.date

Example 2: Uploading File-like Object

we can also upload file-like objects using SCPClient. To do that, we use putfo() method. Let’s see an example.

import io
from paramiko import SSHClient
from scp import SCPClient

ssh = SSHClient()
ssh.load_system_host_keys()
ssh.connect('example.com')

# SCPCLient takes a paramiko transport as an argument
scp = SCPClient(ssh.get_transport())

# generate in-memory file-like object
fl = io.BytesIO()
fl.write(b'test')
fl.seek(0)
# upload it directly from memory
scp.putfo(fl, '/tmp/test.txt')
# close connection
scp.close()
# close file handler
fl.close()

Explanation

In the above example, we initialized the BytesIO object for streaming data in the form of bytes. After that, we mentioned what we wanted to write and the insertion position (from the 0th position in our case). After that, we used putfo() method to send our data stream and location of the file. Once done, we close our connection to free resources from our main memory.

Key Points:

putfo(): This method is used to send/upload files from client computer to server computer in the form of file-like object.

Example 3: Tracking progress while Uploading and Downloading

We can also track our progress using the progress function while uploading and downloading data. Let’s see the example.

from paramiko import SSHClient
from scp import SCPClient
import sys

ssh = SSHClient()
ssh.load_system_host_keys()
ssh.connect('example.com')

# Define progress callback that prints the current percentage completed for the file
def progress(filename, size, sent):
    sys.stdout.write("%s's progress: %.2f%%   \r" % (filename, float(sent)/float(size)*100) )

# SCPCLient takes a paramiko transport and progress callback as its arguments.
scp = SCPClient(ssh.get_transport(), progress=progress)

# you can also use progress4, which adds a 4th parameter to track IP and port
# useful with multiple threads to track source
def progress4(filename, size, sent, peername):
    sys.stdout.write("(%s:%s) %s's progress: %.2f%%   \r" % (peername[0], peername[1], filename, float(sent)/float(size)*100) )
scp = SCPClient(ssh.get_transport(), progress4=progress4)

scp.put('test.txt', '~/test.txt')
# Should now be printing the current progress of your put function.

scp.close()

Explanation

In the above example, we implemented a progress function that writes the progress of the shared file. We need to mention it as the positional argument that initializes the SCPClient class object.

Key Points:

progress(): The user-defined function tracks the progress while sharing the file.

Python SCP create directory

While sending files to remote computers, SCP can also create the directory if mentioned. If the directory exists, it sends the file to that directory. However, if the directory doesn’t exist, it will create the directory and save the file there. The point to note here is that it will not create the complete directory structure but only the last directory if it doesn’t exist.

from paramiko import SSHClient
from scp import SCPClient

# instantiating client object and connecting it to server
ssh = SSHClient()
ssh.load_system_host_keys()
ssh.connect('example.com')

# passing SSHClient transport as the argument to SCPClient
scp = SCPClient(ssh.get_transport())

# '/home/user/dump' remote directory
scp.put('test', recursive=True, remote_path='/home/user/dump')

Python SCP check if file exists

We can also check whether a file exists or not on the remote server. To do that, we need to use paramiko’s SFTP client. We first navigate to the directory in which we want to search and then use the stat() method to check whether the file exists. Let’s see an example.

import paramiko
import getpass

# make a local test file
open('deleteme.txt', 'w').write('you really should delete this]n')

ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
try:
    ssh.connect('localhost', username=getpass.getuser(),
        password=getpass.getpass('password: '))
    sftp = ssh.open_sftp()
    sftp.chdir("/tmp/")  # chnaging to directory we want to search
    try:
        print(sftp.stat('/tmp/deleteme.txt')) # checking whether file exists or not
        print('file exists')   
    except IOError:
        print('copying file')
        sftp.put('deleteme.txt', '/tmp/deleteme.txt')
    ssh.close()
except paramiko.SSHException:
    print("Connection Error")

Trending

[Solved] runtimeerror: cuda error: invalid device ordinal

FAQs on Python SCP

Conclusion

So, today in this article, we learned about the SCP python module. We saw how we could use the different functions available in the module.