K
Q

Can we see transfer progress with kubectl cp?

January 21, 2019

Is it possible to know the progress of file transfer with 

kubectl cp
for Google Cloud?

-- akshitsethi
kubernetes
google-kubernetes-engine

7 Answers

January 21, 2019

No, this doesn't appear to be possible.

kubectl cp
appears to be implemented by doing the equivalent of

<!-- language: sh --> 
kubectl exec podname -c containername \
  tar cf - /whatever/path \
| tar xf -

This means two things:

  1. tar(1) doesn't print any useful progress information. (You could in principle add a 

    v
    flag to print out each file name as it goes by to stderr, but that won't tell you how many files in total there are or how large they are.) So 
    kubectl cp
    as implemented doesn't have any way to get this out.

  2. There's not a richer native Kubernetes API to copy files.

If moving files in and out of containers is a key use case for you, it will probably be easier to build, test, and run by adding a simple HTTP service. You can then rely on things like the HTTP 

Content-Length:
header for progress metering.

-- David Maze
Source: StackOverflow
October 8, 2020

One option is to use pv which will show time elapsed, data transferred and throughput (eg MB/s):

$ kubectl exec podname -c containername -- tar cf - /whatever/path | pv | tar xf -

14.1MB 0:00:10 [1.55MB/s] [       <=>                                   ]

If you know the expected transfer size ahead of time you can also pass this to pv and it will then calculate a % progress and also an ETA, eg for a 100m transfer:

$ kubectl exec podname -c containername -- tar cf - /whatever/path | pv -s 100m | tar xf -

13.4MB 0:00:09 [1.91MB/s] [==>                          ] 13% ETA 0:00:58

You obviously need to have pv installed (locally) for any of the above to work.

-- Freshleaf Media
Source: StackOverflow
April 27, 2022

On MacOS, there is still the hacky way of opening the "Activity Monitor" on the "Network" tab. If you are copying with 

kubectl cp
from your local machine to a distant pod, then the total transfer is shown in the "Sent Bytes" column. Not of super high precision, but it sort of does the job without installing anything new.

enter image description here

-- Jonathan Lurie
Source: StackOverflow
July 25, 2020

I figured out a hacky way to do this. If you have bash access to the container you're copying to, you can do something like 

wc -c <file>
on the remote, then compare that to the size locally. 
du -h <file>
is another option, which gives human-readable output so it may be better

-- Marcus Buffett
Source: StackOverflow
January 21, 2019

It's not possible, but you can find here how to implement rsync with kubernetes, rsync shows you the progress of the transfer file. rsync files to a kubernetes pod

-- javierlga
Source: StackOverflow
October 19, 2022

I know it doesn't show an active progress of each file, but does output a status including byte count for each completed file, which for multiple files run via scripts, is almost as good as active progress:

kubectl cp local.file container:/path/on/container --v=4

Notice the --v=4 is verbose mode and will give you output. I found kubectl cp output shows from v=3 thru v=5.

-- Andrew
Source: StackOverflow
July 11, 2024

Showing off a hacky way to achieve this via a hidden flag in scp:

kubectl exec pod-name -- sh -c 'file=/tmp/test1; name=$(basename $file) ; size=$(wc -c $file | cut -d " " -f 1); echo "C0644 $size $name"; dd if=$file bs=8192 2>/dev/null; printf "\0"' | scp -t /tmp

This will copy the file(

/tmp/test1
) in the container to your 
/tmp
directory with mode 644, with a nice progress bar and speedometer. If you want you can probably read the file mode and set it accordingly in the first line of the SCP message as well.

But we really should be waiting for this PR to be addressed.

ref:

http://nanozen.snert.org/how-the-scp-protocol-works/

-- xiaket
Source: StackOverflow