无论如何，有使用aws cli复制csv内部存在的文件列表吗？

由 eos_et发布于 2020-05-18 14:41:32

I have one folder called all-files on s3 bucket which has 35K files. I have one CSV file (errors.csv) which has 1K filenames and this filenames are sub-set of 35K filenames. I need to copy 1k files (which are from csv) from 35K all-files folder to some new folder called errors on same bucket.

反正有可能吗？如果是，怎么办？

I tried soemthing with cp command with include pattern but it seems does not support input from csv file.

ramet

2020-05-18 14:41:32

由于您不确定是否使用Linux，因此假设使用Linux。

The most basic way of doing this in bash is as follows (you haven't provided any example file content to work with):

# populate example `/tmp/files.csv` file:

cat >/tmp/files.csv << EOF
file1
file2
file3
EOF

Read line-by-line from /tmp/files.csv and execute the aws s3 mv:

while read file; do
  echo "aws s3 mv s3://<bucket>/${file} s3://<otherbucket>/${file}"
done < /tmp/files.csv

结果是：

aws s3 mv s3://<bucket>/file1 s3://<otherbucket>/file1
aws s3 mv s3://<bucket>/file2 s3://<otherbucket>/file2
aws s3 mv s3://<bucket>/file3 s3://<otherbucket>/file3

更复杂的示例包括通过同时移动几个文件来同时运行命令。但是，无论您同时移动10个文件还是一次移动10个文件，想法都是相同的。

eos_et

这家伙很懒，什么都没留下

积分
0
话题
0
评论
3213
注册排名
1512