无论如何，有使用aws cli复制csv内部存在的文件列表吗？

由 发布于 csvextractionamazon-s3amazon-web-servicesaws-cli
 收藏

I have one folder called all-files on s3 bucket which has 35K files. I have one CSV file (errors.csv) which has 1K filenames and this filenames are sub-set of 35K filenames. I need to copy 1k files (which are from csv) from 35K all-files folder to some new folder called errors on same bucket.

反正有可能吗？如果是，怎么办？

I tried soemthing with cp command with include pattern but it seems does not support input from csv file.

评论
ramet
ramet

由于您不确定是否使用Linux，因此假设使用Linux。

The most basic way of doing this in bash is as follows (you haven't provided any example file content to work with):

# populate example `/tmp/files.csv` file:

cat >/tmp/files.csv << EOF
file1
file2
file3
EOF

Read line-by-line from /tmp/files.csv and execute the aws s3 mv:

while read file; do
  echo "aws s3 mv s3://<bucket>/${file} s3://<otherbucket>/${file}"
done < /tmp/files.csv

结果是：

aws s3 mv s3://<bucket>/file1 s3://<otherbucket>/file1
aws s3 mv s3://<bucket>/file2 s3://<otherbucket>/file2
aws s3 mv s3://<bucket>/file3 s3://<otherbucket>/file3

更复杂的示例包括通过同时移动几个文件来同时运行命令。但是，无论您同时移动10个文件还是一次移动10个文件，想法都是相同的。

点赞
评论