为jq命令提供很大的参数以过滤键

我试图解析一个由JSON对象组成的非常大的文件,如下所示:

{"id": "100000002", "title": "some_title",  "year": 1988}

现在,我还有一个很大的ID列表,如果有的话,我想从文件中提取它们。

现在我知道我可以这样做:

jq '[ .[map(.id)|indices("1", "2")[]] ]' 0.txt > p0.json

这将产生我想要的结果,即仅用具有“ id” 1和“ 2”的对象填充p0.json。现在出现了问题:我的ID列表也很长(大约100k)。所以我有一个Python程序来输出相关的ID。我的想法是,首先将其分配给变量:

 REL_IDS=`echo python3 rel_ids.py`

然后执行:

jq --arg ids "$REL_IDS" '[ .[map(.id)|indices($ids)[]] ]' 0.txt > p0.json

I tried both with brackets [$ids] and without brackets, but no luck so far.

我的问题是,考虑到过滤器的大量参数,我将如何将其放入我的jq命令中?

在此先多谢!

评论
  • onam
    onam 回复

    Since the list of ids is long, the trick is NOT to use --arg. However, the details will depend on the details regarding the "long list of ids".

    In general, though, you'd want to present the list of ids to jq as a file so that you could use --rawfile or --slurpfile or some such.

    If for some reason you don't want to bother with an actual file, then provided your shell allows it, you could use these file-oriented options with process substitution: <( ... )