i'm trying to find a way to extract Trailer headers as defined by
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Trailer and RFC7230
From a Web API which provides a streams of json data from unknown amount (1k ...10M JSON records) in chunked transport encoding and also some additional trailer headers at the end of the stream.
to allow processing in a stream fashion, we use spring-webflux , version 5.2.9, WebClient but i'm unable to find a way to retrieve this trailer headers from the webclient.
current test code i use goes like this :
Iterator<JsonNode> x = client
.get()
.uri("http://localhost:8080/api/v1/sources/chunked")
.header(HttpHeaders.TRAILER, "x-stream-records")
.accept(MediaType.APPLICATION_STREAM_JSON)
.exchange()
.map(clientResponse -> {
logger.info("{}",clientResponse.statusCode());
logger.info("{}",clientResponse.headers().asHttpHeaders());
return clientResponse.bodyToFlux(JsonNode.class);
})
.block(Duration.ofSeconds(400)).toIterable().iterator();
long cnt=0;
while (x.hasNext()) {
logger.info("{}",x.next());
cnt++;
}
logger.info("done: {}",cnt);
the goal is to allow access to the trailer x-stream-records= which is set by the api, so we can verify the amount of data send/retrieved.
replacing bodyToFlux(JsonNode.class) with bodyToFlux(String.class) will show 1 EMPTY string record extra. so i suspect the content is somehow is there but not "handled" at this moment.
reviewing the BodyExtractors doesn't seem to include any extractor which would allow retrieving these trailer headers.
so how can we retrieve this... ?
