Need Proxy?

BotProxy: Rotating Proxies Made for professionals. Really fast connection. Built-in IP rotation. Fresh IPs every day.

How to proxy HTTP requests in Spring MVC?

Question

I have an application built on top of Spring MVC.

I want to write simple proxy that processes requests as follows:

send the same HTTP request to some specific server
capture HTTP response from this specific server
return the same answer to requesting client

Here is what I've got so far:

public void proxyRequest(HttpServletRequest request, HttpServletResponse response) {
    try {
        HttpUriRequest proxiedRequest = createHttpUriRequest(request);
        HttpResponse proxiedResponse = httpClient.execute(proxiedRequest);
        writeToResponse(proxiedResponse, response);
    } catch (URISyntaxException e) {
        e.printStackTrace();
    } catch (ClientProtocolException e) {
        e.printStackTrace();
    } catch (IOException e) {
        e.printStackTrace();
    }
}

private void writeToResponse(HttpResponse proxiedResponse, HttpServletResponse response){
    for(Header header : proxiedResponse.getAllHeaders()){
        response.addHeader(header.getName(), header.getValue());
    }
    OutputStream os = null;
    InputStream is = null;
    try {
        is = proxiedResponse.getEntity().getContent();
        os = response.getOutputStream();
        IOUtils.copy(is, os);
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } finally {
        if (os != null) {
            try {
                os.close();
            } catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
        }
        if (is != null) {
            try {
                is.close();
            } catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
        }
    }
}

private HttpUriRequest createHttpUriRequest(HttpServletRequest request) throws URISyntaxException{
    URI uri = new URI(geoserverConfig.getUrl()+"/wms?"+request.getQueryString());

    RequestBuilder rb = RequestBuilder.create(request.getMethod());
    rb.setUri(uri);

    Enumeration<String> headerNames = request.getHeaderNames();
    while(headerNames.hasMoreElements()){
        String headerName = headerNames.nextElement();
        String headerValue = request.getHeader(headerName);
        rb.addHeader(headerName, headerValue);
    }

    HttpUriRequest proxiedRequest = rb.build();
    return proxiedRequest;
}

It works ok but not in all cases. I've checked in Chrome's network monitor and some of the requests that are using this proxy failed.

Here are headers of sample failed request response:

HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Disposition: inline; filename=JEDN_EWID.png
Transfer-Encoding: chunked
Date: Thu, 16 Jul 2015 10:31:49 GMT
Content-Type: image/png;charset=UTF-8
Content-Length: 6727

Here are headers of sample successfull request response:

HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Disposition: inline; filename=JEDN_EWID.png
Transfer-Encoding: chunked
Date: Thu, 16 Jul 2015 10:31:49 GMT
Content-Type: image/png;charset=UTF-8
Transfer-Encoding: chunked

What's more Chrome throws an error in the console:

GET http://localhost:8080/<rest of url> net::ERR_INVALID_CHUNKED_ENCODING

The requests that I'm proxying are WMS GetMap requests and my proxy is forwarding them to hidden Geoserver. I've noticed that failed requests should return transparent 512x512 .png images which are all empty. The successfull ones return 512x512 .png images that are not only transparent but also contain some colors.

Answer

It looks like remote server responds with chunked responses when size become too big, Apache HttpClient library gathers all chunked elements in one big HttpResponse, but leaves the Transfer-Encoding: chunked header.

I could not test, but you should filter out the Transfer-Encoding: chunked to get rid of this problem :

private void writeToResponse(HttpResponse proxiedResponse, HttpServletResponse response){
    for(Header header : proxiedResponse.getAllHeaders()){
        if ((! header.getName().equals("Transfer-Encoding")) || (! header.getValue().equals("chunked"))) {
            response.addHeader(header.getName(), header.getValue());
        }
    }
    ...

cc by-sa 3.0