Published on Thu Feb 20 2025 14:30:00 GMT+0000 (Coordinated Universal Time) by Purusothaman Ramanujam

Working with HTTP Requests Using Apache HttpClient

Introduction

In modern Java applications, you often need to make HTTP requests to external APIs, web services, or other applications. While Java provides built-in HTTP client capabilities, Apache HttpClient offers a more feature-rich and flexible solution for handling HTTP communications.

Apache HttpClient is a robust, feature-complete HTTP client library that supports HTTP/1.1 and HTTP/2, connection pooling, authentication, and many other advanced features.

What is Apache HttpClient?

Apache HttpClient is a powerful HTTP client library that provides:

HTTP/1.1 and HTTP/2 support
Connection pooling for better performance
Authentication (Basic, Digest, NTLM, SPNEGO)
Cookie management
Request/Response interceptors
SSL/TLS support
Proxy support
Automatic redirect handling

Adding Apache HttpClient to Your Project

Maven Dependencies

<dependency>
    <groupId>org.apache.httpcomponents.client5</groupId>
    <artifactId>httpclient5</artifactId>
    <version>5.2.1</version>
</dependency>

Gradle Dependencies

implementation 'org.apache.httpcomponents.client5:httpclient5:5.2.1'

Basic HTTP Requests

Simple GET Request

Let’s start with a simple GET request:

import org.apache.hc.client5.http.classic.methods.HttpGet;
import org.apache.hc.client5.http.impl.classic.CloseableHttpClient;
import org.apache.hc.client5.http.impl.classic.HttpClients;
import org.apache.hc.core5.http.io.entity.EntityUtils;

public class BasicHttpClientExample {
    public static void main(String[] args) {
        try (CloseableHttpClient httpClient = HttpClients.createDefault()) {
            // Create HTTP GET request
            HttpGet httpGet = new HttpGet("https://jsonplaceholder.typicode.com/posts/1");

            // Execute the request
            String response = httpClient.execute(httpGet, response -> {
                return EntityUtils.toString(response.getEntity());
            });

            System.out.println("Response: " + response);

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

POST Request with JSON

Here’s how to make a POST request with JSON data:

import org.apache.hc.client5.http.classic.methods.HttpPost;
import org.apache.hc.core5.http.ContentType;
import org.apache.hc.core5.http.io.entity.StringEntity;

public class PostRequestExample {
    public static void main(String[] args) {
        try (CloseableHttpClient httpClient = HttpClients.createDefault()) {
            // Create HTTP POST request
            HttpPost httpPost = new HttpPost("https://jsonplaceholder.typicode.com/posts");

            // Create JSON payload
            String jsonPayload = "{\"title\":\"Test Post\",\"body\":\"This is a test post\",\"userId\":1}";
            StringEntity entity = new StringEntity(jsonPayload, ContentType.APPLICATION_JSON);
            httpPost.setEntity(entity);

            // Execute the request
            String response = httpClient.execute(httpPost, response -> {
                return EntityUtils.toString(response.getEntity());
            });

            System.out.println("Response: " + response);

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Advanced Configuration

Connection Pooling

HttpClient supports connection pooling for better performance:

import org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager;
import org.apache.hc.core5.util.Timeout;

public class ConnectionPoolingExample {
    public static void main(String[] args) {
        // Create connection manager with pooling
        PoolingHttpClientConnectionManager connectionManager =
            PoolingHttpClientConnectionManagerBuilder.create()
                .setMaxTotal(100) // Maximum total connections
                .setDefaultMaxPerRoute(20) // Maximum connections per route
                .build();

        // Create HTTP client with connection manager
        try (CloseableHttpClient httpClient = HttpClients.custom()
                .setConnectionManager(connectionManager)
                .build()) {

            HttpGet httpGet = new HttpGet("https://api.example.com/data");

            String response = httpClient.execute(httpGet, response -> {
                return EntityUtils.toString(response.getEntity());
            });

            System.out.println("Response: " + response);

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Request Configuration

You can configure various request parameters:

import org.apache.hc.client5.http.config.RequestConfig;
import org.apache.hc.core5.util.Timeout;

public class RequestConfigExample {
    public static void main(String[] args) {
        // Create request configuration
        RequestConfig requestConfig = RequestConfig.custom()
            .setConnectTimeout(Timeout.ofSeconds(10))
            .setResponseTimeout(Timeout.ofSeconds(30))
            .build();

        try (CloseableHttpClient httpClient = HttpClients.custom()
                .setDefaultRequestConfig(requestConfig)
                .build()) {

            HttpGet httpGet = new HttpGet("https://api.example.com/data");

            String response = httpClient.execute(httpGet, response -> {
                return EntityUtils.toString(response.getEntity());
            });

            System.out.println("Response: " + response);

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Authentication

Basic Authentication

import org.apache.hc.client5.http.auth.UsernamePasswordCredentials;
import org.apache.hc.client5.http.impl.auth.BasicCredentialsProvider;
import org.apache.hc.core5.http.HttpHost;

public class BasicAuthExample {
    public static void main(String[] args) {
        // Create credentials provider
        BasicCredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(
            new AuthScope("api.example.com", 443),
            new UsernamePasswordCredentials("username", "password".toCharArray())
        );

        try (CloseableHttpClient httpClient = HttpClients.custom()
                .setDefaultCredentialsProvider(credentialsProvider)
                .build()) {

            HttpGet httpGet = new HttpGet("https://api.example.com/protected");

            String response = httpClient.execute(httpGet, response -> {
                return EntityUtils.toString(response.getEntity());
            });

            System.out.println("Response: " + response);

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Working with Different Content Types

Form Data

import org.apache.hc.client5.http.classic.methods.HttpPost;
import org.apache.hc.core5.http.io.entity.UrlEncodedFormEntity;
import org.apache.hc.core5.http.message.BasicNameValuePair;
import java.util.Arrays;

public class FormDataExample {
    public static void main(String[] args) {
        try (CloseableHttpClient httpClient = HttpClients.createDefault()) {
            HttpPost httpPost = new HttpPost("https://httpbin.org/post");

            // Create form data
            List<BasicNameValuePair> formData = Arrays.asList(
                new BasicNameValuePair("name", "John Doe"),
                new BasicNameValuePair("email", "john@example.com"),
                new BasicNameValuePair("message", "Hello World")
            );

            httpPost.setEntity(new UrlEncodedFormEntity(formData));

            String response = httpClient.execute(httpPost, response -> {
                return EntityUtils.toString(response.getEntity());
            });

            System.out.println("Response: " + response);

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

File Upload

import org.apache.hc.client5.http.classic.methods.HttpPost;
import org.apache.hc.core5.http.ContentType;
import org.apache.hc.core5.http.io.entity.FileEntity;
import java.io.File;

public class FileUploadExample {
    public static void main(String[] args) {
        try (CloseableHttpClient httpClient = HttpClients.createDefault()) {
            HttpPost httpPost = new HttpPost("https://httpbin.org/post");

            // Upload a file
            File file = new File("example.txt");
            FileEntity fileEntity = new FileEntity(file, ContentType.TEXT_PLAIN);
            httpPost.setEntity(fileEntity);

            String response = httpClient.execute(httpPost, response -> {
                return EntityUtils.toString(response.getEntity());
            });

            System.out.println("Response: " + response);

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Error Handling

Custom Response Handler

import org.apache.hc.client5.http.classic.methods.HttpGet;
import org.apache.hc.core5.http.ClassicHttpResponse;
import org.apache.hc.core5.http.HttpStatus;

public class CustomResponseHandlerExample {
    public static void main(String[] args) {
        try (CloseableHttpClient httpClient = HttpClients.createDefault()) {
            HttpGet httpGet = new HttpGet("https://api.example.com/data");

            String response = httpClient.execute(httpGet, (ClassicHttpResponse httpResponse) -> {
                int statusCode = httpResponse.getCode();

                if (statusCode == HttpStatus.SC_OK) {
                    return EntityUtils.toString(httpResponse.getEntity());
                } else if (statusCode == HttpStatus.SC_NOT_FOUND) {
                    throw new RuntimeException("Resource not found");
                } else if (statusCode >= 500) {
                    throw new RuntimeException("Server error: " + statusCode);
                } else {
                    throw new RuntimeException("Unexpected response: " + statusCode);
                }
            });

            System.out.println("Response: " + response);

        } catch (Exception e) {
            System.err.println("Error: " + e.getMessage());
        }
    }
}

SSL/TLS Configuration

Custom SSL Context

import org.apache.hc.core5.ssl.SSLContextBuilder;
import javax.net.ssl.SSLContext;

public class SSLExample {
    public static void main(String[] args) {
        try {
            // Create custom SSL context
            SSLContext sslContext = SSLContextBuilder.create()
                .loadTrustMaterial(null, (chain, authType) -> true) // Trust all certificates (for testing)
                .build();

            try (CloseableHttpClient httpClient = HttpClients.custom()
                    .setSSLContext(sslContext)
                    .build()) {

                HttpGet httpGet = new HttpGet("https://api.example.com/data");

                String response = httpClient.execute(httpGet, response -> {
                    return EntityUtils.toString(response.getEntity());
                });

                System.out.println("Response: " + response);
            }

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Proxy Support

import org.apache.hc.core5.http.HttpHost;

public class ProxyExample {
    public static void main(String[] args) {
        // Create proxy configuration
        HttpHost proxy = new HttpHost("proxy.example.com", 8080);

        try (CloseableHttpClient httpClient = HttpClients.custom()
                .setProxy(proxy)
                .build()) {

            HttpGet httpGet = new HttpGet("https://api.example.com/data");

            String response = httpClient.execute(httpGet, response -> {
                return EntityUtils.toString(response.getEntity());
            });

            System.out.println("Response: " + response);

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Best Practices

Use Connection Pooling: Always use connection pooling for better performance in production applications.
Handle Resources Properly: Use try-with-resources to ensure HttpClient is properly closed.
Set Timeouts: Always configure appropriate connection and response timeouts.
Handle Errors Gracefully: Implement proper error handling for different HTTP status codes.
Use HTTPS: Always use HTTPS for sensitive data transmission.
Validate Responses: Always validate response status codes and content.
Reuse HttpClient: Create one HttpClient instance and reuse it for multiple requests.

Common Pitfalls

Not Closing HttpClient: Always close HttpClient to free up resources.
Ignoring Status Codes: Don’t assume all responses are successful; check status codes.
Not Setting Timeouts: Without timeouts, requests can hang indefinitely.
Memory Leaks: Be careful with large response entities; always consume them.
Thread Safety: HttpClient is thread-safe, but individual HttpRequest objects are not.

Conclusion

Apache HttpClient is a powerful and flexible HTTP client library that provides everything you need for HTTP communication in Java applications. From simple GET requests to complex scenarios with authentication, SSL, and connection pooling, HttpClient has you covered.

Start with basic requests and gradually explore advanced features like connection pooling, authentication, and custom configurations as your needs grow. Remember to follow best practices for resource management and error handling.