29. 12. 2024 Andrea Mariani Log-SIEM, NetEye

How to Configure Kibana to Use a Proxy Server with a Certificate via the NODE_EXTRA_CA_CERTS Variable

When using Kibana in environments that require a proxy to reach external services, you might encounter issues with unrecognized SSL certificates. Specifically, if the proxy is exposed with its own certificate and acts as an SSL terminator, requests made by Kibana to external URLs can fail with HTTP status code errors. In this blog post, we’ll explore how to resolve this issue using the NODE_EXTRA_CA_CERTS environment variable.

The Problem

Consider a scenario where Kibana tries to access the following URL:

https://epr.elastic.co/search?package=elastic_agent&prerelease=false&kibana.version=8.10.2

If this request goes through a proxy using a custom SSL certificate, then without proper configuration you’ll encounter an error like:

Status code was 404 and not [200]: HTTP Error 404: Not Found

This error is not due to a real 404 Not Found issue but rather because Kibana doesn’t recognize the proxy’s certificate. As a result, the Node.js client built into Kibana fails to establish a secure connection.

The Solution: The `NODE_EXTRA_CA_CERTS` Variable

To solve this issue, you can use the NODE_EXTRA_CA_CERTS environment variable provided by Node.js. This variable allows you to specify a file containing additional CA certificates that Node.js should trust.

It’s important to note that Kibana doesn’t automatically use the system’s trusted certificate chain. Even if the proxy’s certificate is added to the operating system’s certificate store, Kibana will not recognize it unless explicitly instructed to by using the NODE_EXTRA_CA_CERTS variable.

Steps to Configure `NODE_EXTRA_CA_CERTS`

Obtain the Proxy Certificate
- Retrieve the proxy’s public certificate (for example, using the openssl command):
  - openssl s_client -showcerts -connect <proxy_host>:<proxy_port>
- Add the certificate into file: /etc/pki/tls/certs/ca-bundle.crt
Modify Kibana Configuration
- Add the NODE_EXTRA_CA_CERTS environment variable to the configuration file or the startup script for Kibana.
  - vim /neteye/shared/kibana/conf/sysconfig/kibana-user-customization
- Add the following line
  - NODE_EXTRA_CA_CERTS="/etc/pki/tls/certs/ca-bundle.crt"
Restart Kibana
- After updating the configuration, reload Systemd configuration files and restart Kibana:
  - sudo systemctl daemon-reload
  - sudo systemctl restart kibana-logmanager
Verify Functionality
- Check the Kibana logs to ensure no certificate-related errors or 404 status code errors were logged:
  - journalctl -fu kibana-logmanager

Why Use `NODE_EXTRA_CA_CERTS`?

Using this variable is especially useful when you want to avoid completely disabling SSL verification (e.g., with the NODE_TLS_REJECT_UNAUTHORIZED=0 option), which poses a security risk. By specifying only the necessary certificates, you ensure connections are secure while still validating certificates correctly.

Conclusion

Configuring Kibana to work with a proxy exposed via a custom certificate might seem complex, but the NODE_EXTRA_CA_CERTS variable simplifies the process significantly. By following the steps outlined above, you can ensure that Kibana securely makes external requests through your proxy.