Jump Server with PAM
Last updated
Was this helpful?
Last updated
Was this helpful?
The organization required a jump server with privileged access management on the internal network. The server will not be accessible from outside the local network. This is a requirement from external stakeholders.
Initially, research was done on a variety of solutions, both free and open source as well as enterprise. We took into consideration the use cases:
Not many users would need this, only some members of the IT department to access a handful of servers, and the Finance department to access the SAP server.
It would be local only, nothing exposed to the internet.
Most users had their own credentials and would not require elevated privileges, meaning the PAM requirements for these servers were minimal and could be handled manually and case-by-case.
We ultimately decided on using . The rationale behind this decision was:
It can allow for a certain level of access management, meaning Admin credentials can be saved in Guacamole itself and thus not shared around.
It supports session recording, and logging
It supports TOTP.
It is accessible from a browser.
It supports the necessary protocols, ie., RDP and SSH.
It is free and open source.
Installing this container was relatively easy, as it also supported the installation of extensions through an environment variable. It was able to reach all of our servers as needed, with either stored credentials or credential passthrough, and 2FA as an extra layer of security.
By default, Guacamole does not have SSL configured. I opted to add a reverse proxy on the machine to enable this, as well as allow me to close the default ports. I decided to use Nginx Proxy Manager for this, due to its ease of use.
Since the server is not exposed to the internet, I could not get a valid Let's Encrypt certificate, and created a self-signed one on the server itself. Initially, I created a certificate with the Active Directory Certificate Authority, however this produced a singular PFX file, while Nginx Proxy Manager wanted separate key and certificate files. I then created the necessary PEM files using OpenSSL. I imported these certs into Nginx Proxy Manager. I created hosts for Guacamole, NPM and Portainer and applied the certificate to all of them.
Automatic updates have been created with a cron job for system updates, and a Watchtower container for container updates. Both run daily at nigh time.
I used UFW to disable all incoming ports except 443. Additionally, the sshd_config file was amended to disallow any SSH connections. If needed, we can connect directly to the jump server through the VSphere terminal.
The target servers, meaning the domain controllers, SAP server, and other sensitive servers were set to deny any incoming RDP and SSH requests except those coming from the PAM server. Other maintenance related ports on the target servers still remain open. The default port of 3389 was not changed as these servers are not exposed to the internet and are thus at much lower risk of automated scans and attacks.
Lastly, I disabled Portainer container as it was not necessary anymore. However, I did not delete it in case we did need it in the future.
After adding the SSL certs, I tested connectivity on multiple browsers before and after closing the ports.
I confirmed the traffic is encrypted with Wireshark.
An Nmap port scan was done to confirm the ports were indeed closed.
Users were able to use Guacamole to RDP into the domain controllers, and could not directly access the domain controllers through standard Remote Desktop.
Google Chrome does not trust self-signed certificates and always lists the connection as insecure. This might make some users wary, as they might not understand the concept of self-signed certificates.
RDP connections through Guacamole were slow, this was resolved by enabling Web Sockets in Nginx Proxy Manager.
Initially Nginx Proxy Manager was created with a directory bind for saving configs, however the container did not have appropriate permissions and did not save anything, meaning all the configs were lost when Watchtower updated it. I removed the bind and used a normal volume instead which resolved the issue.
To fix the above issue, I tried regaining access via SSH and was initially unable to, eventually got it working.
This was my first big production project. I learned a lot here relating to certificates and server hardening, namely:
How to generate self-signed certificates using AD CA and OpenSSL
The pros and cons of using self-signed certificates over trusted CA certificate
Configuring automatic updates for both Linux and Docker
SSH hardening
Packet sniffing with Wireshark
While I did encounter problems, none of them were insurmountable.
I opted to install the service via docker, because we can easily recreate, edit or remove instances as needed. First, I installed , as it makes installing and managing docker containers easy. There were various solutions to this. The official guide had three separate containers; the Guacamole instance, the Guacd instance and the Postgres instance. I initially tried this method, however I had difficulty setting Postgres up correctly. I then opted for an "all-in-one" container, specifically the one created by as it had a 500k pulls and looked to be frequently updated. I then installed to act as a reverse proxy and also serve the Guacamole instance through SSL, and lastly I installed to enable automatic updates to all containers.
All setup steps were documented, along with troubleshooting tips, for future reference and stored in the IT documentation folder. I also updated in our NIST profile.